cant download microsoft cert file

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

cant download microsoft cert file

robert k Wild
hi all,

i have squid installed and its awesome i can whitelist and block mime types but when im trying to activate my office 365 going through the proxy i get a http denied on this

TCP_DENIED/403 3661 GET http://www.microsoft.com/pki/certs/MicRooCerAut2011_2011_03_22.crt 

but i have whitelisted the domain ".microsoft.com" so i really dont understand why this is being denied as all the other domains i have whitelisted are fone ie there not being denied

thanks,

rob

--
Regards,

Robert K Wild.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Matus UHLAR - fantomas
On 14.12.19 01:46, robert k Wild wrote:

>i have squid installed and its awesome i can whitelist and block mime types
>but when im trying to activate my office 365 going through the proxy i get
>a http denied on this
>
>TCP_DENIED/403 3661 GET
>http://www.microsoft.com/pki/certs/MicRooCerAut2011_2011_03_22.crt
>
>but i have whitelisted the domain ".microsoft.com" so i really dont
>understand why this is being denied as all the other domains i have
>whitelisted are fone ie there not being denied

looks like you whitelisted incorrectly. Maybe showing the http_access lines
with related acls could help us to find the issue...

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Save the whales. Collect the whole set.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild

so this is my config file -

#
# Recommended minimum configuration:
#

#SSL
http_port 3128 ssl-bump \
cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8   # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10   # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16   # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12   # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16   # RFC 1918 local private network (LAN)
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10       # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80   # http
acl Safe_ports port 21   # ftp
acl Safe_ports port 443   # https
acl Safe_ports port 70   # gopher
acl Safe_ports port 210   # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280   # http-mgmt
acl Safe_ports port 488   # gss-http
acl Safe_ports port 591   # filemaker
acl Safe_ports port 777   # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# Squid normally listens to port 3128
http_port 3128

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:   1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern .   0 20% 4320

as you can see i have removed the whitelist/mime config lines

but when i come into activating office it just cant get online to do it via the client app installed on my pc

but internet isnt blocked as i can go to any website


On Sat, 14 Dec 2019 at 08:39, Matus UHLAR - fantomas <[hidden email]> wrote:
On 14.12.19 01:46, robert k Wild wrote:
>i have squid installed and its awesome i can whitelist and block mime types
>but when im trying to activate my office 365 going through the proxy i get
>a http denied on this
>
>TCP_DENIED/403 3661 GET
>http://www.microsoft.com/pki/certs/MicRooCerAut2011_2011_03_22.crt
>
>but i have whitelisted the domain ".microsoft.com" so i really dont
>understand why this is being denied as all the other domains i have
>whitelisted are fone ie there not being denied

looks like you whitelisted incorrectly. Maybe showing the http_access lines
with related acls could help us to find the issue...

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Save the whales. Collect the whole set.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Regards,

Robert K Wild.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Amos Jeffries
Administrator
On 15/12/19 4:21 am, robert k Wild wrote:

> so this is my config file -
>
> #
> # Recommended minimum configuration:
> #
>
> #SSL
> http_port 3128 ssl-bump \
> cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB

> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>

(elided default localnet and port ACL definitions)

>
> #
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
>
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
>
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #

  ^^^ HINT.

>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
>
> # Squid normally listens to port 3128
> http_port 3128
>

This is the second port 3128 config, and it does not match the earlier one.


>
> as you can see i have removed the whitelist/mime config lines
>
> but when i come into activating office it just cant get online to do it
> via the client app installed on my pc

If that is still happening with this default config I would be starting
to suspect things outside of Squid. Like firewall or routing rules, the
client app not supporting proxies properly - stuff like that.

Though 403 in the proxy log does indicate an explicitly forbidden
action. The way you truncated the log line cut away most of the useful
info that points at where to focus the troubleshooting efforts.


>
> but internet isnt blocked as i can go to any website
>

Do you want it to work with the whitelisting ACL you mentioned?

If yes, then you do need to show at least the http_access directives
using it and the exact entry you added for the microsoft.com domain(s).

Same for the "mime" config lines you mention, but for those any part of
it could be relevant so we will need to see the whole of that stuff.


You have omitted the default "http_access deny all" which should be the
last http_access line in your config. Not a problem in the config as
shown, but if you have other rules they can change the implicit default
into a bad situation very easily.



Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
hi Amos,

thank you for getting back to me about this :)

this is my new config

#
#SSL
http_port 3128 ssl-bump \
cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

#Windows Updates
acl windowsupdate dstdomain "/usr/local/squid/etc/wu.txt"
acl CONNECT method CONNECT
acl wuCONNECT dstdomain "/usr/local/squid/etc/wu.txt"
http_access allow CONNECT wuCONNECT
http_access allow windowsupdate

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10       # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

the reason why i have added the windows update lines at the beginning is that the link says so (below)


this is my domain list


and when im looking at the logs real time

1576368417.620     48 10.100.1.5 NONE/200 0 CONNECT fe3cr.delivery.mp.microsoft.com:443 - HIER_DIRECT/191.232.139.2 -
1576368417.647      0 10.100.1.5 NONE/503 4363 POST https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx - HIER_NONE/- text/html
1576368419.702      0 - TCP_MEM_HIT/200 807 GET http://www.microsoft.com/pkiops/certs/Microsoft%20ECC%20Product%20Root%20Certificate%20Authority%202018.crt - HIER_NONE/- application/octet-st
ream

squid works fine just as you said on certain apps/programs, so im really struggling on this one

thanks,
rob

On Sat, 14 Dec 2019 at 22:35, Amos Jeffries <[hidden email]> wrote:
On 15/12/19 4:21 am, robert k Wild wrote:
> so this is my config file -
>
> #
> # Recommended minimum configuration:
> #
>
> #SSL
> http_port 3128 ssl-bump \
> cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB

> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>

(elided default localnet and port ACL definitions)

>
> #
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
>
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
>
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #

  ^^^ HINT.

>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
>
> # Squid normally listens to port 3128
> http_port 3128
>

This is the second port 3128 config, and it does not match the earlier one.


>
> as you can see i have removed the whitelist/mime config lines
>
> but when i come into activating office it just cant get online to do it
> via the client app installed on my pc

If that is still happening with this default config I would be starting
to suspect things outside of Squid. Like firewall or routing rules, the
client app not supporting proxies properly - stuff like that.

Though 403 in the proxy log does indicate an explicitly forbidden
action. The way you truncated the log line cut away most of the useful
info that points at where to focus the troubleshooting efforts.


>
> but internet isnt blocked as i can go to any website
>

Do you want it to work with the whitelisting ACL you mentioned?

If yes, then you do need to show at least the http_access directives
using it and the exact entry you added for the microsoft.com domain(s).

Same for the "mime" config lines you mention, but for those any part of
it could be relevant so we will need to see the whole of that stuff.


You have omitted the default "http_access deny all" which should be the
last http_access line in your config. Not a problem in the config as
shown, but if you have other rules they can change the implicit default
into a bad situation very easily.



Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Regards,

Robert K Wild.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Amos Jeffries
Administrator
On 15/12/19 1:16 pm, robert k Wild wrote:

> hi Amos,
>
> thank you for getting back to me about this :)
>
> this is my new config
>
> #
> #SSL
> http_port 3128 ssl-bump \
> cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB
> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>
> #Windows Updates
> acl windowsupdate dstdomain "/usr/local/squid/etc/wu.txt"
> acl CONNECT method CONNECT
> acl wuCONNECT dstdomain "/usr/local/squid/etc/wu.txt"
> http_access allow CONNECT wuCONNECT
> http_access allow windowsupdate
>
...

>
> #
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
>
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
>
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #
>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
>
...

>
> the reason why i have added the windows update lines at the beginning is
> that the link says so (below)
>
> https://linuxnlenux.wordpress.com/2014/10/14/howto-allow-windows-updates-through-squid/
>

That is a copy-n-paste of an old email without any of the context. See
<https://wiki.squid-cache.org/SquidFaq/WindowsUpdate> for the full
context and more up to date info.

Note that the things that need to be first are very specifically a
sub-set of the MS domains which use a non-443 port for call-home traffic
so they would normally get blocked by the SSL_ports protection.


For a generic whitelist you should still have your list where the config
says "INSERT YOUR OWN RULES ..." .


>
> and when im looking at the logs real time
>
> 1576368417.620     48 10.100.1.5 NONE/200 0 CONNECT
> fe3cr.delivery.mp.microsoft.com:443
> <http://fe3cr.delivery.mp.microsoft.com:443> - HIER_DIRECT/191.232.139.2
> <http://191.232.139.2> -
> 1576368417.647      0 10.100.1.5 NONE/503 4363 POST
> https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx -
> HIER_NONE/- text/html
> 1576368419.702      0 - TCP_MEM_HIT/200 807 GET
> http://www.microsoft.com/pkiops/certs/Microsoft%20ECC%20Product%20Root%20Certificate%20Authority%202018.crt
> - HIER_NONE/- application/octet-st
> ream
>

These show good progress from where you started off. The cert is being
downloaded fine. The tunnel being bumped fine. But the POST request
which was decrypted could not be serviced.

Can you find out what the 503 message says?


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
hi Amos,

so this is my new config -

#
# Recommended minimum configuration:
#

#SSL
http_port 3128 ssl-bump \
cert=/usr/local/squid/ssl_cert/myCA.pem \
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10       # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
#Windows Update
acl windowsupdate dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
acl CONNECT method CONNECT
acl wuCONNECT dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
http_access allow CONNECT wuCONNECT
http_access allow windowsupdate

acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex -i .microsoft.com .windows.com .windowsupdate.com .windows.net
ssl_bump splice NoSSLIntercept
ssl_bump peek DiscoverSNIHost
ssl_bump bump all

acl BrokenButTrustedServers dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

#HTTP_HTTPS whitelist websites
acl whitelist ssl::server_name "/usr/local/squid/etc/urlwhite.txt"
http_access allow whitelist

#URL deny MIME types
acl mimetype rep_mime_type "/usr/local/squid/etc/mimedeny.txt"
http_reply_access deny mimetype
http_access deny all

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

but im still getting the exact same logs

error 503 means

503

Service Unavailable

1945, 2616

thanks,
rob

On Sun, 15 Dec 2019 at 10:40, Amos Jeffries <[hidden email]> wrote:
On 15/12/19 1:16 pm, robert k Wild wrote:
> hi Amos,
>
> thank you for getting back to me about this :)
>
> this is my new config
>
> #
> #SSL
> http_port 3128 ssl-bump \
> cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB
> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>
> #Windows Updates
> acl windowsupdate dstdomain "/usr/local/squid/etc/wu.txt"
> acl CONNECT method CONNECT
> acl wuCONNECT dstdomain "/usr/local/squid/etc/wu.txt"
> http_access allow CONNECT wuCONNECT
> http_access allow windowsupdate
>
...
>
> #
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
>
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
>
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #
>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
>
...

>
> the reason why i have added the windows update lines at the beginning is
> that the link says so (below)
>
> https://linuxnlenux.wordpress.com/2014/10/14/howto-allow-windows-updates-through-squid/
>

That is a copy-n-paste of an old email without any of the context. See
<https://wiki.squid-cache.org/SquidFaq/WindowsUpdate> for the full
context and more up to date info.

Note that the things that need to be first are very specifically a
sub-set of the MS domains which use a non-443 port for call-home traffic
so they would normally get blocked by the SSL_ports protection.


For a generic whitelist you should still have your list where the config
says "INSERT YOUR OWN RULES ..." .


>
> and when im looking at the logs real time
>
> 1576368417.620     48 10.100.1.5 NONE/200 0 CONNECT
> fe3cr.delivery.mp.microsoft.com:443
> <http://fe3cr.delivery.mp.microsoft.com:443> - HIER_DIRECT/191.232.139.2
> <http://191.232.139.2> -
> 1576368417.647      0 10.100.1.5 NONE/503 4363 POST
> https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx -
> HIER_NONE/- text/html
> 1576368419.702      0 - TCP_MEM_HIT/200 807 GET
> http://www.microsoft.com/pkiops/certs/Microsoft%20ECC%20Product%20Root%20Certificate%20Authority%202018.crt
> - HIER_NONE/- application/octet-st
> ream
>

These show good progress from where you started off. The cert is being
downloaded fine. The tunnel being bumped fine. But the POST request
which was decrypted could not be serviced.

Can you find out what the 503 message says?


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Regards,

Robert K Wild.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
i have done it

i can now whitelist urls, block mime types and now i can download/install windows updates

#
#
#Windows Update Download
acl windowsupdate dstdomain .microsoft.com .windows.com .windowsupdate.com
acl CONNECT method CONNECT
acl wuCONNECT dstdomain .microsoft.com .windows.com .windowsupdate.com
http_access allow CONNECT wuCONNECT
http_access allow windowsupdate

range_offset_limit 200 MB windowsupdate
maximum_object_size 200 MB
quick_abort_min -1

refresh_pattern -i .<a href="http://microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)">microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i .<a href="http://windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)">windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i .<a href="http://windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)">windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex -i .microsoft.com .windows.com .windowsupdate.com
ssl_bump splice NoSSLIntercept
ssl_bump peek DiscoverSNIHost
ssl_bump bump all

acl BrokenButTrustedServers dstdomain .microsoft.com .windows.com .windowsupdate.com
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all
#
#SSL
http_port 3128 ssl-bump \
cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10       # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
#HTTP_HTTPS whitelist websites
acl whitelist ssl::server_name "/usr/local/squid/etc/urlwhite.txt"
http_access allow whitelist

#URL deny MIME types
acl mimetype rep_mime_type "/usr/local/squid/etc/mimedeny.txt"
http_reply_access deny mimetype
http_access deny all

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

Amos, do you think i could make the windows update section a bit smaller or do i need all the lines in there?

many thanks,
rob

On Sun, 15 Dec 2019 at 16:24, robert k Wild <[hidden email]> wrote:
hi Amos,

so this is my new config -

#
# Recommended minimum configuration:
#

#SSL
http_port 3128 ssl-bump \
cert=/usr/local/squid/ssl_cert/myCA.pem \
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10       # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
#Windows Update
acl windowsupdate dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
acl CONNECT method CONNECT
acl wuCONNECT dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
http_access allow CONNECT wuCONNECT
http_access allow windowsupdate

acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex -i .microsoft.com .windows.com .windowsupdate.com .windows.net
ssl_bump splice NoSSLIntercept
ssl_bump peek DiscoverSNIHost
ssl_bump bump all

acl BrokenButTrustedServers dstdomain .microsoft.com .windows.com .windowsupdate.com .windows.net
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all

#HTTP_HTTPS whitelist websites
acl whitelist ssl::server_name "/usr/local/squid/etc/urlwhite.txt"
http_access allow whitelist

#URL deny MIME types
acl mimetype rep_mime_type "/usr/local/squid/etc/mimedeny.txt"
http_reply_access deny mimetype
http_access deny all

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

but im still getting the exact same logs

error 503 means

503

Service Unavailable

1945, 2616

thanks,
rob

On Sun, 15 Dec 2019 at 10:40, Amos Jeffries <[hidden email]> wrote:
On 15/12/19 1:16 pm, robert k Wild wrote:
> hi Amos,
>
> thank you for getting back to me about this :)
>
> this is my new config
>
> #
> #SSL
> http_port 3128 ssl-bump \
> cert=/usr/local/squid/etc/ssl_cert/myCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /var/lib/ssl_db -M 4MB
> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>
> #Windows Updates
> acl windowsupdate dstdomain "/usr/local/squid/etc/wu.txt"
> acl CONNECT method CONNECT
> acl wuCONNECT dstdomain "/usr/local/squid/etc/wu.txt"
> http_access allow CONNECT wuCONNECT
> http_access allow windowsupdate
>
...
>
> #
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
>
> # We strongly recommend the following be uncommented to protect innocent
> # web applications running on the proxy server who think the only
> # one who can access services on "localhost" is a local user
> #http_access deny to_localhost
>
> #
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #
>
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
>
...

>
> the reason why i have added the windows update lines at the beginning is
> that the link says so (below)
>
> https://linuxnlenux.wordpress.com/2014/10/14/howto-allow-windows-updates-through-squid/
>

That is a copy-n-paste of an old email without any of the context. See
<https://wiki.squid-cache.org/SquidFaq/WindowsUpdate> for the full
context and more up to date info.

Note that the things that need to be first are very specifically a
sub-set of the MS domains which use a non-443 port for call-home traffic
so they would normally get blocked by the SSL_ports protection.


For a generic whitelist you should still have your list where the config
says "INSERT YOUR OWN RULES ..." .


>
> and when im looking at the logs real time
>
> 1576368417.620     48 10.100.1.5 NONE/200 0 CONNECT
> fe3cr.delivery.mp.microsoft.com:443
> <http://fe3cr.delivery.mp.microsoft.com:443> - HIER_DIRECT/191.232.139.2
> <http://191.232.139.2> -
> 1576368417.647      0 10.100.1.5 NONE/503 4363 POST
> https://fe3cr.delivery.mp.microsoft.com/ClientWebService/client.asmx -
> HIER_NONE/- text/html
> 1576368419.702      0 - TCP_MEM_HIT/200 807 GET
> http://www.microsoft.com/pkiops/certs/Microsoft%20ECC%20Product%20Root%20Certificate%20Authority%202018.crt
> - HIER_NONE/- application/octet-st
> ream
>

These show good progress from where you started off. The cert is being
downloaded fine. The tunnel being bumped fine. But the POST request
which was decrypted could not be serviced.

Can you find out what the 503 message says?


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Regards,

Robert K Wild.


--
Regards,

Robert K Wild.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Amos Jeffries
Administrator
On 16/12/19 10:26 am, robert k Wild wrote:
> i have done it
>
> i can now whitelist urls, block mime types and now i can
> download/install windows updates
>

Excellent.

>
> Amos, do you think i could make the windows update section a bit smaller
> or do i need all the lines in there?
>

Depends on what Windows OS versions and MS software you are supporting
on the network. The list we have covers Win2k and later and some older
Office products that had different domains. If you want to, you can scan
your logs for a few months and see which are actually needed.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
Thanks for that Amos,

The line below, do you think I could add multiple domains to it, ie


Thanks, 
Rob

On Mon, 16 Dec 2019, 04:57 Amos Jeffries, <[hidden email]> wrote:
On 16/12/19 10:26 am, robert k Wild wrote:
> i have done it
>
> i can now whitelist urls, block mime types and now i can
> download/install windows updates
>

Excellent.

>
> Amos, do you think i could make the windows update section a bit smaller
> or do i need all the lines in there?
>

Depends on what Windows OS versions and MS software you are supporting
on the network. The list we have covers Win2k and later and some older
Office products that had different domains. If you want to, you can scan
your logs for a few months and see which are actually needed.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Amos Jeffries
Administrator
On 16/12/19 8:44 pm, robert k Wild wrote:
> Thanks for that Amos,
>
> The line below, do you think I could add multiple domains to it, ie
>
> refresh_pattern -i .microsoft.com .windows.com
> .windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)
> <http://microsoft.com/.*%5C.%28cab%7Cexe%7Cms%5Bi%7Cu%7Cf%5D%7C%5Bap%5Dsf%7Cwm%5Bv%7Ca%5D%7Cdat%7Czip%29> 4320
> 80% 43200 reload-into-ims
>

The parameter is a single regex pattern. You can make a pattern that
matches multiple domains.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
How can I make a pattern that matches multiple domains please Amos? 

On Mon, 16 Dec 2019, 08:01 Amos Jeffries, <[hidden email]> wrote:
On 16/12/19 8:44 pm, robert k Wild wrote:
> Thanks for that Amos,
>
> The line below, do you think I could add multiple domains to it, ie
>
> refresh_pattern -i .microsoft.com .windows.com
> .windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)
> <http://microsoft.com/.*%5C.%28cab%7Cexe%7Cms%5Bi%7Cu%7Cf%5D%7C%5Bap%5Dsf%7Cwm%5Bv%7Ca%5D%7Cdat%7Czip%29> 4320
> 80% 43200 reload-into-ims
>

The parameter is a single regex pattern. You can make a pattern that
matches multiple domains.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Alex Crow
On 16/12/2019 08:06, robert k Wild wrote:
How can I make a pattern that matches multiple domains please Amos? 


That's not really a subject for this list - search online for "regex" and you will see multiple tutorials about it.

You use a syntax like "(.microsoft.com|.windows.com|(.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)|foo.com)"

eg (x|y|z(a|b)) would match x, y, za and zb.

Cheers

Alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
Would this work aswell

refresh_pattern -i /etc/squid/wu.txt/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

And in wu.txt


Exactly like my dstdomain

On Mon, 16 Dec 2019, 08:37 Alex Crow, <[hidden email]> wrote:
On 16/12/2019 08:06, robert k Wild wrote:
How can I make a pattern that matches multiple domains please Amos? 


That's not really a subject for this list - search online for "regex" and you will see multiple tutorials about it.

You use a syntax like "(.microsoft.com|.windows.com|(.windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip)|foo.com)"

eg (x|y|z(a|b)) would match x, y, za and zb.

Cheers

Alex

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

Alex Crow


On 16/12/2019 09:10, robert k Wild wrote:
Would this work aswell

refresh_pattern -i /etc/squid/wu.txt/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

And in wu.txt


Exactly like my dstdomain



No, because /etc/squid/wu.txt would be taken literally as part of the URL. And I don't think filenames are supported by that directive anyway.

Alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: cant download microsoft cert file

robert k Wild
OK thanks Alex

Thanks guys for all your help really much appreciated, thanks so much

Rob

On Mon, 16 Dec 2019, 09:39 Alex Crow, <[hidden email]> wrote:


On 16/12/2019 09:10, robert k Wild wrote:
Would this work aswell

refresh_pattern -i /etc/squid/wu.txt/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

And in wu.txt


Exactly like my dstdomain



No, because /etc/squid/wu.txt would be taken literally as part of the URL. And I don't think filenames are supported by that directive anyway.

Alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users