why squid can block https when i point my browser to port , and cant when its transparent ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

why squid can block https when i point my browser to port , and cant when its transparent ?

Dr.x
hi all ,

i have 2 questions.


1- why when i make a normal squid with normal http port , and i direct my browser to ip/port it can block https facebook


while
if it was transparent proxy it cant block https facebook ??

im talking about im configuraing normal http proxy not https !

wish a clarification.


2-now if i use ssl pump and used transparent tproxy with https ... can i buy a trusted certificate and install it on squid and the users will not face "certificate not trusted" message ?


i mean , in production network with much users , i need to block https  youtube/facebook while keep using  transparent tproxy.



with to help

regards
Dr.x
Reply | Threaded
Open this post in threaded view
|

Re: why squid can block https when i point my browser to port , and cant when its transparent ?

Alex Crow

On 27/07/14 16:00, Dr.x wrote:
> hi all ,
>
> i have 2 questions.
>
>
> 1- why when i make a normal squid with normal http port , and i direct my
> browser to ip/port it can block https facebook

Because the browser is aware of the cache and issues CONNECT requests
for SSL sites. Squid can see these and block them,

>
>
> while
> if it was transparent proxy it cant block https facebook ??

You can't use CONNECT with a transparent proxy as it implies the client
has been configured with a proxy (which would not be transparent).

>
> im talking about im configuraing normal http proxy not https !
>
> wish a clarification.
>
>
> 2-now if i use ssl pump and used transparent tproxy with https ... can i buy
> a trusted certificate and install it on squid and the users will not face
> "certificate not trusted" message ?

NO! This is about the 3rd or 4th time this question has appeared on this
list. You can't use a cert from a commercial provider because you need
the cert's private key to produce new certs signed by it (which the cert
provider will not give you in a million years). If this worked it would
make SSL useless.

>
>
> i mean , in production network with much users , i need to block https
> youtube/facebook while keep using  transparent tproxy.
>

You need to create your own CA, import the CA cert into your client
browsers (which will get rid of the warning) and use the key to do
dynamic cert generation in squid. Then it is possible to do either WPAD
based browser config, or, I think (harder) do TPROXY with bumping.

NB unless you can import your own CA cert into all client browsers you
*WILL* get certificate validation failures in the browser.

Cheers

Alex

>
> with to help
>
> regards
>
>
>
> -----
> Dr.x
> --
> View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/why-squid-can-block-https-when-i-point-my-browser-to-port-and-cant-when-its-transparent-tp4667069.html
> Sent from the Squid - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: why squid can block https when i point my browser to port , and cant when its transparent ?

Jason Haar-2
On 28/07/14 05:15, Alex Crow wrote:
>
> You need to create your own CA, import the CA cert into your client
> browsers (which will get rid of the warning) and use the key to do
> dynamic cert generation in squid. Then it is possible to do either
> WPAD based browser config, or, I think (harder) do TPROXY with bumping.
>
> NB unless you can import your own CA cert into all client browsers you
> *WILL* get certificate validation failures in the browser.
>

It's also a bit harder than that. Google chrome uses cert pinning to
ensure any time Chrome goes to any Google https site, that the server
cert is signed by the CA that Google knows it was signed by. This means
MITM SSL interception is noticed by Chrome and it shrieks and screams :-)

So even with SSL interception, you need to create an "exception acl" of
sites that are not to be fiddled with - which entirely undoes the reason
for doing intercept in the first place(*) - or somehow ban the use of Chrome

I do wonder where this will end. How long before Firefox starts pinning,
then MSIE, then it gets generalized, etc?

(*) Being able to view Cloud-provider HTTPS content is actually one of
the primary reasons I want to do SSL interception. In the past year
we've seen a major uptick in malware being delivered to clients via
https (Amazon, Google, Dropbox), and being able to get an AV filter in
there would really help. Unfortunately, Google  has this thing about
trying to stop nasty governments from spying on their citizens (ie both
motivations are justified, but diametrically opposed)


--
Cheers

Jason Haar
Corporate Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

Reply | Threaded
Open this post in threaded view
|

Re: why squid can block https when i point my browser to port , and cant when its transparent ?

Alex Rousskov
On 07/27/2014 04:49 PM, Jason Haar wrote:

> I do wonder where this will end.

Since one cannot combine interception, inspection, and secure delivery,
this can only end when at least one of those components dies.

Interception is probably the weak link here because it can be removed(*)
by technological means if enough folks decide it has to go. Inspection
(by trusted intermediaries) and secure delivery (through trusted
intermediaries) will probably stay (with modifications) because their
existence sprouts from the human nature (rather than just lack of
development discipline, will, and resources).


> How long before Firefox starts pinning,
> then MSIE, then it gets generalized, etc?

If applied broadly, pinning in an interception world will clash with
government, corporate, and parental desire to protect "assets".  With
todays technology, pinning can only survive on a limited scale IMHO. The
day after tomorrow, if interception dies, replaced by trusted
intermediaries, pinning will not be a problem.


Either that, or the entire web content is going to be owned by a few
content providers that would guarantee that their content is safe and
appropriate (hence, does not need to be inspected). This is what Google
claims with its pinning solution today, and I suspect it is not the
responsibility they actually want and enjoy.


Cheers,

Alex.
(*) I am only discussing overt technologies and needs here. Needless to
say, covert interception will stay with us for the foreseeable future.

Reply | Threaded
Open this post in threaded view
|

unbound and squid not resolving SSL sites

squid-8
In my network I have unbound redirecting some sites through the proxy  
server and checking authentication, If I redirect www.thisite.com it  
works corectly. However, as soon as SSL is used  
https://www.thissite.com it doesn't resolve at all. Any ideas what I  
have to do to enable ssl redirects in unbound or squid?

squid.conf
#
# Recommended minimum configuration:
#
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# We strongly recommend the following be uncommented to protect innocent
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
acl interval_auth external time_squid_auth
http_access allow interval_auth
http_access deny all
http_port 80 accel vhost allow-direct
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0% 0
refresh_pattern .               0 20%     4320


Reply | Threaded
Open this post in threaded view
|

Re: why squid can block https when i point my browser to port , and cant when its transparent ?

Amos Jeffries
Administrator
In reply to this post by Alex Rousskov
On 30/07/2014 11:59 a.m., Alex Rousskov wrote:

> On 07/27/2014 04:49 PM, Jason Haar wrote:
>
>> I do wonder where this will end.
>
> Since one cannot combine interception, inspection, and secure delivery,
> this can only end when at least one of those components dies.
>
> Interception is probably the weak link here because it can be removed(*)
> by technological means if enough folks decide it has to go. Inspection
> (by trusted intermediaries) and secure delivery (through trusted
> intermediaries) will probably stay (with modifications) because their
> existence sprouts from the human nature (rather than just lack of
> development discipline, will, and resources).
>
>
>> How long before Firefox starts pinning,
>> then MSIE, then it gets generalized, etc?
>
> If applied broadly, pinning in an interception world will clash with
> government, corporate, and parental desire to protect "assets".  With
> todays technology, pinning can only survive on a limited scale IMHO. The
> day after tomorrow, if interception dies, replaced by trusted
> intermediaries, pinning will not be a problem.
>
>
> Either that, or the entire web content is going to be owned by a few
> content providers that would guarantee that their content is safe and
> appropriate (hence, does not need to be inspected). This is what Google
> claims with its pinning solution today, and I suspect it is not the
> responsibility they actually want and enjoy.

It is also a false claim.
<http://www.thewhir.com/web-hosting-news/aws-supports-41-malware-hosting-sites-web-host-isp>

Shared hosting providers are a well known source of malware and viral
infection. Google hosted sites are no different even though their
https:// service is pinned. They do well enough to only get an "also
ran" mention but that is still not clean enough to warrant a bypass of
inspection (hundreds or a few thousand infection points make up their
their low % rating).

Amos

Reply | Threaded
Open this post in threaded view
|

unbound and squid not resolving SSL sites

squid-8
In reply to this post by squid-8
In my network I have unbound redirecting some sites through the proxy  
server and checking authentication, If I redirect www.thisite.com it  
works corectly. However, as soon as SSL is used  
https://www.thissite.com it doesn't resolve at all. Any ideas what I  
have to do to enable ssl redirects in unbound or squid?

squid.conf
#
# Recommended minimum configuration:
#
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7        # RFC 4193 local private network range
acl localnet src fe80::/10        # RFC 4291 link-local (directly  
plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# We strongly recommend the following be uncommented to protect innocent
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7        # RFC 4193 local private network range
acl localnet src fe80::/10        # RFC 4291 link-local (directly  
plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
acl interval_auth external time_squid_auth
http_access allow interval_auth
http_access deny all
http_port 80 accel vhost allow-direct
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%        0
refresh_pattern .               0        20%     4320

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Amos Jeffries
Administrator
On 5/08/2014 1:13 p.m., [hidden email] wrote:
> In my network I have unbound redirecting some sites through the proxy
> server and checking authentication, If I redirect www.thisite.com it
> works corectly. However, as soon as SSL is used https://www.thissite.com
> it doesn't resolve at all. Any ideas what I have to do to enable ssl
> redirects in unbound or squid?

Handle port 443 traffic and the encrypted traffic there.
You are only receiving port 80 traffic in this config file.


There are other problems in the config file displayed. Notes inline.

>
> squid.conf
> #
> # Recommended minimum configuration:
> #
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
>
> # Example rule allowing access from your local networks.
> # Adapt to list your (internal) IP networks from where browsing
> # should be allowed
> acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> acl localnet src fc00::/7        # RFC 4193 local private network range
> acl localnet src fe80::/10        # RFC 4291 link-local (directly
> plugged) machines
>
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl CONNECT method CONNECT
>

You should erase all of the lines above. They are duplicated below.

> #
> # Recommended minimum Access Permission configuration:
> #
> # Only allow cachemgr access from localhost
> http_access allow manager localhost
> http_access deny manager
>

NOTE: Current best practice recommendation is to have the manager access
control lines after the CONNECT one below. Saves on a couple of slow
regex calculations on certain types of DoS attacks.

> # Deny requests to certain unsafe ports
> http_access deny !Safe_ports
>
> # Deny CONNECT to other than secure SSL ports
> http_access deny CONNECT !SSL_ports
>
> # We strongly recommend the following be uncommented to protect innocent
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
>
> acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> acl localnet src fc00::/7        # RFC 4193 local private network range
> acl localnet src fe80::/10        # RFC 4291 link-local (directly
> plugged) machines
>
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl CONNECT method CONNECT
>
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports

NP: these four lines above are now occuring three times in a row in your
http_access rules. Only the first occurance will have any useful effect,
the rest just waste processing time.

>
> external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth

What does this helper do exactly to earn the term "authentication"?
TCP/IP address alone is insufficient to verify the end-users identity.


> acl interval_auth external time_squid_auth
> http_access allow interval_auth
> http_access deny all
> http_port 80 accel vhost allow-direct
> hierarchy_stoplist cgi-bin ?
> coredump_dir /var/spool/squid
>
> refresh_pattern ^ftp:           1440    20%     10080
> refresh_pattern ^gopher:        1440    0%      1440
> refresh_pattern -i (/cgi-bin/|\?) 0     0%        0
> refresh_pattern .               0        20%     4320
>

Amos
Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

squid-8
Current config below:

>> In my network I have unbound redirecting some sites through the proxy
>> server and checking authentication, If I redirect www.thisite.com it
>> works corectly. However, as soon as SSL is used https://www.thissite.com
>> it doesn't resolve at all. Any ideas what I have to do to enable ssl
>> redirects in unbound or squid?
>
> Handle port 443 traffic and the encrypted traffic there.
> You are only receiving port 80 traffic in this config file.

I am already redirecting 443 traffic but the proxy won't pick it up.
There is a SSL ports directive in the squid.conf so it should accept them?
For example, this line redirect all HTTP traffic but as soon as the  
browser wants a SSL connection, it is dropped:
local-data: "anywhere.mysite.com. 600 IN A 109.xxx.xx.xxx"
local-zone: "identity.mysite.com." redirect



>> external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
>
> What does this helper do exactly to earn the term "authentication"?
> TCP/IP address alone is insufficient to verify the end-users identity.
This helper checks that an IP address is contained within a database table.
If the IP address exists, then it allows them to use the proxy server.

acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#http_access deny to_localhost
external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
acl interval_auth external time_squid_auth
http_access allow interval_auth
#http_access allow all
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 80 accel vhost allow-direct
hierarchy_stoplist cgi-bin ?
#cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0     0% 0
refresh_pattern .               0 20%     4320



Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Amos Jeffries
Administrator
On 8/08/2014 8:55 a.m., [hidden email] wrote:

> Current config below:
>
>>> In my network I have unbound redirecting some sites through the proxy
>>> server and checking authentication, If I redirect www.thisite.com it
>>> works corectly. However, as soon as SSL is used https://www.thissite.com
>>> it doesn't resolve at all. Any ideas what I have to do to enable ssl
>>> redirects in unbound or squid?
>>
>> Handle port 443 traffic and the encrypted traffic there.
>> You are only receiving port 80 traffic in this config file.
>
> I am already redirecting 443 traffic but the proxy won't pick it up.
> There is a SSL ports directive in the squid.conf so it should accept them?

You mean SSL_Ports ACL?  that only restricts HTTP "CONNECT" method
tunnel requests to the port(s) usually used by SSL.

It does nothing to receive and decrypt HTTPS in its native port 443
format. Which is what you need to do, since your unbound server is
claiming that your Squid is the origin web server for those https://
traffic.

You are at least missing https_port and all the sslproxy_* directives
for outgoing HTTPS. Then also you are probably missing the TLS/SSL
certificate security keys, including any DNS entries for IPSEC, DNSSEC,
DANE, HSTS etc.


> For example, this line redirect all HTTP traffic but as soon as the
> browser wants a SSL connection, it is dropped:
> local-data: "anywhere.mysite.com. 600 IN A 109.xxx.xx.xxx"
> local-zone: "identity.mysite.com." redirect

Of course. Your Squid box is not listening on port 443 (HTTPS). By using
DNS in this way you are claiming that your 109.xxx.xx.xxx machine is
providing *all* services of that domain. Things naturally break when you
overlook one or more services your clients are using from it.

Amos
Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

squid-8
> You are at least missing https_port and all the sslproxy_* directives
> for outgoing HTTPS. Then also you are probably missing the TLS/SSL
> certificate security keys, including any DNS entries for IPSEC, DNSSEC,
> DANE, HSTS etc.
>

Ok, so I generated some keys and added the directives.
On restarting squid it askes for the certificate password and starts  
ok but it still won;t resolve the SSL websites.
I also added an iptables forward directive:
iptables  -t nat -A PREROUTING  -i eth0 -p tcp --dport  443 -j  
REDIRECT --to-port 3130

CONF:
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged)
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
acl interval_auth external time_squid_auth
http_access allow interval_auth
http_access deny all
http_port 80 accel vhost allow-direct
https_port 3130 transparent cert=/etc/squid/server.crt  
key=/etc/squid/server.key
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0     0% 0
refresh_pattern .               0 20%     4320




Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Eliezer Croitoru
Take a look at:
http://wiki.squid-cache.org/EliezerCroitoru/Drafts/SSLBUMP

Your squid.conf seems to be too incomplete to allow SSL-Bump to work.

Eliezer

On 08/18/2014 02:16 AM, [hidden email] wrote:

> Ok, so I generated some keys and added the directives.
> On restarting squid it askes for the certificate password and starts ok
> but it still won;t resolve the SSL websites.
> I also added an iptables forward directive:
> iptables  -t nat -A PREROUTING  -i eth0 -p tcp --dport  443 -j REDIRECT
> --to-port 3130
>
> CONF:
> acl manager proto cache_object
> acl localhost src 127.0.0.1/32 ::1
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
> acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
> acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
> acl localnet src fc00::/7    # RFC 4193 local private network range
> acl localnet src fe80::/10    # RFC 4291 link-local (directly plugged)
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl CONNECT method CONNECT
> http_access allow manager localhost
> http_access deny manager
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
> acl interval_auth external time_squid_auth
> http_access allow interval_auth
> http_access deny all
> http_port 80 accel vhost allow-direct
> https_port 3130 transparent cert=/etc/squid/server.crt
> key=/etc/squid/server.key
> hierarchy_stoplist cgi-bin ?
> coredump_dir /var/spool/squid
> refresh_pattern ^ftp:           1440    20%     10080
> refresh_pattern ^gopher:        1440    0%    1440
> refresh_pattern -i (/cgi-bin/|\?) 0     0%    0
> refresh_pattern .               0    20%     4320

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

squid-8

> Take a look at:
> http://wiki.squid-cache.org/EliezerCroitoru/Drafts/SSLBUMP
>
> Your squid.conf seems to be too incomplete to allow SSL-Bump to work.
>
> Eliezer

I recompiled to 3.4.6 and ran everything in your page there.
squid started correctly.
However, it is the same problem. Any https page that I had configured  
does not resolve. It is being redirected by unbound but as soon as it  
hits the proxy, it just gets dropped somehow:

# Generated by iptables-save v1.4.7 on Tue Aug 19 03:14:13 2014
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [5454:2633080]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT
-A INPUT -s 213.171.217.173/32 -p udp -m udp --dport 161 -m state  
--state NEW -j ACCEPT
-A INPUT -p udp -m udp --dport 161 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 161 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT
-A INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 25 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 110 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 143 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 20 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 21 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 3306 -m state --state NEW -j ACCEPT
-A INPUT -p udp -m udp --dport 3306 -m state --state NEW -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state NEW -j ACCEPT
COMMIT
# Completed on Tue Aug 19 03:14:13 2014
# Generated by iptables-save v1.4.7 on Tue Aug 19 03:14:13 2014
*nat
:PREROUTING ACCEPT [23834173:1866373947]
:POSTROUTING ACCEPT [22194:1519446]
:OUTPUT ACCEPT [22194:1519446]
-A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130
-A POSTROUTING -s 0.0.0.0/32 -o eth0 -j MASQUERADE
COMMIT
# Completed on Tue Aug 19 03:14:13 2014

#acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7        # RFC 4193 local private network range
acl localnet src fe80::/10        # RFC 4291 link-local (directly  
plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#http_access deny to_localhost
external_acl_type time_squid_auth ttl=5 %SRC /usr/local/bin/squidauth
acl interval_auth external time_squid_auth
http_access allow interval_auth
http_access deny all
http_port 80 accel vhost allow-direct
https_port 3130 intercept ssl-bump generate-host-certificates=on  
dynamic_cert_mem_cache_size=16MB  
cert=/usr/local/squid/ssl_cert/myCA.pem
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s  
/usr/local/squid/var/lib/ssl_db -M 16MB
sslcrtd_children 10
ssl_bump server-first all
#sslproxy_cert_error allow all
#sslproxy_flags DONT_VERIFY_PEER
hierarchy_stoplist cgi-bin ?
coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%        1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%        0
refresh_pattern .               0        20%     4320

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Eliezer Croitoru
I wasn't sure but I am now.
You are doing something wrong and I cannot tell what exactly.
Try to share this script output:
http://www1.ngtech.co.il/squid/basic_data.sh

There are missing parts in the whole setup such as clients IP and server
IP, what GW are you using etc..

Eliezer

On 08/19/2014 02:37 PM, [hidden email] wrote:

>
>> Take a look at:
>> http://wiki.squid-cache.org/EliezerCroitoru/Drafts/SSLBUMP
>>
>> Your squid.conf seems to be too incomplete to allow SSL-Bump to work.
>>
>> Eliezer
>
> I recompiled to 3.4.6 and ran everything in your page there.
> squid started correctly.
> However, it is the same problem. Any https page that I had configured
> does not resolve. It is being redirected by unbound but as soon as it
> hits the proxy, it just gets dropped somehow:
>
> # Generated by iptables-save v1.4.7 on Tue Aug 19 03:14:13 2014
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [5454:2633080]
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT
> -A INPUT -s 213.171.217.173/32 -p udp -m udp --dport 161 -m state
> --state NEW -j ACCEPT
> -A INPUT -p udp -m udp --dport 161 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 161 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT
> -A INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 25 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 110 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 143 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 20 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 21 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 3306 -m state --state NEW -j ACCEPT
> -A INPUT -p udp -m udp --dport 3306 -m state --state NEW -j ACCEPT
> -A INPUT -j REJECT --reject-with icmp-port-unreachable
> -A OUTPUT -o lo -j ACCEPT
> -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A OUTPUT -m state --state NEW -j ACCEPT
> COMMIT
> # Completed on Tue Aug 19 03:14:13 2014
> # Generated by iptables-save v1.4.7 on Tue Aug 19 03:14:13 2014
> *nat
> :PREROUTING ACCEPT [23834173:1866373947]
> :POSTROUTING ACCEPT [22194:1519446]
> :OUTPUT ACCEPT [22194:1519446]
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130
> -A POSTROUTING -s 0.0.0.0/32 -o eth0 -j MASQUERADE
> COMMIT
> # Completed on Tue Aug 19 03:14:13 2014

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Amos Jeffries
Administrator
On 20/08/2014 1:12 p.m., Eliezer Croitoru wrote:
> I wasn't sure but I am now.
> You are doing something wrong and I cannot tell what exactly.
> Try to share this script output:
> http://www1.ngtech.co.il/squid/basic_data.sh
>
> There are missing parts in the whole setup such as clients IP and server
> IP, what GW are you using etc..
>
> Eliezer


Probably expecting DNS based forgery to hijack the connections is the
mistake.

When receiving HTTPS all Squid has to work with are the two TCP packet
IP addresses. If one of them is the client IP and the other is forged by
DNS (unbound), what server is to be contacted?

Hostname from the "accel" hack is buried inside the encryption which has
not yet arrived from the client. So Squid has to decrypt some future
traffic in order to discover what server to contact right now to get the
cert details which need to be emitted in order to start decrypting that
future traffic. Impossible situation.
 But Squid is not aware of that, it just uses the TCP packet dst IP
(itself) and tries to get server TLS certificate from there. Entering in
an infinite loop of lookups instead of a useful decryption.


proxyplayer.co.uk;
 why are you using unbound for this at all?

Amos

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

squid-8
why are you using unbound for this at all?

Well, we use a geo location service much like a VPN or a proxy.
For transparent proxies, it works fine, squid passes through the SSL  
request and back to the client.
For VPN, everything is passed through.
But with unbound, we only want to pass through certain requests and  
some of them have SSL sites.
Surely, there's a way to pass a request from unbound, and redirect it  
through the transparent proxy, returning it straight to the client?




Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Amos Jeffries
Administrator
On 21/08/2014 8:59 a.m., [hidden email] wrote:

> why are you using unbound for this at all?
>
> Well, we use a geo location service much like a VPN or a proxy.
> For transparent proxies, it works fine, squid passes through the SSL
> request and back to the client.
> For VPN, everything is passed through.
> But with unbound, we only want to pass through certain requests and some
> of them have SSL sites.
> Surely, there's a way to pass a request from unbound, and redirect it
> through the transparent proxy, returning it straight to the client?
>

I'm not sure what you mean, unbound is a DNS server it does not process
HTTP protocol at all. All it does is tell the client where the *web
server* for a domain is located. But the client only needs to know which
route to use.

With a client connecting over WAN through a proxy you have:
 client --WAN--> proxy --> Internet
 client <--WAN-- proxy <-- Internet
plus for non-proxied traffic:
 client --WAN--> Internet
 client <--WAN-- Internet

With a client connecting over a VPN you have:
 client --VPN--> proxy --> Internet
 client <--VPN-- proxy <-- Internet
plus for non-proxied traffic:
 client --VPN--NAT--> Internet
 client <--VPN--NAT-- Internet

in both above cases the gateway router receiving WAN or VPN traffic is
responsible for the NAT/TPROXY/WCCP interception.

What I've gathered so far is that you are trying to achieve one of these:

A)
 client --VPN--> proxy --> Internet
 client <--VPN-- proxy <-- Internet
plus for non-proxied traffic:
 client --WAN--> Internet
 client <--WAN-- Internet


B)
 client --VPN--> proxy --> Internet
 client <--WAN-- proxy <-- Internet
plus for non-proxied traffic:
 client --VPN--> Internet
 client <--WAN-- Internet


which one?

Amos

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

squid-8

> which one?
It's client --> unbound --> if IP listed in unbound.conf --> forwarded  
to proxy --> page or stream returned to client

For others it's client --> unbound --> direct to internet with normal DNS

Reply | Threaded
Open this post in threaded view
|

Re: unbound and squid not resolving SSL sites

Amos Jeffries
Administrator
On 21/08/2014 2:37 p.m., [hidden email] wrote:
>
>> which one?
> It's client --> unbound --> if IP listed in unbound.conf --> forwarded
> to proxy --> page or stream returned to client
>
> For others it's client --> unbound --> direct to internet with normal DNS
>

Replace "forwarded to proxy" with "IP address forged as proxy".
Which is the source of the problem, your proxy does not have any TLS
security certificates or keys to handle the HTTPS traffic properly, and
no way to identify what the real server actually is.

Squid does not yet support receiving SNI, nor do many client software
support sending it. So the only way this can work is with packets
*routed* through the Squid device. The unbound setup you have cannot work.


What I am looking for is the network topology over which the TCP
connections are supposed to flow. VPN connection, LAN connection, WAN
connection, etc.
 This is necessary in order to identify which device is the suitable
gateway to setup a "tunnel" to the proxy. Then we can look at what types
of tunnel are appropriate for your situation.

Amos