Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Charlie Orford
Hi list

We're using squid 3.5.23 and trying to achieve the following:

client https request (not proxy aware) -> squid1 (https NAT intercept) -> upstream squid2 (configured as a cache_peer in squid1) -> origin server (e.g. www.google.com)

Amos mentioned in this thread http://lists.squid-cache.org/pipermail/squid-users/2016-March/009468.html that:

> Squid can:
>
>  A) relay CONNECT message from client to any upstream proxy.
>
>  B) generate CONNECT message on arriving intercepted HTTPS and relay
> that to upstream proxy *IF* (and only if) ssl_bump selects the 'splice'
> action.
>
>  C) relay <a class="moz-txt-link-freetext" href="https://">https:// URLs to an upstream TLS proxy.
>
>
> That is all at present.
>
> Squid cannot (yet) generate CONNECT messages to try and fetch TLS
> details via a non-TLS cache_peer. If you are able to sponsor that
> enhancement work patches are welcome, or sponsorship $$ to help pay
> persons working on these things (Christos / measurement-factory) are
> also welcome.

Option B seems to cover what we need i.e. squid1 wraps arriving intercepted HTTPS traffic in a CONNECT and sends it upstream to squid2 which in turn tunnels it to the origin server. Unfortunately, we can't get it to work: as soon as squid1 receives a client HTTPS request it exits with "assertion failed: PeerConnector.cc:116: "peer->use_ssl"" in cache.log

Relevant config for squid1:
######################################
acl localnet src 10.100.0.0/24
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
acl Blocked_domains dstdomain "/etc/squid3/blocked.domains.acl"
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
acl MITM_TRAFFIC myportname MITM_port

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny Blocked_domains
http_access allow localhost
http_access allow localnet
http_access deny all
http_reply_access allow all

https_port 8443 ssl-bump intercept cert=/etc/squid3/root_ca.combined.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB name=MITM_port
sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/squid3/ssl_db -M 4MB

ssl_bump peek all
ssl_bump splice all

nonhierarchical_direct off
never_direct allow all
prefer_direct off
cache_peer 192.168.0.1 parent 3128 0 no-query no-digest no-netdb-exchange name=WWW_GATEWAY


Relevant config for squid2:
######################################
acl localnet src 192.168.0.0/24
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_port 3128

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet
http_access deny all

http_reply_access allow all


Is what we want to do currently achievable with the latest 3.5 branch or have we misunderstood what Amos was stating (some of his posts about ssl interception and cache_peer support can be fairly cryptic)?

If it is achievable, does the cache_peer link itself also need to be encrypted (via the ssl flag) to make it work?

Thanks,
Charlie




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Charlie Orford
To follow up:

Adding ssl to the cache_peer directive on squid1 (and changing squid2 so it listens for connections on an https_port) gets us a little further but still doesn't work.

Clients get a SQUID_X509_V_ERR_DOMAIN_MISMATCH error (because the auto-generated cert squid1 gives to the client contains the domain of the cache_peer *not* the ultimate origin server).

The above is with the following ssl_bump directives set in squid1's config:

ssl_bump peek step1
ssl_bump peek step2
ssl_bump splice step3

A post from another user on this list seems to suggest they successfully got squid to do what we want (http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html) but when emulating their setup (i.e. peeking at step1, staring at step2 and then bumping at step3) we get the same
SQUID_X509_V_ERR_DOMAIN_MISMATCH error.

Setting sslflags=DONT_VERIFY_DOMAIN on the cache_peer directive has no effect.

Connecting to squid1
with a proxy aware client (on a standard http_port with the ssl-bump flag set but no intercept) also results in the same problem.

Where are we going wrong?

Charlie

On 27/01/2017 18:24, Charlie Orford wrote:
Hi list

We're using squid 3.5.23 and trying to achieve the following:

client https request (not proxy aware) -> squid1 (https NAT intercept) -> upstream squid2 (configured as a cache_peer in squid1) -> origin server (e.g. www.google.com)

Amos mentioned in this thread http://lists.squid-cache.org/pipermail/squid-users/2016-March/009468.html that:

> Squid can:
>
>  A) relay CONNECT message from client to any upstream proxy.
>
>  B) generate CONNECT message on arriving intercepted HTTPS and relay
> that to upstream proxy *IF* (and only if) ssl_bump selects the 'splice'
> action.
>
>  C) relay <a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://">https:// URLs to an upstream TLS proxy.
>
>
> That is all at present.
>
> Squid cannot (yet) generate CONNECT messages to try and fetch TLS
> details via a non-TLS cache_peer. If you are able to sponsor that
> enhancement work patches are welcome, or sponsorship $$ to help pay
> persons working on these things (Christos / measurement-factory) are
> also welcome.

Option B seems to cover what we need i.e. squid1 wraps arriving intercepted HTTPS traffic in a CONNECT and sends it upstream to squid2 which in turn tunnels it to the origin server. Unfortunately, we can't get it to work: as soon as squid1 receives a client HTTPS request it exits with "assertion failed: PeerConnector.cc:116: "peer->use_ssl"" in cache.log

Relevant config for squid1:
######################################
acl localnet src 10.100.0.0/24
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
acl Blocked_domains dstdomain "/etc/squid3/blocked.domains.acl"
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
acl MITM_TRAFFIC myportname MITM_port

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access deny Blocked_domains
http_access allow localhost
http_access allow localnet
http_access deny all
http_reply_access allow all

https_port 8443 ssl-bump intercept cert=/etc/squid3/root_ca.combined.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB name=MITM_port
sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/squid3/ssl_db -M 4MB

ssl_bump peek all
ssl_bump splice all

nonhierarchical_direct off
never_direct allow all
prefer_direct off
cache_peer 192.168.0.1 parent 3128 0 no-query no-digest no-netdb-exchange name=WWW_GATEWAY


Relevant config for squid2:
######################################
acl localnet src 192.168.0.0/24
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 443         # https
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

http_port 3128

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
http_access allow localnet
http_access deny all

http_reply_access allow all


Is what we want to do currently achievable with the latest 3.5 branch or have we misunderstood what Amos was stating (some of his posts about ssl interception and cache_peer support can be fairly cryptic)?

If it is achievable, does the cache_peer link itself also need to be encrypted (via the ssl flag) to make it work?

Thanks,
Charlie





_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Alex Rousskov
On 01/27/2017 04:04 PM, Charlie Orford wrote:

> Clients get a SQUID_X509_V_ERR_DOMAIN_MISMATCH error (because the
> auto-generated cert squid1 gives to the client contains the domain of
> the cache_peer *not* the ultimate origin server).

Under normal circumstances, Squid should generate no certificates in
your setup AFAICT.


> The above is with the following ssl_bump directives set in squid1's config:
>
> ssl_bump peek step1
> ssl_bump peek step2
> ssl_bump splice step3

In other words:

  ssl_bump peek all
  ssl_bump splice all

Why not just do this instead:

  ssl_bump splice all

I have not tested this, but I do not understand why you want a
three-step SslBump to blindly forward an SSL connection to a peer. I
would use the minimal number of steps possible: one if it works or two
if I have to because of some Squid bugs/missing features.

When everything goes OK, Squid should generate no certificates in either
case, but with three-step SslBump, there are a lot more opportunities
for Squid to detect problems and want to send an error response to the
client. To send an error message, Squid bumps the connection to the
client (which does require fake certificate generation).

Finally, I do not know whether Squid is capable of peeking at the origin
server through a peer, but I doubt it is. If my guess is correct, then
three-step splicing will not work for you because during step3 your
Squid will already be talking to the origin server rather than a
cache_peer (or will fail because it cannot do that).


> A post from another user on this list seems to suggest they successfully
> got squid to do what we want
> (http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html)
> but when emulating their setup (i.e. peeking at step1, staring at step2
> and then bumping at step3) we get the same
> SQUID_X509_V_ERR_DOMAIN_MISMATCH error.

I suggest the following order:

  1. Decide whether your Squid should bump or splice.
  2. Find the configuration that does what you decided in #1.

So far, you have given no reasons to warrant bumping so I assume you do
not need or want to bump anything. Thus, you should ignore any
configurations that contain "stare", "bump", or deprecated "*-first"
ssl_bump actions.


HTH,

Alex.


> On 27/01/2017 18:24, Charlie Orford wrote:
>> Hi list
>>
>> We're using squid 3.5.23 and trying to achieve the following:
>>
>> client https request (not proxy aware) -> squid1 (https NAT intercept)
>> -> upstream squid2 (configured as a cache_peer in squid1) -> origin
>> server (e.g. www.google.com)
>>
>> Amos mentioned in this thread
>> http://lists.squid-cache.org/pipermail/squid-users/2016-March/009468.html
>> that:
>>
>> > Squid can:
>> >
>> >  A) relay CONNECT message from client to any upstream proxy.
>> >
>> >  B) generate CONNECT message on arriving intercepted HTTPS and relay
>> > that to upstream proxy *IF* (and only if) ssl_bump selects the 'splice'
>> > action.
>> >
>> >  C) relay https:// URLs to an upstream TLS proxy.
>> >
>> >
>> > That is all at present.
>> >
>> > Squid cannot (yet) generate CONNECT messages to try and fetch TLS
>> > details via a non-TLS cache_peer. If you are able to sponsor that
>> > enhancement work patches are welcome, or sponsorship $$ to help pay
>> > persons working on these things (Christos / measurement-factory) are
>> > also welcome.
>>
>> Option B seems to cover what we need i.e. squid1 wraps arriving
>> intercepted HTTPS traffic in a CONNECT and sends it upstream to squid2
>> which in turn tunnels it to the origin server. Unfortunately, we can't
>> get it to work: as soon as squid1 receives a client HTTPS request it
>> exits with "assertion failed: PeerConnector.cc:116: "peer->use_ssl""
>> in cache.log
>>
>> Relevant config for squid1:
>> ######################################
>> acl localnet src 10.100.0.0/24
>> acl SSL_ports port 443
>> acl Safe_ports port 80          # http
>> acl Safe_ports port 443         # https
>> acl Safe_ports port 777         # multiling http
>> acl CONNECT method CONNECT
>> acl Blocked_domains dstdomain "/etc/squid3/blocked.domains.acl"
>> acl step1 at_step SslBump1
>> acl step2 at_step SslBump2
>> acl step3 at_step SslBump3
>> acl MITM_TRAFFIC myportname MITM_port
>>
>> http_access allow manager localhost
>> http_access deny manager
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_ports
>> http_access deny to_localhost
>> http_access deny Blocked_domains
>> http_access allow localhost
>> http_access allow localnet
>> http_access deny all
>> http_reply_access allow all
>>
>> https_port 8443 ssl-bump intercept
>> cert=/etc/squid3/root_ca.combined.pem generate-host-certificates=on
>> dynamic_cert_mem_cache_size=8MB name=MITM_port
>> sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/squid3/ssl_db -M 4MB
>>
>> ssl_bump peek all
>> ssl_bump splice all
>>
>> nonhierarchical_direct off
>> never_direct allow all
>> prefer_direct off
>> cache_peer 192.168.0.1 parent 3128 0 no-query no-digest
>> no-netdb-exchange name=WWW_GATEWAY
>>
>>
>> Relevant config for squid2:
>> ######################################
>> acl localnet src 192.168.0.0/24
>> acl SSL_ports port 443
>> acl Safe_ports port 80          # http
>> acl Safe_ports port 443         # https
>> acl Safe_ports port 777         # multiling http
>> acl CONNECT method CONNECT
>>
>> http_port 3128
>>
>> http_access allow manager localhost
>> http_access deny manager
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_ports
>> http_access deny to_localhost
>> http_access allow localnet
>> http_access deny all
>>
>> http_reply_access allow all
>>
>>
>> Is what we want to do currently achievable with the latest 3.5 branch
>> or have we misunderstood what Amos was stating (some of his posts
>> about ssl interception and cache_peer support can be fairly cryptic)?
>>
>> If it is achievable, does the cache_peer link itself also need to be
>> encrypted (via the ssl flag) to make it work?
>>
>> Thanks,
>> Charlie
>>
>>
>>
>>
>>
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Charlie Orford
On 27/01/2017 23:43, Alex Rousskov wrote:

> On 01/27/2017 04:04 PM, Charlie Orford wrote:
>> A post from another user on this list seems to suggest they successfully
>> got squid to do what we want
>> (http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html)
>> but when emulating their setup (i.e. peeking at step1, staring at step2
>> and then bumping at step3) we get the same
>> SQUID_X509_V_ERR_DOMAIN_MISMATCH error.
> I suggest the following order:
>
>    1. Decide whether your Squid should bump or splice.
>    2. Find the configuration that does what you decided in #1.
>
> So far, you have given no reasons to warrant bumping so I assume you do
> not need or want to bump anything. Thus, you should ignore any
> configurations that contain "stare", "bump", or deprecated "*-first"
> ssl_bump actions.

Sorry if my original intent wasn't clear. Obviously it makes no sense
intercepting ssl traffic if we're going to splice everything.

Our design goal is: intercept and bump local client https traffic on
squid1 (so we can filter certain urls, cache content etc.) and then
forward the request on to the origin server via an upstream squid2
(which has internet access).

The user who posted
http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html 
seems to have successfully done this but I can't replicate it. After
doing a lot of googling (and semi-successfully trying to interpret Amos'
various replies whenever bumping and cache_peers come up on this list)
I'm beginning to wonder if it is indeed possible or if that user simple
mistook what he was seeing when he posted that message (e.g. didn't
notice that squid was actually not bumping his client connections).

Charlie









_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Amos Jeffries
Administrator
On 28/01/2017 1:32 p.m., Charlie Orford wrote:

> On 27/01/2017 23:43, Alex Rousskov wrote:
>> On 01/27/2017 04:04 PM, Charlie Orford wrote:
>>> A post from another user on this list seems to suggest they successfully
>>> got squid to do what we want
>>> (http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html)
>>>
>>> but when emulating their setup (i.e. peeking at step1, staring at step2
>>> and then bumping at step3) we get the same
>>> SQUID_X509_V_ERR_DOMAIN_MISMATCH error.
>> I suggest the following order:
>>
>>    1. Decide whether your Squid should bump or splice.
>>    2. Find the configuration that does what you decided in #1.
>>
>> So far, you have given no reasons to warrant bumping so I assume you do
>> not need or want to bump anything. Thus, you should ignore any
>> configurations that contain "stare", "bump", or deprecated "*-first"
>> ssl_bump actions.
>
> Sorry if my original intent wasn't clear. Obviously it makes no sense
> intercepting ssl traffic if we're going to splice everything.
>
> Our design goal is: intercept and bump local client https traffic on
> squid1 (so we can filter certain urls, cache content etc.) and then
> forward the request on to the origin server via an upstream squid2
> (which has internet access).

Under a narrow set of splice conditions you can get traffic through the
2-proxy heirarchy. But that is a very limited set of circumstances and
definitely not working with 'bump' anywhere involved.

As Alex pointed out step3 also eliminates the CONNECT ability. Which I
was not aware of a year ago when I wrote that original email you referenced.


The problem is that *any* server or peer TLS communication prior to
deciding to splice eliminates the ability to use a fake-CONNECT. That is
absolute because *all* TLS server/peer communication has to go through
the CONNECT tunnel - or none can. Anything happening prior to its
existence wont be TLS authenticated with the origin server.

>
> The user who posted
> http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html
> seems to have successfully done this but I can't replicate it. After

They did not because Squid _cannot_ do it.

Note that their cache_peer has 'ssl' flag enabled. So their transparent
traffic is using the peer certificate to base the auto-generated certs
on. Which you already tried and decided was not workable for you.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Alex Rousskov
In reply to this post by Charlie Orford
On 01/27/2017 05:32 PM, Charlie Orford wrote:
> Obviously it makes no sense
> intercepting ssl traffic if we're going to splice everything.

It actually does make a lot of sense in many environments, but not
necessarily yours.


> Our design goal is: intercept and bump local client https traffic on
> squid1 (so we can filter certain urls, cache content etc.) and then
> forward the request on to the origin server via an upstream squid2
> (which has internet access).

Understood. Squid can be enhanced to do what you want. There is nothing
fundamentally impossible in what you are describing AFAICT. We need to
add an insecure peer connector, and then using that connector code on
the regular request forwarding path. The low-level code to do that
already exists in tunnel.cc, but needs to be refactored/moved. This is
an architecturally challenging work, but it is certainly doable.

After those Squid modifications, in the simplest case (ignoring that you
cannot bump some sites and that you may not want to bump some of the
clients either), your squid1 configuration would be something like this:

  ssl_bump stare all
  ssl_bump bump all

Your squid2 will not do SslBump. In fact, to achieve the stated goal,
squid2 does not need to support SSL at all -- it can blindly forward
[encrypted] traffic from squid1 to the internet.

Your next steps to make the above happen are outlined at
http://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F


> http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html
> seems to have successfully done this but I can't replicate it.

The configuration posted at the above URL is broken because it does not
tell Squid what to do after step1. If it did work, it was a bug like,
for example, bug 3209. Most likely, Squid just spliced everything (as
you suspect). Ignore that email. To learn why that configuration makes
no sense, study http://wiki.squid-cache.org/Features/SslPeekAndSplice


HTH,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarity on sending intercepted HTTPS traffic upstream to a cache_peer

Charlie Orford
On 28/01/2017 17:47, Alex Rousskov wrote:

>
>> Our design goal is: intercept and bump local client https traffic on
>> squid1 (so we can filter certain urls, cache content etc.) and then
>> forward the request on to the origin server via an upstream squid2
>> (which has internet access).
> Understood. Squid can be enhanced to do what you want. There is nothing
> fundamentally impossible in what you are describing AFAICT. We need to
> add an insecure peer connector, and then using that connector code on
> the regular request forwarding path. The low-level code to do that
> already exists in tunnel.cc, but needs to be refactored/moved. This is
> an architecturally challenging work, but it is certainly doable.
>
> After those Squid modifications, in the simplest case (ignoring that you
> cannot bump some sites and that you may not want to bump some of the
> clients either), your squid1 configuration would be something like this:
>
>    ssl_bump stare all
>    ssl_bump bump all
>
> Your squid2 will not do SslBump. In fact, to achieve the stated goal,
> squid2 does not need to support SSL at all -- it can blindly forward
> [encrypted] traffic from squid1 to the internet.
>
> Your next steps to make the above happen are outlined at
> http://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F
>
>
>> http://lists.squid-cache.org/pipermail/squid-users/2015-November/007955.html
>> seems to have successfully done this but I can't replicate it.
> The configuration posted at the above URL is broken because it does not
> tell Squid what to do after step1. If it did work, it was a bug like,
> for example, bug 3209. Most likely, Squid just spliced everything (as
> you suspect). Ignore that email. To learn why that configuration makes
> no sense, study http://wiki.squid-cache.org/Features/SslPeekAndSplice
>
>
> HTH,
>
> Alex.
>

Thanks Alex and Amos for your thoughtful replies, they've given me the
clarity I was seeking.

In terms of next steps, patching this functionality in to squid is
certainly well beyond my skill level. Sponsorship of the work may be
possible but not something I could  arrange quickly. I'll see how viable
this is and get in contact with Measurement Factory directly if I'm able
to secure the budget.

Kind regards,
Charlie



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...