HTTPS caching is not working in squid with ssl-bump enabled

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

HTTPS caching is not working in squid with ssl-bump enabled

Vignesh Ramessh
Hi Team,

Currently am running squid version 4.14 on RPi3.
Trying to cache https responses with cache-control:max-age headers available,
using ssl bump - peek n splice feature with examples available in this link :- https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/
But the https caching doesnt seem to work, https connections are getting established as TCP_TUNNEL/200 in squid access logs.
I wasnt able to find any proper documentation on https caching using squid.
Can you kindly help me with this request.

Regards,
Vignesh

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS caching is not working in squid with ssl-bump enabled

Alex Rousskov
On 3/23/21 2:10 AM, Vignesh Ramessh wrote:

> Currently am running squid version 4.14 on RPi3.
> Trying to cache https responses with cache-control:max-age headers
> available,
> using ssl bump - peek n splice feature with examples available in this
> link :- https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/


   ssl_bump peek step1
   ssl_bump bump all
   ssl_bump splice all

AFAICT, the above "bump all during step2" configuration (the last line
does not do anything and should be removed) should bump all traffic. Is
that your configuration?


> But the https caching doesnt seem to work, https connections are getting
> established as TCP_TUNNEL/200 in squid access logs.

Do you see non-CONNECT HTTP traffic (e.g. GET, POST, etc.) in access
log? If not, then Squid is not bumping traffic OR the client does not
like what Squid is doing. Please post your http*_port and ssl_bump
configuration with access.log lines corresponding to a single test
transaction that you think should be bumped.

Also, does the client (e.g. curl, wget, or browser) get an error from
Squid? Does the client display any kind of warning or error at all? What
certificate does the client show for the test connection?


> I wasnt able to find any proper documentation on https caching using squid.

What you call "HTTPS caching" consists of two virtually independent
actions: Bumping HTTPS connections and caching. Documentation exists for
each action. Currently, it sounds like the first action (bumping) is not
working in your setup. Until that is addressed, you can ignore the
caching part.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS caching is not working in squid with ssl-bump enabled

Vignesh Ramessh
Hi Alex,

We have just started to integrate squid proxy in our project, thanks for your reply and support.

Previously we were seeing TCP_TUNNEL for https://www.google.com.
Now, we are able to see the TCP_MISS transactions for https://www.google.com
As https://www.google.com does not contain a cache-control header in response, the response cannot be cached which we are aware of.


1616580079.857     73 ::1 NONE/200 0 CONNECT www.google.com:443 - HIER_DIRECT/172.217.163.68 -
1616580079.945     44 ::1 TCP_MISS/200 967 HEAD https://www.google.com/ - HIER_DIRECT/172.217.163.68 text/html

I generated key.pem and cert.pem files using openssl and when i tried to connect a local python https web server which has cache-control headers in https response, got the below error,

root@raspberrypi-rdk-hybrid:~# curl -I https://192.168.1.41:443 --proxy-cacert cert.pem --proxy http://localhost:3128
HTTP/1.1 200 Connection established

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Have attached the squid.conf file for your reference. Can you please check and let me know if I am missing something ?

Regards,
Vignesh 

On Tue, Mar 23, 2021 at 7:44 PM Alex Rousskov <[hidden email]> wrote:
On 3/23/21 2:10 AM, Vignesh Ramessh wrote:

> Currently am running squid version 4.14 on RPi3.
> Trying to cache https responses with cache-control:max-age headers
> available,
> using ssl bump - peek n splice feature with examples available in this
> link :- https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/


   ssl_bump peek step1
   ssl_bump bump all
   ssl_bump splice all

AFAICT, the above "bump all during step2" configuration (the last line
does not do anything and should be removed) should bump all traffic. Is
that your configuration?


> But the https caching doesnt seem to work, https connections are getting
> established as TCP_TUNNEL/200 in squid access logs.

Do you see non-CONNECT HTTP traffic (e.g. GET, POST, etc.) in access
log? If not, then Squid is not bumping traffic OR the client does not
like what Squid is doing. Please post your http*_port and ssl_bump
configuration with access.log lines corresponding to a single test
transaction that you think should be bumped.

Also, does the client (e.g. curl, wget, or browser) get an error from
Squid? Does the client display any kind of warning or error at all? What
certificate does the client show for the test connection?


> I wasnt able to find any proper documentation on https caching using squid.

What you call "HTTPS caching" consists of two virtually independent
actions: Bumping HTTPS connections and caching. Documentation exists for
each action. Currently, it sounds like the first action (bumping) is not
working in your setup. Until that is addressed, you can ignore the
caching part.


HTH,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

squid.conf (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS caching is not working in squid with ssl-bump enabled

Amos Jeffries
Administrator
On 24/03/21 11:11 pm, Vignesh Ramessh wrote:
> Hi Alex,
>
> We have just started to integrate squid proxy in our project, thanks for
> your reply and support.
>
> Previously we were seeing TCP_TUNNEL for https://www.google.com 
> <https://www.google.com>.
> Now, we are able to see the TCP_MISS transactions for
> https://www.google.com <https://www.google.com>.

> As https://www.google.com <https://www.google.com> does not contain a
> cache-control header in response, the response cannot be cached which we
> are aware of.
>

That is not true. The request method is primarily what determines
cacheability. Other things like Cache-Control only modify what the
method implies.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS caching is not working in squid with ssl-bump enabled

Alex Rousskov
In reply to this post by Vignesh Ramessh
On 3/24/21 6:11 AM, Vignesh Ramessh wrote:


> Previously we were seeing TCP_TUNNEL for https://www.google.com
> Now, we are able to see the TCP_MISS transactions for
> https://www.google.com <https://www.google.com>. 
> As https://www.google.com <https://www.google.com> does not contain a
> cache-control header in response, the response cannot be cached which we
> are aware of.

Well, the Cache-Control header is not required for responses to be
cachable (and, depending on the value, may even prevent caching) -- a
lot of factors go into that decision.


> 1616580079.857     73 ::1 NONE/200 0 CONNECT www.google.com:443
> <http://www.google.com:443> - HIER_DIRECT/172.217.163.68
> <http://172.217.163.68> -
> 1616580079.945     44 ::1 TCP_MISS/200 967 HEAD https://www.google.com/
> <https://www.google.com/> - HIER_DIRECT/172.217.163.68
> <http://172.217.163.68> text/html

It looks like your Squid is bumping TLS connections.


> I generated key.pem and cert.pem files using openssl and when i tried to
> connect a local python https web server which has cache-control headers
> in https response, got the below error,
>
> root@raspberrypi-rdk-hybrid:~# curl -I https://192.168.1.41:443
> --proxy-cacert cert.pem --proxy http://localhost:3128

Your are telling curl to connect to a plain HTTP proxy (no encryption)
so --proxy-cacert does not apply to this transaction. The insides of the
CONNECT tunnel will be encrypted, of course, but those insides are meant
for the origin server, not the proxy.

For curl to trust a Squid-bumped connection, you want to specify
--cacert (or equivalent), and the file you specify must match the CA
certificate that Squid uses to generate fake certificates --
squid-ca-cert-key.pem in your squid.conf.


HTH,

Alex.


> HTTP/1.1 200 Connection established
>
> curl: (60) SSL certificate problem: unable to get local issuer certificate
> More details here: https://curl.haxx.se/docs/sslcerts.html
> <https://curl.haxx.se/docs/sslcerts.html>
>
> curl failed to verify the legitimacy of the server and therefore could not
> establish a secure connection to it. To learn more about this situation and
> how to fix it, please visit the web page mentioned above.
>
> Have attached the squid.conf file for your reference. Can you please
> check and let me know if I am missing something ?


> On Tue, Mar 23, 2021 at 7:44 PM Alex Rousskov wrote:
>
>     On 3/23/21 2:10 AM, Vignesh Ramessh wrote:
>
>     > Currently am running squid version 4.14 on RPi3.
>     > Trying to cache https responses with cache-control:max-age headers
>     > available,
>     > using ssl bump - peek n splice feature with examples available in this
>     > link
>     :- https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/
>     <https://elatov.github.io/2019/01/using-squid-to-proxy-ssl-sites/>
>
>
>        ssl_bump peek step1
>        ssl_bump bump all
>        ssl_bump splice all
>
>     AFAICT, the above "bump all during step2" configuration (the last line
>     does not do anything and should be removed) should bump all traffic. Is
>     that your configuration?
>
>
>     > But the https caching doesnt seem to work, https connections are
>     getting
>     > established as TCP_TUNNEL/200 in squid access logs.
>
>     Do you see non-CONNECT HTTP traffic (e.g. GET, POST, etc.) in access
>     log? If not, then Squid is not bumping traffic OR the client does not
>     like what Squid is doing. Please post your http*_port and ssl_bump
>     configuration with access.log lines corresponding to a single test
>     transaction that you think should be bumped.
>
>     Also, does the client (e.g. curl, wget, or browser) get an error from
>     Squid? Does the client display any kind of warning or error at all? What
>     certificate does the client show for the test connection?
>
>
>     > I wasnt able to find any proper documentation on https caching
>     using squid.
>
>     What you call "HTTPS caching" consists of two virtually independent
>     actions: Bumping HTTPS connections and caching. Documentation exists for
>     each action. Currently, it sounds like the first action (bumping) is not
>     working in your setup. Until that is addressed, you can ignore the
>     caching part.
>
>
>     HTH,
>
>     Alex.
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users