Ubuntu 18 with Squid 4.11 SSL_BUMP

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Ubuntu 18 with Squid 4.11 SSL_BUMP

AMead
1. Compiled Squid 4.11 on Ubuntu 18 T3 EC2 instance:

./configure \
    --prefix=/usr \
    --exec-prefix=/usr \
    --bindir=/usr/bin \
    --sbindir=/usr/sbin \
    --libdir=/usr/lib \
    --libexecdir=/usr/libexec/squid \
    --includedir=/usr/include \
    --mandir=/usr/share/man \
    --infodir=/usr/share/info \
    --datadir=/usr/share/squid \
    --sysconfdir=/etc/squid \
    --localstatedir=/var \
    --sharedstatedir=/var/lib \
    --with-logdir=/var/log/squid \
    --with-pidfile=/var/run/squid.pid \
    --with-default-user=squid \
    --with-openssl \
    --enable-ssl \
    --enable-ssl-crtd


2. Initialized the ssl database:

sudo /usr/libexec/squid/security_file_certgen -c -s /var/cache/squid/ssl_db
-M 4MB


3. I've tried to read through a few similar posts, and got something
reasonably working for the allowance, but now it's appearing to allow
everything:

> /etc/squid/whitelist.txt
*.github.com

> /etc/squid/squid.conf

visible_hostname squid
cache deny all

# Handling HTTP requests
http_port 3128
http_port 3129 intercept
acl allowed_http_sites dstdomain "/etc/squid/whitelist.txt"
http_access allow allowed_http_sites

# Handling HTTPS requests
acl SSL_port port 443
http_access allow SSL_port

https_port 3130 intercept ssl-bump    \
        cert=/etc/squid/ssl/squid.pem \
        # generate-host-certificates=on \ # Defaulted with 4.11
        dynamic_cert_mem_cache_size=16MB

# HTTPS - Peek & Splice
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

# Alex R
# 10.0.1.93 NONE_ABORTED/200 0 CONNECT 209.216.230.240:443 - HIER_NONE/- -
#
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-4-6-Transparent-HTTP-amp-HTTPS-Proxy-td4687578.html
#ssl_bump peek step1
#ssl_bump peek step2 allowed_https_sites
#ssl_bump terminate step2
#ssl_bump splice all

# Berger
# 10.0.1.93 NONE_ABORTED/200 0 CONNECT 209.216.230.240:443 - HIER_NONE/- -
#
http://squid-web-proxy-cache.1019090.n4.nabble.com/squid-4-1-transparent-https-issue-quot-curl-60-SSL-certificate-problem-self-signed-certificate-in-ce-td4688553.html
#ssl_bump peek step1 all
#ssl_bump peek step2 allowed_https_sites
#ssl_bump splice step3 allowed_https_sites
#ssl_bump terminate

#dkanejs
# 10.0.1.93 TCP_TUNNEL/200 25082 CONNECT 185.199.111.153:443
# Allows https://example.com, https://github.com, but not
https://news.ycombinator.com
ssl_bump peek all
acl allowed_https_sites ssl::server_name "/etc/squid/whitelist.txt"
ssl_bump splice allowed_https_sites
ssl_bump terminate all

http_access deny all




--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Ubuntu 18 with Squid 4.11 SSL_BUMP

Amos Jeffries
Administrator
On 30/04/20 4:10 am, AMead wrote:
> 1. Compiled Squid 4.11 on Ubuntu 18 T3 EC2 instance:
>
> ./configure \

...
>     --with-openssl \
>     --enable-ssl \

"--enable-ssl" is not a Squid build option.

>     --enable-ssl-crtd
>
>
> 2. Initialized the ssl database:
>
> sudo /usr/libexec/squid/security_file_certgen -c -s /var/cache/squid/ssl_db
> -M 4MB
>
>
> 3. I've tried to read through a few similar posts, and got something
> reasonably working for the allowance, but now it's appearing to allow
> everything:
>
>> /etc/squid/whitelist.txt
> *.github.com
>

This is not dstdomain syntax. Remove the "*" character.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Ubuntu 18 with Squid 4.11 SSL_BUMP

AMead
Thanks!  I've re-compiled without the unnecessary flag, and restarted the service with a new whitelist, unfortunately i'm getting such a varying of /var/log/squid/access.log messages that I'm not sure what to google anymore.

I want to deny all access to external sites except http/https github.com but some sites seem to connect, while others don't:

~$ # this is correct
~$ curl http://github.com/
10.0.1.180 TCP_MISS/301 200 GET http://github.com/

~$ # this is correct
~$ curl https://github.com/ 
10.0.1.180 TCP_TUNNEL/200 107323 CONNECT 140.82.114.4:443

~$ # this should deny
~$ curl https://youtube.com/
10.0.1.180 TCP_TUNNEL/200 4844 CONNECT 172.217.15.110:443

~$ # this should deny
~$ curl https://google.com/
10.0.1.180 TCP_TUNNEL_ABORTED/200 5103 CONNECT 172.217.2.110:443

~$ # this is denying - but not from squid, but openssl?
~$ curl https://news.ycombinator.com/
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to news.ycombinator.com:443
10.0.1.180 NONE_ABORTED/200 0 CONNECT 209.216.230.240:443



On 4/29/20, 2:59 PM, "squid-users on behalf of Amos Jeffries" <[hidden email] on behalf of [hidden email]> wrote:

    On 30/04/20 4:10 am, AMead wrote:
    > 1. Compiled Squid 4.11 on Ubuntu 18 T3 EC2 instance:
    >
    > ./configure \

    ...
    >     --with-openssl \
    >     --enable-ssl \

    "--enable-ssl" is not a Squid build option.

    >     --enable-ssl-crtd
    >
    >
    > 2. Initialized the ssl database:
    >
    > sudo /usr/libexec/squid/security_file_certgen -c -s /var/cache/squid/ssl_db
    > -M 4MB
    >
    >
    > 3. I've tried to read through a few similar posts, and got something
    > reasonably working for the allowance, but now it's appearing to allow
    > everything:
    >
    >> /etc/squid/whitelist.txt
    > *.github.com
    >

    This is not dstdomain syntax. Remove the "*" character.


    Amos
    _______________________________________________
    squid-users mailing list
    [hidden email]
    http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Ubuntu 18 with Squid 4.11 SSL_BUMP

Amos Jeffries
Administrator
On 30/04/20 8:15 am, Anthony Mead wrote:
> Thanks!  I've re-compiled without the unnecessary flag, and restarted the service with a new whitelist, unfortunately i'm getting such a varying of /var/log/squid/access.log messages that I'm not sure what to google anymore.
>
> I want to deny all access to external sites except http/https github.com but some sites seem to connect, while others don't:
>

There are a lot of details missing from your quoted log lines. Details
such as which server was contacted are important when there are more
than one TCP connection involved.

Since this is SSL-Bump _each_ curl request should result in _3_
access.log lines - with varying client, server and URI values.

You are only showing us one log line at a time. With only the client and
URI parts.


Bellow is a *guess* about what is going on, based on what the status
says. This is only to demonstrate that for each line you show there is
at least one situation where your squid.conf file tells Squid to do an
action which would result in that line. Whether these guesses are right
requires all the information you are omitting.



> ~$ # this is correct
> ~$ curl http://github.com/
> 10.0.1.180 TCP_MISS/301 200 GET http://github.com/
>

 acl allowed_http_sites dstdomain "/etc/squid/whitelist.txt"
 http_access allow allowed_http_sites


> ~$ # this is correct
> ~$ curl https://github.com/ 
> 10.0.1.180 TCP_TUNNEL/200 107323 CONNECT 140.82.114.4:443
>

  acl SSL_port port 443
  http_access allow SSL_port

  ssl_bump peek all


> ~$ # this should deny
> ~$ curl https://youtube.com/
> 10.0.1.180 TCP_TUNNEL/200 4844 CONNECT 172.217.15.110:443
>

  acl SSL_port port 443
  http_access allow SSL_port

  ssl_bump peek all


> ~$ # this should deny
> ~$ curl https://google.com/
> 10.0.1.180 TCP_TUNNEL_ABORTED/200 5103 CONNECT 172.217.2.110:443
>

  acl SSL_port port 443
  http_access allow SSL_port

  ssl_bump peek all


> ~$ # this is denying - but not from squid, but openssl?
> ~$ curl https://news.ycombinator.com/
> curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to news.ycombinator.com:443
> 10.0.1.180 NONE_ABORTED/200 0 CONNECT 209.216.230.240:443
>

  acl SSL_port port 443
  http_access allow SSL_port

  ssl_bump terminate all



Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Ubuntu 18 with Squid 4.11 SSL_BUMP

AMead
Hmm, if there were more logs I'd share them!  Any reason why I'd only see a access.log line?

I promise if I curl https://google.com  this is the only line I see:
1588193897.852     20 10.0.1.180 TCP_TUNNEL_ABORTED/200 5103 CONNECT 172.217.15.78:443 - ORIGINAL_DST/172.217.15.78 -

Or curl https://youtube.com :
1588194262.880     32 10.0.1.180 TCP_TUNNEL/200 4824 CONNECT 172.217.13.78:443 - ORIGINAL_DST/172.217.13.78 -

Or curl https://github.com/:
1588194657.291     45 10.0.1.180 TCP_TUNNEL/200 107344 CONNECT 140.82.113.4:443 - ORIGINAL_DST/140.82.113.4 -

To avoid an X/Y problem the rest of my setup mimics a few blog posts - An EC2 in a private subnet that has all traffic being forwarded to the squid instance, which has iptables forwarding http/https to 3129/3130.  All approved traffic is then forwarded onto a NAT Gateway.  Maybe another piece of the "puzzle" is capturing the logs.

Also I really appreciate your help so far!


On 4/29/20, 4:35 PM, "squid-users on behalf of Amos Jeffries" <[hidden email] on behalf of [hidden email]> wrote:

    On 30/04/20 8:15 am, Anthony Mead wrote:
    > Thanks!  I've re-compiled without the unnecessary flag, and restarted the service with a new whitelist, unfortunately i'm getting such a varying of /var/log/squid/access.log messages that I'm not sure what to google anymore.
    >
    > I want to deny all access to external sites except http/https github.com but some sites seem to connect, while others don't:
    >

    There are a lot of details missing from your quoted log lines. Details
    such as which server was contacted are important when there are more
    than one TCP connection involved.

    Since this is SSL-Bump _each_ curl request should result in _3_
    access.log lines - with varying client, server and URI values.

    You are only showing us one log line at a time. With only the client and
    URI parts.


    Bellow is a *guess* about what is going on, based on what the status
    says. This is only to demonstrate that for each line you show there is
    at least one situation where your squid.conf file tells Squid to do an
    action which would result in that line. Whether these guesses are right
    requires all the information you are omitting.



    > ~$ # this is correct
    > ~$ curl http://github.com/
    > 10.0.1.180 TCP_MISS/301 200 GET http://github.com/
    >

     acl allowed_http_sites dstdomain "/etc/squid/whitelist.txt"
     http_access allow allowed_http_sites


    > ~$ # this is correct
    > ~$ curl https://github.com/ 
    > 10.0.1.180 TCP_TUNNEL/200 107323 CONNECT 140.82.114.4:443
    >

      acl SSL_port port 443
      http_access allow SSL_port

      ssl_bump peek all


    > ~$ # this should deny
    > ~$ curl https://youtube.com/
    > 10.0.1.180 TCP_TUNNEL/200 4844 CONNECT 172.217.15.110:443
    >

      acl SSL_port port 443
      http_access allow SSL_port

      ssl_bump peek all


    > ~$ # this should deny
    > ~$ curl https://google.com/
    > 10.0.1.180 TCP_TUNNEL_ABORTED/200 5103 CONNECT 172.217.2.110:443
    >

      acl SSL_port port 443
      http_access allow SSL_port

      ssl_bump peek all


    > ~$ # this is denying - but not from squid, but openssl?
    > ~$ curl https://news.ycombinator.com/
    > curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to news.ycombinator.com:443
    > 10.0.1.180 NONE_ABORTED/200 0 CONNECT 209.216.230.240:443
    >

      acl SSL_port port 443
      http_access allow SSL_port

      ssl_bump terminate all



    Amos
    _______________________________________________
    squid-users mailing list
    [hidden email]
    http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: [EXTERNAL] Re: Ubuntu 18 with Squid 4.11 SSL_BUMP

Amos Jeffries
Administrator
On 30/04/20 9:11 am, Anthony Mead wrote:

> Hmm, if there were more logs I'd share them!  Any reason why I'd only see a access.log line?
>
> I promise if I curl https://google.com  this is the only line I see:
> 1588193897.852     20 10.0.1.180 TCP_TUNNEL_ABORTED/200 5103 CONNECT 172.217.15.78:443 - ORIGINAL_DST/172.217.15.78 -
>
> Or curl https://youtube.com :
> 1588194262.880     32 10.0.1.180 TCP_TUNNEL/200 4824 CONNECT 172.217.13.78:443 - ORIGINAL_DST/172.217.13.78 -
>
> Or curl https://github.com/:
> 1588194657.291     45 10.0.1.180 TCP_TUNNEL/200 107344 CONNECT 140.82.113.4:443 - ORIGINAL_DST/140.82.113.4 -
>


Hm. There should at least be a second line showing what server name was
sent in the peek'd SNI or server cert.

The first looks like it reached "terminate all" at step3 of the bumping
process.

The last looks like it was spliced (by the data size transferred). But
that definitely requires the server name to happen.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users