HTTPS proxy setup questions

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

HTTPS proxy setup questions

subhish.pillai
Hi All,

I have a client application that sends periodic usage data to an external
application server over HTTPS using REST API calls. I want to tunnel this
connection through an HTTPS proxy at the client location. I am trying to
setup Squid v4.4 on Centos 7 server for doing this.

The clients are explicitly configured to connect through the proxy server
and has the CA certificate from the application server installed.

Being new to squid and proxy servers in general, I have a few basic
questions that I want to clarify --
1. What is the difference between SSL bumping and SSL interception?
2. What is the difference between "http_port 3128 intercept" and "http_port
3128 transparent"? Do i need to setup the http_port as either of these?
3. Do I need to create self-signed certs on the proxy server and distribute
it to the client and application server?

I have tried setting up ssl-bump with self-signed certs on the proxy server,
but I get a "http/1.1 400 bad request" when the client attempts a connection
and also see this in the squid access log "NONE/400 3820 NONE
error:invalid-request - HIER_NONE/- text/html"

At this point I am not sure if this is a client connection problem or a
squid config issue. I would really appreciate if somebody could validate my
squid configuration below or can point me in the right direction.

Thanks,
Subhish

Here is my squid.conf config --
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged)
machines

acl SSL_ports port 443
acl SSL_ports port 8449
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
#http_access deny CONNECT !SSL_ports
http_access allow CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access allow all

# Squid normally listens to port 3128
#http_port 3128
http_port 3128 ssl-bump cert=/etc/squid/ssl_cert/myCA.pem
key=/etc/squid/ssl_cert/myCA.pem generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB options=NO_SSLv2
#http_port 3128 ssl-bump cert=/etc/squid/ssl_cert/ca-bundle.pem
key=/etc/squid/ssl_cert/ca-bundle.pem generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB options=NO_SSLv2

#dns_v4_first on
sslcrtd_program /usr/lib64/squid/security_file_certgen -s /var/lib/ssl_db -M
4MB

acl step1 at_step SslBump1

ssl_bump peek step1
ssl_bump bump all

#tls_outgoing_options cafile=/etc/squid/ssl_cert/ca-bundle.pem

## Allow server side certificate errors such as untrusted certificates,
otherwise the connection is closed for such errors
sslproxy_cert_error allow all

## Accept certificates that fail verification (should only be needed if
using 'sslproxy_cert_error allow all')
sslproxy_flags DONT_VERIFY_PEER

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/cache/squid
shutdown_lifetime 5 seconds
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320





--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

Alex Rousskov
On 12/12/18 12:58 PM, subhish.pillai wrote:


> 1. What is the difference between SSL bumping and SSL interception?

These concepts describe activities at different layers:

* SSL bumping is, in Squid context, inspection of SSL traffic that often
also involves impersonating the origin server and decrypting encrypted
HTTP traffic (i.e. a MitM attack on the client-server HTTPS communication).

* SSL interception is, in this context, directing (TCP/IP traffic that
presumably carries) SSL traffic off its "natural" TCP/IP path so that it
gets to Squid. Interception itself works at protocol layers below SSL
and HTTP. What happens when the SSL traffic gets to Squid is outside
"SSL interception" scope.

Usually, folks intercept SSL traffic to bump it, but YMMV. It is
possible, for example, to simply log TCP-level information about the
intercepted traffic without any MitM attacks on SSL.


> 2. What is the difference between "http_port 3128 intercept" and "http_port
> 3128 transparent"? Do i need to setup the http_port as either of these?

The difference is in whether Squid impersonates the IP client, but you
need neither because your "clients are explicitly configured to connect
through the proxy server". You do not need to divert traffic from its
natural TCP/IP path to proxy it because that natural TCP/IP path already
goes through your proxy.


> 3. Do I need to create self-signed certs on the proxy server and distribute
> it to the client and application server?

* Yes if you want to inspect encrypted HTTP traffic of your client
application (i.e. get to the HTTP stuff inside the SSL layer).

* Yes if you want client to be able to read Squid-generated error pages.

* No otherwise. In this case, Squid will be just a blind TCP tunnel.


What do you want to use Squid for? The answer to that question has a
significant effect on your Squid configuration.


> # And finally deny all other access to this proxy
> http_access allow all

FWIW, your rule does not match the comment and creates an open proxy.
Both are bad.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

subhish.pillai
Thanks Alex, that was very helpful.

Based on your explanation, I just want to use squid as a blind TCP tunnel carrying the HTTPS connection from client to app server. 

In that case, I don't need to use ssl_bump feature and the ssl_crtd program for certificate management, is that correct?

Would this config file work to setup the TCP tunnel --

acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 3128

dns_v4_first on

## Allow server side certificate errors such as untrusted certificates, otherwise the connection is closed for such errors
sslproxy_cert_error allow all

## Accept certificates that fail verification (should only be needed if using 'sslproxy_cert_error allow all')
sslproxy_flags DONT_VERIFY_PEER

On Wed, Dec 12, 2018 at 3:49 PM Alex Rousskov <[hidden email]> wrote:
On 12/12/18 12:58 PM, subhish.pillai wrote:


> 1. What is the difference between SSL bumping and SSL interception?

These concepts describe activities at different layers:

* SSL bumping is, in Squid context, inspection of SSL traffic that often
also involves impersonating the origin server and decrypting encrypted
HTTP traffic (i.e. a MitM attack on the client-server HTTPS communication).

* SSL interception is, in this context, directing (TCP/IP traffic that
presumably carries) SSL traffic off its "natural" TCP/IP path so that it
gets to Squid. Interception itself works at protocol layers below SSL
and HTTP. What happens when the SSL traffic gets to Squid is outside
"SSL interception" scope.

Usually, folks intercept SSL traffic to bump it, but YMMV. It is
possible, for example, to simply log TCP-level information about the
intercepted traffic without any MitM attacks on SSL.


> 2. What is the difference between "http_port 3128 intercept" and "http_port
> 3128 transparent"? Do i need to setup the http_port as either of these?

The difference is in whether Squid impersonates the IP client, but you
need neither because your "clients are explicitly configured to connect
through the proxy server". You do not need to divert traffic from its
natural TCP/IP path to proxy it because that natural TCP/IP path already
goes through your proxy.


> 3. Do I need to create self-signed certs on the proxy server and distribute
> it to the client and application server?

* Yes if you want to inspect encrypted HTTP traffic of your client
application (i.e. get to the HTTP stuff inside the SSL layer).

* Yes if you want client to be able to read Squid-generated error pages.

* No otherwise. In this case, Squid will be just a blind TCP tunnel.


What do you want to use Squid for? The answer to that question has a
significant effect on your Squid configuration.


> # And finally deny all other access to this proxy
> http_access allow all

FWIW, your rule does not match the comment and creates an open proxy.
Both are bad.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--

Subhish Pillai

R&D Software Quality Engineer

Broadcom | Brocade Storage Networking

T (720) 462-2900


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

Amos Jeffries
Administrator
In reply to this post by Alex Rousskov
On 13/12/18 11:49 am, Alex Rousskov wrote:
> On 12/12/18 12:58 PM, subhish.pillai wrote:
>
>> 2. What is the difference between "http_port 3128 intercept" and "http_port
>> 3128 transparent"? Do i need to setup the http_port as either of these?
>
> The difference is in whether Squid impersonates the IP client,

No. There is no difference in Squid between the http(s)_port
"transparent" and "intercept" options behaviour.

"transparent" is a deprecated option. Do not use it at all in any
current or recent Squid versions.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

Amos Jeffries
Administrator
In reply to this post by subhish.pillai
On 13/12/18 12:50 pm, Subhish Pillai wrote:
> Thanks Alex, that was very helpful.
>
> Based on your explanation, I just want to use squid as a blind TCP
> tunnel carrying the HTTPS connection from client to app server. 
>
> In that case, I don't need to use ssl_bump feature and the ssl_crtd
> program for certificate management, is that correct?
>

Going by the description you gave of the client configuration, it should be.


> Would this config file work to setup the TCP tunnel --

...
> ## Allow server side certificate errors such as untrusted certificates,
> otherwise the connection is closed for such errors
> sslproxy_cert_error allow all
>
> ## Accept certificates that fail verification (should only be needed if
> using 'sslproxy_cert_error allow all')
> sslproxy_flags DONT_VERIFY_PEER
>

These sslproxy_* options only apply when Squid is actively performing
TLS to upstream servers. They have no place in the "blind tunnel" situation.
(They also are deprecated in Squid-4, replaced by the
tls_outgoing_options directive
<http://www.squid-cache.org/Doc/config/tls_outgoing_options/>).

If the client software is sending CONNECT requests containing the HTTPS
traffic, then there is absolutely nothing your config needs to do than
let them send those requests to the proxy (as the default config does).

You do not even need Squid to be built with TLS/SSL support. That is the
meaning of "blind" in this setup.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

subhish.pillai
I was able to get https working over the http connect tunnel, but I was still having issues with my client application connecting over the proxy. After some research it so happens that we have implemented the HTTPS proxy on the client side with libcurl. (implementing this - "https://daniel.haxx.se/blog/2016/11/26/https-proxy-with-curl/").

So now my use case for the squid proxy is to be able to accept a HTTPS_proxy request from the client and tunnel it forward to the destination server. For that I copied over the CA bundle from the client into the proxy server and pointed the "tls-cert" option to that file --

https_port 3128 tls-cert=/etc/squid/ssl_cert/ca-bundle.crt

I get the following error when starting the squid server --
Dec 14 10:03:42 squid[131539]: FATAL: No valid signing certificate configured for HTTPS_port [::]:3128

How do I get this to work without having to create self-signed certs on the proxy server and importing that into the client ca-bundle. 

Am I missing any config steps in the squid.conf file? I think I have misunderstood the function of the "tls-cert" and "tls-cafile" options in the https_port line. 

Thanks again for all the help!

Subhish


On Wed, Dec 12, 2018 at 6:53 PM Amos Jeffries <[hidden email]> wrote:
On 13/12/18 12:50 pm, Subhish Pillai wrote:
> Thanks Alex, that was very helpful.
>
> Based on your explanation, I just want to use squid as a blind TCP
> tunnel carrying the HTTPS connection from client to app server. 
>
> In that case, I don't need to use ssl_bump feature and the ssl_crtd
> program for certificate management, is that correct?
>

Going by the description you gave of the client configuration, it should be.


> Would this config file work to setup the TCP tunnel --

...
> ## Allow server side certificate errors such as untrusted certificates,
> otherwise the connection is closed for such errors
> sslproxy_cert_error allow all
>
> ## Accept certificates that fail verification (should only be needed if
> using 'sslproxy_cert_error allow all')
> sslproxy_flags DONT_VERIFY_PEER
>

These sslproxy_* options only apply when Squid is actively performing
TLS to upstream servers. They have no place in the "blind tunnel" situation.
(They also are deprecated in Squid-4, replaced by the
tls_outgoing_options directive
<http://www.squid-cache.org/Doc/config/tls_outgoing_options/>).

If the client software is sending CONNECT requests containing the HTTPS
traffic, then there is absolutely nothing your config needs to do than
let them send those requests to the proxy (as the default config does).

You do not even need Squid to be built with TLS/SSL support. That is the
meaning of "blind" in this setup.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--

Subhish Pillai

R&D Software Quality Engineer

Broadcom | Brocade Storage Networking

T (720) 462-2900


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

Alex Rousskov
On 12/14/18 12:03 PM, Subhish Pillai wrote:

> my use case for the squid proxy is to be able to accept a
> HTTPS_proxy request from the client and tunnel it forward to the
> destination server.

> How do I get this to work without having to create self-signed certs on
> the proxy server and importing that into the client ca-bundle.

Get a server certificate from a CA authority that the client trusts,
issued for the Squid proxy domain. Give Squid that certificate. For
example, you may be able to use a free letsencrypt.org CA.

An HTTPS proxy needs a certificate it can sign its traffic with. That
certificate must be issued by a client-trusted CA. Whether that is a
fake CA that you operate (what you may have referred to as a
"self-signed cert" above) or a real CA trusted by millions of other
clients (e.g., letsencrypt), is your choice.


> For that I copied over the CA bundle from the client
> into the proxy server and pointed the "tls-cert" option to that file

Why? Please suggest specific documentation changes that would remove the
implication that doing the above has something to do with your goals.
That option is for specifying the signing certificate (i.e. the
certificate the proxy is going to sign traffic with).


> Am I missing any config steps in the squid.conf file?

You are missing a clientca or tls-cafile option that triggers client
certificate request (from Squid to client) and gives Squid CA
certificates to trust when validating the client-supplied certificate.
This is unrelated to the Squid signing certificate discussed above.

Alex.


> On Wed, Dec 12, 2018 at 6:53 PM Amos Jeffries <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 13/12/18 12:50 pm, Subhish Pillai wrote:
>     > Thanks Alex, that was very helpful.
>     >
>     > Based on your explanation, I just want to use squid as a blind TCP
>     > tunnel carrying the HTTPS connection from client to app server. 
>     >
>     > In that case, I don't need to use ssl_bump feature and the ssl_crtd
>     > program for certificate management, is that correct?
>     >
>
>     Going by the description you gave of the client configuration, it
>     should be.
>
>
>     > Would this config file work to setup the TCP tunnel --
>
>     ...
>     > ## Allow server side certificate errors such as untrusted
>     certificates,
>     > otherwise the connection is closed for such errors
>     > sslproxy_cert_error allow all
>     >
>     > ## Accept certificates that fail verification (should only be
>     needed if
>     > using 'sslproxy_cert_error allow all')
>     > sslproxy_flags DONT_VERIFY_PEER
>     >
>
>     These sslproxy_* options only apply when Squid is actively performing
>     TLS to upstream servers. They have no place in the "blind tunnel"
>     situation.
>     (They also are deprecated in Squid-4, replaced by the
>     tls_outgoing_options directive
>     <http://www.squid-cache.org/Doc/config/tls_outgoing_options/>).
>
>     If the client software is sending CONNECT requests containing the HTTPS
>     traffic, then there is absolutely nothing your config needs to do than
>     let them send those requests to the proxy (as the default config does).
>
>     You do not even need Squid to be built with TLS/SSL support. That is the
>     meaning of "blind" in this setup.
>
>     Amos
>     _______________________________________________
>     squid-users mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> --
>
> *Subhish Pillai*
>
> R&D Software Quality Engineer
>
> Broadcom | Brocade Storage Networking
>
> T (720) 462-2900
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS proxy setup questions

subhish.pillai
Thank you for the directions, I have the https proxy working now.

I got a signed CA cert and installed it on the squid server and after importing the intermediate cert into the client, it is working as expected.

https_port 3128 tls-cert=/etc/squid/ssl_cert/ssl_certificate.cer tls-key=/etc/squid/ssl_cert/proxy.key

Thanks for all the help and the responsiveness.


Subhish


On Fri, Dec 14, 2018 at 2:33 PM Alex Rousskov <[hidden email]> wrote:
On 12/14/18 12:03 PM, Subhish Pillai wrote:

> my use case for the squid proxy is to be able to accept a
> HTTPS_proxy request from the client and tunnel it forward to the
> destination server.

> How do I get this to work without having to create self-signed certs on
> the proxy server and importing that into the client ca-bundle.

Get a server certificate from a CA authority that the client trusts,
issued for the Squid proxy domain. Give Squid that certificate. For
example, you may be able to use a free letsencrypt.org CA.

An HTTPS proxy needs a certificate it can sign its traffic with. That
certificate must be issued by a client-trusted CA. Whether that is a
fake CA that you operate (what you may have referred to as a
"self-signed cert" above) or a real CA trusted by millions of other
clients (e.g., letsencrypt), is your choice.


> For that I copied over the CA bundle from the client
> into the proxy server and pointed the "tls-cert" option to that file

Why? Please suggest specific documentation changes that would remove the
implication that doing the above has something to do with your goals.
That option is for specifying the signing certificate (i.e. the
certificate the proxy is going to sign traffic with).


> Am I missing any config steps in the squid.conf file?

You are missing a clientca or tls-cafile option that triggers client
certificate request (from Squid to client) and gives Squid CA
certificates to trust when validating the client-supplied certificate.
This is unrelated to the Squid signing certificate discussed above.

Alex.


> On Wed, Dec 12, 2018 at 6:53 PM Amos Jeffries <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 13/12/18 12:50 pm, Subhish Pillai wrote:
>     > Thanks Alex, that was very helpful.
>     >
>     > Based on your explanation, I just want to use squid as a blind TCP
>     > tunnel carrying the HTTPS connection from client to app server. 
>     >
>     > In that case, I don't need to use ssl_bump feature and the ssl_crtd
>     > program for certificate management, is that correct?
>     >
>
>     Going by the description you gave of the client configuration, it
>     should be.
>
>
>     > Would this config file work to setup the TCP tunnel --
>
>     ...
>     > ## Allow server side certificate errors such as untrusted
>     certificates,
>     > otherwise the connection is closed for such errors
>     > sslproxy_cert_error allow all
>     >
>     > ## Accept certificates that fail verification (should only be
>     needed if
>     > using 'sslproxy_cert_error allow all')
>     > sslproxy_flags DONT_VERIFY_PEER
>     >
>
>     These sslproxy_* options only apply when Squid is actively performing
>     TLS to upstream servers. They have no place in the "blind tunnel"
>     situation.
>     (They also are deprecated in Squid-4, replaced by the
>     tls_outgoing_options directive
>     <http://www.squid-cache.org/Doc/config/tls_outgoing_options/>).
>
>     If the client software is sending CONNECT requests containing the HTTPS
>     traffic, then there is absolutely nothing your config needs to do than
>     let them send those requests to the proxy (as the default config does).
>
>     You do not even need Squid to be built with TLS/SSL support. That is the
>     meaning of "blind" in this setup.
>
>     Amos
>     _______________________________________________
>     squid-users mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> --
>
> *Subhish Pillai*
>
> R&D Software Quality Engineer
>
> Broadcom | Brocade Storage Networking
>
> T (720) 462-2900
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--

Subhish Pillai

R&D Software Quality Engineer

Broadcom | Brocade Storage Networking

T (720) 462-2900


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users