HTTPS interception proxy having issues

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

HTTPS interception proxy having issues

aashutosh kalyankar

Hi! I am trying to set up a HTTPs intercept proxy but I cannot get it to work. Can someone point me in the right direction? 

I tried following the tutorial @ https://www.youtube.com/watch?v=Bogdplu_lsE (Transparent HTTP+HTTPS Proxy with Squid and iptables)  for squid file.
and https://github.com/diladele/squid-ubuntu for building squid 3.5 on ubuntu. 

squid.conf file 

acl clients src 172.16.10.0/24
acl clients src 172.18.10.0/24

http_access allow localhost
http_access allow clients
http_access deny all
http_port 8080
http_port 3128 intercept
https_port 3129 intercept ssl-bump cert=/etc/squid/ssl_certs/myCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

# only wait 5 seconds to terminate active connections
shutdown_lifetime 5
~                                

I am forced to use old 3.5 version of squid as I am running very old version of Vsphere supporting ubuntu 14.04 and below. 
Squid Cache: Version 3.5.19 
Service Name: squid
Ubuntu linux
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,smb_lm' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--with-openssl' '--enable-ssl' '--enable-ssl-crtd' '--enable-build-info=Ubuntu linux' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security'


Firewall & Nat rules added 
sudo iptables -A INPUT -j ACCEPT -p tcp --dport 3128 -m comment --comment "squid http proxy"
sudo iptables -A INPUT -j ACCEPT -p tcp --dport 3129 -m comment --comment "squid https proxy"
sudo iptables -A INPUT -j ACCEPT -p tcp  --dport 8080 -m comment -comment "squid http8080 proxy

 sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -m comment --comment "transparent http proxy" -j REDIRECT --to-ports 3128
 sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -m comment --comment "transparent https proxy" -j REDIRECT --to-ports 3129
 sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -m comment --comment " http 8080 proxy" -j REDIRECT --to-ports 8080

CACHE.log
My machine ip: 172.16.10.5
Squid server ip(vmware): 172.18.10.15
2019/12/09 19:42:00.677 kid1| SECURITY ALERT: Host header forgery detected on local=172.18.10.15:3128 remote=172.16.10.5:35346 FD 21 flags=33 (intercepted port does not match 443)
2019/12/09 19:42:00.677 kid1| SECURITY ALERT: By user agent: com.google.android.youtube/1447503000 (Linux; U; Android 7.1.1; en_US; Google Chromebook Pixel (2015); Build/R79-12607.47.0; Cronet/80.0.3955.6)
2019/12/09 19:42:00.677 kid1| SECURITY ALERT: on URL: www.googleadservices.com:443
2019/12/09 19:42:00.677 kid1| abandoning local=172.18.10.15:3128 remote=172.16.10.5:35346 FD 21 flags=33

access.log 
1575949926.409      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949935.727      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949935.834      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949937.667      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949939.207      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949939.799      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949945.905      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949946.688      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949950.602      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949952.727      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949958.849      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -

I am able to access neverssl.com  & example.com  (http) site but not https site.
1575949960.868     23 172.16.10.5 TCP_MISS/200 1869 GET http://vzwctrdxkflsnbhm.neverssl.com/online - HIER_DIRECT/13.35.127.108 text/html
1575949960.889      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949960.939      8 172.16.10.5 TCP_MISS/200 687 GET http://vzwctrdxkflsnbhm.neverssl.com/favicon.ico - HIER_DIRECT/13.35.127.108 image/png
1575949986.583      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949986.709      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949991.755      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575949998.720      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950005.659      1 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950015.981     32 172.16.10.5 TCP_MISS/301 387 GET http://www.apple.com/ - HIER_DIRECT/72.247.5.53 -
1575950015.987      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950041.486      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950046.063      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950052.787      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950055.532      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950091.821      9 172.16.10.5 TCP_MISS/200 1123 GET http://www.example.com/ - HIER_DIRECT/93.184.216.34 text/html
1575950091.891      3 172.16.10.5 TCP_MISS/404 1131 GET http://www.example.com/favicon.ico - HIER_DIRECT/93.184.216.34 text/html
1575950092.554      0 172.18.10.15 TCP_MISS/403 4474 POST http://stt.wifimaster.mobi/nw/ne - HIER_NONE/- text/html
1575950092.555     14 172.16.10.5 TCP_MISS/403 4576 POST http://stt.wifimaster.mobi/nw/ne - ORIGINAL_DST/172.18.10.15 text/html
1575950092.719      0 172.16.10.5 TAG_NONE/409 4266 CONNECT googlehomefoyer-pa.googleapis.com:443 - HIER_NONE/- text/html
1575950093.732      0 172.16.10.5 TAG_NONE/409 4266 CONNECT googlehomefoyer-pa.googleapis.com:443 - HIER_NONE/- text/html
1575950094.152      0 172.16.10.5 TAG_NONE/409 4068 CONNECT cast.google.com:443 - HIER_NONE/- text/html
1575950094.820      0 172.16.10.5 TAG_NONE/409 4266 CONNECT googlehomefoyer-pa.googleapis.com:443 - HIER_NONE/- text/html
1575950095.895      0 172.16.10.5 TAG_NONE/409 4266 CONNECT googlehomefoyer-pa.googleapis.com:443 - HIER_NONE/- text/html
1575950096.704      0 172.16.10.5 TAG_NONE/409 4266 CONNECT googlehomefoyer-pa.googleapis.com:443 - HIER_NONE/- text/html
1575950099.451      0 172.16.10.5 TAG_NONE/409 4115 CONNECT play.googleapis.com:443 - HIER_NONE/- text/html
1575950099.684      0 172.16.10.5 TAG_NONE/409 4115 CONNECT play.googleapis.com:443 - HIER_NONE/- text/html
1575950099.780      0 172.16.10.5 TAG_NONE/409 4115 CONNECT play.googleapis.com:443 - HIER_NONE/- text/html
1575950108.646      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950112.638      2 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950113.655     16 172.16.10.5 TCP_MISS/301 592 GET http://www.cnn.com/ - HIER_DIRECT/151.101.1.67 -
1575950113.665      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950113.808      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950118.839      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950119.920      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950127.161      1 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950132.158      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950133.481      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950134.155      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950140.548      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950140.633      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950145.675      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950146.415      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950152.852      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950155.864      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950156.948      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950187.018      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950192.630      0 172.16.10.5 TAG_NONE/200 0 CONNECT 172.18.10.15:3129 - HIER_NONE/- -
1575950196.056      7 172.16.10.5 TCP_MISS/204 449 GET http://www.gstatic.com/generate_204 - HIER_DIRECT/172.217.6.35 -



Thanks!
Aashutosh 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: HTTPS interception proxy having issues

Amos Jeffries
Administrator
On 11/12/19 3:48 am, aashutosh kalyankar wrote:

>
> Hi! I am trying to set up a HTTPs intercept proxy but I cannot get it to
> work. Can someone point me in the right direction? 
>
> I tried following the
> tutorial @ https://www.youtube.com/watch?v=Bogdplu_lsE (Transparent
> HTTP+HTTPS Proxy with Squid and iptables)  for squid file.
> and https://github.com/diladele/squid-ubuntu for building squid 3.5 on
> ubuntu. 
>
> *squid.conf file *
>
> acl clients src 172.16.10.0/24
> acl clients src 172.18.10.0/24
>
> http_access allow localhost
> http_access allow clients
> http_access deny all
> http_port 8080
> http_port 3128 intercept
> https_port 3129 intercept ssl-bump cert=/etc/squid/ssl_certs/myCA.pem
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
>
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump bump all
>
> # only wait 5 seconds to terminate active connections
> shutdown_lifetime 5
> ~                                
>
> I am forced to use old 3.5 version of squid as I am running very old
> version of Vsphere supporting ubuntu 14.04 and below.

Such things do not apply when building from source. You can build any
version which your build tools can handle. That should be any Squid-3.5
release, including the daily auto-generated code.



> *Squid Cache: Version 3.5.19 *
> Service Name: squid
> Ubuntu linux
> configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr'
> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
> '--infodir=${prefix}/share/info' '--sysconfdir=/etc'
> '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.'
> '--disable-maintainer-mode' '--disable-dependency-tracking'
> '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector
> --param=ssp-buffer-size=4 -Wformat -Werror=format-security
> -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro
> -Wl,-z,now' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
> '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man'
> '--enable-inline' '--disable-arch-native' '--enable-async-io=8'
> '--enable-storeio=ufs,aufs,diskd,rock'
> '--enable-removal-policies=lru,heap' '--enable-delay-pools'
> '--enable-cache-digests' '--enable-icap-client'
> '--enable-follow-x-forwarded-for'
> '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB'
> '--enable-auth-digest=file,LDAP'
> '--enable-auth-negotiate=kerberos,wrapper'
> '--enable-auth-ntlm=fake,smb_lm'
> '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group'
> '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi'
> '--enable-icmp' '--enable-zph-qos' '--enable-ecap'
> '--disable-translation' '--with-swapdir=/var/spool/squid'
> '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
> '--with-filedescriptors=65536' '--with-large-files'
> '--with-default-user=proxy' '--with-openssl' '--enable-ssl'
> '--enable-ssl-crtd' '--enable-build-info=Ubuntu linux'
> '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2
> -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat
> -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE
> -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-D_FORTIFY_SOURCE=2'
> 'CXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4
> -Wformat -Werror=format-security'
>
>
> *Firewall & Nat rules added *
> sudo iptables -A INPUT -j ACCEPT -p tcp --dport 3128 -m comment
> --comment "squid http proxy"
> sudo iptables -A INPUT -j ACCEPT -p tcp --dport 3129 -m comment
> --comment "squid https proxy"
> sudo iptables -A INPUT -j ACCEPT -p tcp  --dport 8080 -m comment
> -comment "squid http8080 proxy


Irrelevant unless you have a local policy of requiring these for any
port to receive traffic.

There should be mangle table PREROUTING chain rule(s) to DROP or REJECT
any packets headed to Squid intercept ports.


>
>  sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -m
> comment --comment "transparent http proxy" -j REDIRECT --to-ports 3128
>  sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -m
> comment --comment "transparent https proxy" -j REDIRECT --to-ports 3129
>  sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -m
> comment --comment " http 8080 proxy" -j REDIRECT --to-ports 8080

You already REDIRECT port 80 to port 3128. This last rule will do nothing.

>
> *CACHE.log*
> My machine ip: 172.16.10.5
> Squid server ip(vmware): 172.18.10.15
> 2019/12/09 19:42:00.677 kid1| SECURITY ALERT: Host header forgery
> detected on local=172.18.10.15:3128
> <http://172.18.10.15:3128/> remote=172.16.10.5:35346
> <http://172.16.10.5:35346/> FD 21 flags=33 (intercepted port does not
> match 443)


Squid is receiving a request for the URL https://172.18.10.15:3128/ or
equivalent.

It looks to me like you are making the classic mistake of sending
traffic directly to the Squid intercept port.

To test an interceptor proxy you MUST have a client making normal
requests like you would see them do in production environment ...
directly to the HTTP(S) origin servers.
 Let the intercept/NAT systems catch the traffic and deliver it to the
proxy - only then will that proxy have a chance at working as intended.



Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users