Squid v3.3.8 & SSL Bumping Issues

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid v3.3.8 & SSL Bumping Issues

David Marcos
Hi,

I have been attempting to configure SSL bumping with Squid v3.3.8.  I
have a well configured Squid proxy for HTTP and HTTP Intercept
proxying.  I am now trying to expand the configuration to bump SSL
connections.  I believe I have the basics of the configuration correct
for both direct HTTPS proxying as well as intercepted HTTPS, but am
having a few issues that I would appreciate some input on.
Specifically:

     a. HTTPS Page Rendering: Some HTTPS pages load fine.  However, I
have found that if I try to login to online banking or other secure
pages that either (1) the page does not render properly (I get flat,
unorganized text) or (2) the page simply does not load.  With respect
to the latter, some pages simply bring me right back to the login
page; there seems to be some kind of behind-the-scenes redirection
that is being rejected and preventing logging in.  What
recommendations might anyone have to tweak my configuration to address
these issues?

     b. HTTP Strict Transport Security (HSTS): Some pages flat-out
reject any SSL bumping due to HSTS.  I am using Chrome, which I'm sure
aggravates the issue.  Is there a way to configure Squid to get around
HSTS?  (Yes, I know this may be a dumb question given how HSTS works,
but would appreciate any insight.)

Fundamentally, my intent is to set up Squid for home use to block
advertising, malware, and in particular, perform content adaptation.
One of my specific goals is to modify search URL paths to restrict
explicit search returns (e.g. affixing "&safe=active" to any Google
search path).  I have additionally configured ICAP with SquidClamav,
multiple ACLs for blocking of ads and malware, and SquidGuard for
additional domain and url blocking.  SquidGuard is also successfully
manipulating *unencrypted* Google, Yahoo, and Bing URL paths to insert
commands to suppress explicit search returns.  (I should note that
when I tested out SSL bumping, I disabled ICAP, Squidguard, and ACLs
for blocking of ads and malware; the issues described above
persisted.)

Below is my squid.conf file to help out.

Thanks in advance,

Dave

#########BEGIN FILE#############
hosts_file /etc/hosts
visible_hostname proxyserver
shutdown_lifetime 5 seconds
coredump_dir /tmp


dns_nameservers 192.168.1.1 208.67.222.222 208.67.220.220
half_closed_clients off
negative_ttl 0
negative_dns_ttl 2 minutes

http_port 127.0.0.1:3128

http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/squid3/certs/cert.crt
key=/etc/squid3/certs/cert.key

http_port 192.168.1.1:3129 intercept

https_port 192.168.1.1:3130 intercept ssl-bump
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
cert=/etc/squid3/certs/cert.crt key=/etc/squid3/certs/cert.key

sslcrtd_program /usr/lib/squid3/ssl_crtd -s /disk/dyn-certs/sslcrtd_db -M 4MB
sslcrtd_children 5

udp_incoming_address 192.168.1.1
pinger_enable off
forwarded_for delete
via off

memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
maximum_object_size_in_memory 1 MB
minimum_object_size 0 KB
maximum_object_size 64 MB
memory_pools off
cache_mem 256 MB
cache_dir aufs /disk/squid-cache 25000 32 512
cache_swap_low 95
cache_swap_high 97
ipcache_size 10240
fqdncache_size 2048
quick_abort_min 0 KB
quick_abort_max 0 KB
max_filedescriptors 4096
read_ahead_gap 512 KB

client_lifetime 6 hours
connect_timeout 10 seconds

log_icp_queries off
buffered_logs on
debug_options ALL,1
logformat squid %tg %6tr %>A %Ss/%03>Hs UA=%{User-Agent}>h
XFF=%{X-Forwarded-For}>h CKE=- %rm %ru %un %Sh/%<A %mt BYTES=%<st
access_log stdio:/var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log none #/var/log/squid/store.log

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service sqclamav_req reqmod_precache bypass=1
icap://127.0.0.1:1344/squidclamav
adaptation_access sqclamav_req allow all
icap_service sqclamav_resp respmod_precache bypass=1
icap://127.0.0.1:1344/squidclamav
adaptation_access sqclamav_resp allow all

refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp)$ 10080 90% 43200
override-expire ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|m3u|mp4|mpeg|swf|flv|x-flv)$
43200 90% 259200 override-expire ignore-no-store ignore-no-cache
ignore-private
refresh_pattern -i
\.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|pptx|doc|docx|xls|xlsx|tiff)$
10080 90% 43200 override-expire ignore-no-store ignore-private
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern -i exe$ 0 50% 259200
refresh_pattern -i zip$ 0 50% 259200
refresh_pattern -i tar\.gz$ 0 50% 259200
refresh_pattern -i tgz$ 0 50% 259200
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (\.cgi$|/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

acl SSL_ports port 443 21 563 445 1494 2598 5006
acl Safe_ports port 21 80 443 563 70 210 280 488 591 777 1025-65535
acl purge method PURGE
acl CONNECT method CONNECT
acl FTP proto FTP

acl lan src 192.168.1.0/24
acl vpn src 192.168.12.0/24
acl to_lan dst 192.168.1.0/24
acl to_vpn dst 192.168.12.0/24

acl malware_stinks dstdomain
"/disk/squidguard/blacklist_acls/suspiciousdomains_Medium.txt"
acl ad_regex url_regex -i
"/disk/squidguard/blacklist_acls/ad_block_easylist.txt"
acl privacy_regex url_regex -i
"/disk/squidguard/blacklist_acls/privacy_easylist.txt"
deny_info http://www.lan/malware.html malware_stinks
deny_info http://www.lan/adregex.html ad_regex
deny_info http://www.lan/privacy.html privacy_regex

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

http_access deny malware_stinks
http_access deny ad_regex
http_access deny privacy_regex

http_access allow localhost manager
http_access deny manager
http_access allow localhost purge
http_access deny purge
http_access allow FTP localhost
http_access allow FTP lan
http_access allow FTP vpn
http_access allow lan
http_access allow to_lan
http_access allow vpn
http_access allow localhost

http_access deny all

http_reply_access allow all

icp_access allow lan
icp_access allow vpn
icp_access deny all

snmp_access allow lan
snmp_access allow vpn
snmp_access deny all

htcp_access allow lan
htcp_access allow vpn
htcp_access deny all

always_direct allow lan
always_direct allow vpn

ssl_bump none localhost
ssl_bump server-first lan
ssl_bump server-first vpn
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

request_header_replace Accept */*
request_header_replace Accept-Encoding gzip
request_header_replace Accept-Language en
request_header_replace From
request_header_replace Referer
request_header_replace Cookie
request_header_replace Cookie2

request_header_add DNT 1

request_header_access Allow allow all
request_header_access Authorization allow all
# corresponds to above replace acl
request_header_access From deny !to_lan
# corresponds to above replace acl
request_header_access Referer deny !to_lan
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
# corresponds to above replace acl
request_header_access Accept deny !to_lan
request_header_access Accept-Charset deny !to_lan
# corresponds to above replace acl
request_header_access Accept-Encoding deny !to_lan
# corresponds to above replace acl
request_header_access Accept-Language deny !to_lan
request_header_access Content-Language allow all
request_header_access Mime-Version allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access Connection allow all
request_header_access Proxy-Connection allow all
request_header_access User-Agent allow all
request_header_access Other allow all
request_header_access Cookie allow all
request_header_access Cookie2 allow all
# Corresponds to DNT header add/replace above
request_header_access DNT deny all
request_header_access All deny !to_lan

reply_header_access Allow allow all
reply_header_access Authorization allow all
reply_header_access Proxy-Authorization allow all
reply_header_access Proxy-Authenticate allow all
reply_header_access Content-Encoding allow all
reply_header_access Content-Length allow all
reply_header_access Content-Type allow all
reply_header_access Date allow all
reply_header_access Expires allow all
reply_header_access Host allow all
reply_header_access If-Modified-Since allow all
reply_header_access Last-Modified allow all
reply_header_access Location allow all
reply_header_access Pragma allow all
reply_header_access Accept allow all
reply_header_access Accept-Charset allow all
reply_header_access Accept-Encoding allow all
reply_header_access Accept-Language allow all
reply_header_access Content-Language allow all
reply_header_access Mime-Version allow all
reply_header_access Retry-After allow all
reply_header_access Title allow all
reply_header_access Connection allow all
reply_header_access User-Agent allow all
reply_header_access Other allow all
reply_header_access Set-Cookie allow all
reply_header_access Set-Cookie2 allow all
reply_header_access Transfer-Encoding allow all
reply_header_access Strict-Transport-Security allow all
reply_header_access DNT allow all

reply_header_access WWW-Authenticate deny !to_lan
reply_header_access Server deny !to_lan
reply_header_access Link deny !to_lan
reply_header_access All deny !to_lan

## SquidGuard
# This sets up 10 SquidGuard programs; 5 ready-to-go; 10
idled-and-prepped-to-go;
# and threading for a total of 15 SquidGuard handlers available to
tackle blocking
# Note *do not* use the concurrency option; it crashes SquidGuard
url_rewrite_program /usr/bin/squidGuard -c /etc/squid3/squidGuard.conf
url_rewrite_children 15 startup=5 idle=10
url_rewrite_bypass off
url_rewrite_access allow all
##########END FILE################
Reply | Threaded
Open this post in threaded view
|

Re: Squid v3.3.8 & SSL Bumping Issues

Cassiano Martin-2
> Fundamentally, my intent is to set up Squid for home use to block
> advertising, malware, and in particular, perform content adaptation.
> One of my specific goals is to modify search URL paths to restrict
> explicit search returns (e.g. affixing "&safe=active" to any Google
> search path)

Hi David, I did some work to filter google explicit search by DNS
hijacking and tinyproxy

I redirect google.* to nosslsearch.google.com and use my modified
version of tinyproxy, which transparently intercepts *only* google
traffic for now. I did it for an embedded platform.

If you'd like to check it out and test it:
https://github.com/polaco1782/tinyproxy

2014-07-08 23:17 GMT-03:00 David Marcos <[hidden email]>:

> Hi,
>
> I have been attempting to configure SSL bumping with Squid v3.3.8.  I
> have a well configured Squid proxy for HTTP and HTTP Intercept
> proxying.  I am now trying to expand the configuration to bump SSL
> connections.  I believe I have the basics of the configuration correct
> for both direct HTTPS proxying as well as intercepted HTTPS, but am
> having a few issues that I would appreciate some input on.
> Specifically:
>
>      a. HTTPS Page Rendering: Some HTTPS pages load fine.  However, I
> have found that if I try to login to online banking or other secure
> pages that either (1) the page does not render properly (I get flat,
> unorganized text) or (2) the page simply does not load.  With respect
> to the latter, some pages simply bring me right back to the login
> page; there seems to be some kind of behind-the-scenes redirection
> that is being rejected and preventing logging in.  What
> recommendations might anyone have to tweak my configuration to address
> these issues?
>
>      b. HTTP Strict Transport Security (HSTS): Some pages flat-out
> reject any SSL bumping due to HSTS.  I am using Chrome, which I'm sure
> aggravates the issue.  Is there a way to configure Squid to get around
> HSTS?  (Yes, I know this may be a dumb question given how HSTS works,
> but would appreciate any insight.)
>
> Fundamentally, my intent is to set up Squid for home use to block
> advertising, malware, and in particular, perform content adaptation.
> One of my specific goals is to modify search URL paths to restrict
> explicit search returns (e.g. affixing "&safe=active" to any Google
> search path).  I have additionally configured ICAP with SquidClamav,
> multiple ACLs for blocking of ads and malware, and SquidGuard for
> additional domain and url blocking.  SquidGuard is also successfully
> manipulating *unencrypted* Google, Yahoo, and Bing URL paths to insert
> commands to suppress explicit search returns.  (I should note that
> when I tested out SSL bumping, I disabled ICAP, Squidguard, and ACLs
> for blocking of ads and malware; the issues described above
> persisted.)
>
> Below is my squid.conf file to help out.
>
> Thanks in advance,
>
> Dave
>
> #########BEGIN FILE#############
> hosts_file /etc/hosts
> visible_hostname proxyserver
> shutdown_lifetime 5 seconds
> coredump_dir /tmp
>
>
> dns_nameservers 192.168.1.1 208.67.222.222 208.67.220.220
> half_closed_clients off
> negative_ttl 0
> negative_dns_ttl 2 minutes
>
> http_port 127.0.0.1:3128
>
> http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid3/certs/cert.crt
> key=/etc/squid3/certs/cert.key
>
> http_port 192.168.1.1:3129 intercept
>
> https_port 192.168.1.1:3130 intercept ssl-bump
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> cert=/etc/squid3/certs/cert.crt key=/etc/squid3/certs/cert.key
>
> sslcrtd_program /usr/lib/squid3/ssl_crtd -s /disk/dyn-certs/sslcrtd_db -M 4MB
> sslcrtd_children 5
>
> udp_incoming_address 192.168.1.1
> pinger_enable off
> forwarded_for delete
> via off
>
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> maximum_object_size_in_memory 1 MB
> minimum_object_size 0 KB
> maximum_object_size 64 MB
> memory_pools off
> cache_mem 256 MB
> cache_dir aufs /disk/squid-cache 25000 32 512
> cache_swap_low 95
> cache_swap_high 97
> ipcache_size 10240
> fqdncache_size 2048
> quick_abort_min 0 KB
> quick_abort_max 0 KB
> max_filedescriptors 4096
> read_ahead_gap 512 KB
>
> client_lifetime 6 hours
> connect_timeout 10 seconds
>
> log_icp_queries off
> buffered_logs on
> debug_options ALL,1
> logformat squid %tg %6tr %>A %Ss/%03>Hs UA=%{User-Agent}>h
> XFF=%{X-Forwarded-For}>h CKE=- %rm %ru %un %Sh/%<A %mt BYTES=%<st
> access_log stdio:/var/log/squid/access.log squid
> cache_log /var/log/squid/cache.log
> cache_store_log none #/var/log/squid/store.log
>
> icap_enable on
> icap_send_client_ip on
> icap_send_client_username on
> icap_client_username_encode off
> icap_client_username_header X-Authenticated-User
> icap_preview_enable on
> icap_preview_size 1024
> icap_service sqclamav_req reqmod_precache bypass=1
> icap://127.0.0.1:1344/squidclamav
> adaptation_access sqclamav_req allow all
> icap_service sqclamav_resp respmod_precache bypass=1
> icap://127.0.0.1:1344/squidclamav
> adaptation_access sqclamav_resp allow all
>
> refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp)$ 10080 90% 43200
> override-expire ignore-no-store ignore-private
> refresh_pattern -i \.(iso|avi|wav|mp3|m3u|mp4|mpeg|swf|flv|x-flv)$
> 43200 90% 259200 override-expire ignore-no-store ignore-no-cache
> ignore-private
> refresh_pattern -i
> \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|pptx|doc|docx|xls|xlsx|tiff)$
> 10080 90% 43200 override-expire ignore-no-store ignore-private
> refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
> refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
> refresh_pattern -i exe$ 0 50% 259200
> refresh_pattern -i zip$ 0 50% 259200
> refresh_pattern -i tar\.gz$ 0 50% 259200
> refresh_pattern -i tgz$ 0 50% 259200
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (\.cgi$|/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
>
> acl SSL_ports port 443 21 563 445 1494 2598 5006
> acl Safe_ports port 21 80 443 563 70 210 280 488 591 777 1025-65535
> acl purge method PURGE
> acl CONNECT method CONNECT
> acl FTP proto FTP
>
> acl lan src 192.168.1.0/24
> acl vpn src 192.168.12.0/24
> acl to_lan dst 192.168.1.0/24
> acl to_vpn dst 192.168.12.0/24
>
> acl malware_stinks dstdomain
> "/disk/squidguard/blacklist_acls/suspiciousdomains_Medium.txt"
> acl ad_regex url_regex -i
> "/disk/squidguard/blacklist_acls/ad_block_easylist.txt"
> acl privacy_regex url_regex -i
> "/disk/squidguard/blacklist_acls/privacy_easylist.txt"
> deny_info http://www.lan/malware.html malware_stinks
> deny_info http://www.lan/adregex.html ad_regex
> deny_info http://www.lan/privacy.html privacy_regex
>
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
>
> http_access deny malware_stinks
> http_access deny ad_regex
> http_access deny privacy_regex
>
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost purge
> http_access deny purge
> http_access allow FTP localhost
> http_access allow FTP lan
> http_access allow FTP vpn
> http_access allow lan
> http_access allow to_lan
> http_access allow vpn
> http_access allow localhost
>
> http_access deny all
>
> http_reply_access allow all
>
> icp_access allow lan
> icp_access allow vpn
> icp_access deny all
>
> snmp_access allow lan
> snmp_access allow vpn
> snmp_access deny all
>
> htcp_access allow lan
> htcp_access allow vpn
> htcp_access deny all
>
> always_direct allow lan
> always_direct allow vpn
>
> ssl_bump none localhost
> ssl_bump server-first lan
> ssl_bump server-first vpn
> sslproxy_cert_error allow all
> sslproxy_flags DONT_VERIFY_PEER
>
> request_header_replace Accept */*
> request_header_replace Accept-Encoding gzip
> request_header_replace Accept-Language en
> request_header_replace From
> request_header_replace Referer
> request_header_replace Cookie
> request_header_replace Cookie2
>
> request_header_add DNT 1
>
> request_header_access Allow allow all
> request_header_access Authorization allow all
> # corresponds to above replace acl
> request_header_access From deny !to_lan
> # corresponds to above replace acl
> request_header_access Referer deny !to_lan
> request_header_access WWW-Authenticate allow all
> request_header_access Proxy-Authorization allow all
> request_header_access Proxy-Authenticate allow all
> request_header_access Content-Encoding allow all
> request_header_access Content-Length allow all
> request_header_access Content-Type allow all
> request_header_access Date allow all
> request_header_access Expires allow all
> request_header_access Host allow all
> request_header_access If-Modified-Since allow all
> request_header_access Last-Modified allow all
> request_header_access Location allow all
> request_header_access Pragma allow all
> # corresponds to above replace acl
> request_header_access Accept deny !to_lan
> request_header_access Accept-Charset deny !to_lan
> # corresponds to above replace acl
> request_header_access Accept-Encoding deny !to_lan
> # corresponds to above replace acl
> request_header_access Accept-Language deny !to_lan
> request_header_access Content-Language allow all
> request_header_access Mime-Version allow all
> request_header_access Retry-After allow all
> request_header_access Title allow all
> request_header_access Connection allow all
> request_header_access Proxy-Connection allow all
> request_header_access User-Agent allow all
> request_header_access Other allow all
> request_header_access Cookie allow all
> request_header_access Cookie2 allow all
> # Corresponds to DNT header add/replace above
> request_header_access DNT deny all
> request_header_access All deny !to_lan
>
> reply_header_access Allow allow all
> reply_header_access Authorization allow all
> reply_header_access Proxy-Authorization allow all
> reply_header_access Proxy-Authenticate allow all
> reply_header_access Content-Encoding allow all
> reply_header_access Content-Length allow all
> reply_header_access Content-Type allow all
> reply_header_access Date allow all
> reply_header_access Expires allow all
> reply_header_access Host allow all
> reply_header_access If-Modified-Since allow all
> reply_header_access Last-Modified allow all
> reply_header_access Location allow all
> reply_header_access Pragma allow all
> reply_header_access Accept allow all
> reply_header_access Accept-Charset allow all
> reply_header_access Accept-Encoding allow all
> reply_header_access Accept-Language allow all
> reply_header_access Content-Language allow all
> reply_header_access Mime-Version allow all
> reply_header_access Retry-After allow all
> reply_header_access Title allow all
> reply_header_access Connection allow all
> reply_header_access User-Agent allow all
> reply_header_access Other allow all
> reply_header_access Set-Cookie allow all
> reply_header_access Set-Cookie2 allow all
> reply_header_access Transfer-Encoding allow all
> reply_header_access Strict-Transport-Security allow all
> reply_header_access DNT allow all
>
> reply_header_access WWW-Authenticate deny !to_lan
> reply_header_access Server deny !to_lan
> reply_header_access Link deny !to_lan
> reply_header_access All deny !to_lan
>
> ## SquidGuard
> # This sets up 10 SquidGuard programs; 5 ready-to-go; 10
> idled-and-prepped-to-go;
> # and threading for a total of 15 SquidGuard handlers available to
> tackle blocking
> # Note *do not* use the concurrency option; it crashes SquidGuard
> url_rewrite_program /usr/bin/squidGuard -c /etc/squid3/squidGuard.conf
> url_rewrite_children 15 startup=5 idle=10
> url_rewrite_bypass off
> url_rewrite_access allow all
> ##########END FILE################
Reply | Threaded
Open this post in threaded view
|

Re: Squid v3.3.8 & SSL Bumping Issues

Alex Rousskov
In reply to this post by David Marcos
On 07/08/2014 08:17 PM, David Marcos wrote:

>      b. HTTP Strict Transport Security (HSTS): Some pages flat-out
> reject any SSL bumping due to HSTS.  I am using Chrome, which I'm sure
> aggravates the issue.  Is there a way to configure Squid to get around
> HSTS?  (Yes, I know this may be a dumb question given how HSTS works,
> but would appreciate any insight.)


HSTS is an active area of research so I do not have final answers for
you, but my current understanding is:

a) HSTS itself is more-or-less compatible with SslBump. If you can
successfully convince an HTTP client to trust the Root certificate used
by Squid, then sites visited by that client will not violate any
standard HSTS rules.

b) Bumping errors unrelated to HSTS may be misinterpreted as
HSTS-related errors because the browser says "I cannot render that site
because of HSTS". What the browser means, in some cases, is that "I do
not trust that site [because there was a bumping problem] and HSTS rules
prevent me from showing you the sites I do not trust". In this
particular case, HSTS is mostly irrelevant. Once you fix the true cause
of distrust and, everything should work.

c) If a browser or browser plugin "pins" a certificate to a site, it
will not trust any other certificate for that site, possibly resulting
in HSTS errors. See item (b) above for why these are not actually HSTS
errors. In this case, there may be no solution -- you cannot force the
browser to unpin the certificate if that pinning was hard-coded.


Corrections welcomed!


HTH,

Alex.

Reply | Threaded
Open this post in threaded view
|

Re: Squid v3.4.6 SMP errors

talikarni
Running into this issue on one powerful system. OS (Scientific Linux
6.5) sees 16 CPU cores (which is 2 CPU sockets, each with 4 cores +
Hyperthreading). The unusual part is that this same setup works fine on
another system with dual core + HT using 3 workers.

I tried to setup the SMP options in squid.conf which work on other
systems but not this one. I first tried with 7 workers, then 3 but
neither worked, continued getting the error mentioned at the bottom of
this message. Only if I use the standard cache setup, it works without a
problem. I use odd numbers with the cpu_affinity_map so the parent-coord
can use the first core, and then the kids will be tied to the other
cores mentioned. This allows more single affinity processes to use the
first core as needed with minimal i/o impact.
The /var/cache/squid (and all subfolders) shows ownership as squid:squid
Also worth a mention: selinux is disabled.

Squid.conf basics with ssl-bump:

####
http_port 8080
# above port is what will be used for SSL Proxy on client browser
http_port 8081 intercept
https_port 8082 intercept ssl-bump connection-auth=off
generate-host-certificates=on dynamic_cert_mem_cache_size=16MB
cert=/etc/squid/ssl/squid.pem key=/etc/squid/ssl/squid.key
cipher=ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:DHE-RSA-CAMELLIA128-SHA:AES128-SHA:RC4-SHA:HIGH:!aNULL:!MD5:!ADH

sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid_ssl_db -M 16MB
sslcrtd_children 50 startup=5 idle=1
ssl_bump server-first all
ssl_bump none localhost
cache_log /var/log/squid/cache.log
cache_effective_user squid
debug_options ALL,0
logfile_rotate 10
cache_mgr [hidden email]
pinger_enable off
####
The SMP related items since everything else is fairly standard, here are
2 options I tried:

workers 3
cpu_affinity_map process_numbers=1,2,3 cores=2,3,4

workers 7
cpu_affinity_map process_numbers=1,2,3,4,5,6,7 cores=2,3,4,9,10,11,12

I used cores 2-4 and 9-12, since 5-8 is the first CPU Hyperthread cores.
CPU0 - core: 1-4, HT: 5-8
CPU1 - core: 9-12, HT: 13-16

and the related cache_dir entries, with "workers 7" had process number
up to 8 in the same manner (1 for coord, 7 for workers). Showing it as
commented since that is how it currently sits:

#if ${process_number} = 1
#cache_dir ufs /var/cache/squid/1 10000 32 512
#endif

#if ${process_number} = 2
#cache_dir ufs /var/cache/squid/2 10000 32 512
#endif

#if ${process_number} = 3
#cache_dir ufs /var/cache/squid/3 10000 32 512
#endif

#if ${process_number} = 4
#cache_dir ufs /var/cache/squid/4 10000 32 512
#endif


The error:

(squid-coord-8): Ipc::Mem::Segment::attach failed to
mmap(/squid-squid-page-pool.shm): (22) Invalid argument

Which then kills the squid kid processes resulting in "process 1234 will
not be restarted due to repeated, frequent failures"

Now I saw mentions on the squid page
http://wiki.squid-cache.org/Features/SmpScale
with this info, which did not work:

Add the following line to your */etc/fstab file*:

shm        /dev/shm    tmpfs    nodev,nosuid,noexec    0    0

After that use (as root):

mount shm


The only other thing I can think of is for process_numbers, does that
need to count to workers +1 (for the coord/parent)? So 4 or 8 in my
case? I have it as 3 on another working system with no problems.
Any help is greatly appreciated.

Mike
Reply | Threaded
Open this post in threaded view
|

Re: Squid v3.3.8 & SSL Bumping Issues

David Marcos
In reply to this post by Alex Rousskov
Alex, et al,

Thanks very much for the suggestions.  The tip-off that HSTS issues
may actually be a symptom, not the problem, was key.  Turns out I did
not properly install my self-signed root certificate into my laptop.
Once I fixed that, everything started working.

Thanks again for the help!

   -Dave

On Wed, Jul 9, 2014 at 1:59 PM, Alex Rousskov
<[hidden email]> wrote:

> On 07/08/2014 08:17 PM, David Marcos wrote:
>
>>      b. HTTP Strict Transport Security (HSTS): Some pages flat-out
>> reject any SSL bumping due to HSTS.  I am using Chrome, which I'm sure
>> aggravates the issue.  Is there a way to configure Squid to get around
>> HSTS?  (Yes, I know this may be a dumb question given how HSTS works,
>> but would appreciate any insight.)
>
>
> HSTS is an active area of research so I do not have final answers for
> you, but my current understanding is:
>
> a) HSTS itself is more-or-less compatible with SslBump. If you can
> successfully convince an HTTP client to trust the Root certificate used
> by Squid, then sites visited by that client will not violate any
> standard HSTS rules.
>
> b) Bumping errors unrelated to HSTS may be misinterpreted as
> HSTS-related errors because the browser says "I cannot render that site
> because of HSTS". What the browser means, in some cases, is that "I do
> not trust that site [because there was a bumping problem] and HSTS rules
> prevent me from showing you the sites I do not trust". In this
> particular case, HSTS is mostly irrelevant. Once you fix the true cause
> of distrust and, everything should work.
>
> c) If a browser or browser plugin "pins" a certificate to a site, it
> will not trust any other certificate for that site, possibly resulting
> in HSTS errors. See item (b) above for why these are not actually HSTS
> errors. In this case, there may be no solution -- you cannot force the
> browser to unpin the certificate if that pinning was hard-coded.
>
>
> Corrections welcomed!
>
>
> HTH,
>
> Alex.
>



--
___________________________________________________________
David J. Marcos
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Squid v3.4.6 SMP errors

Alex Rousskov
In reply to this post by talikarni
On 07/09/2014 02:42 PM, Mike wrote:

> (squid-coord-8): Ipc::Mem::Segment::attach failed to
> mmap(/squid-squid-page-pool.shm): (22) Invalid argument

If there are no other errors before that, try stracing Squid and its
kids (one strace file per kid). The problem may be happening _before_
the mmap() system call mentioned above, and the error code may be more
specific at that point. Post the relevant strace tail if you can.

BTW, cpu_affinity_map and the exact number of workers are most likely
unrelated to this issue. Something in your environment screws up shared
memory operations.


Cheers,

Alex.