SSLBump and squid process CPU usage

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

SSLBump and squid process CPU usage

masterx81
Hi!
I've configured squid with ssl_bump and now the squid process (not the
helpers) takes quite load. There aren't too much clients on it (max 50).
This is the config (ripped some acl to make it readable):

------------------------------------------------------

cache_mgr [hidden email]
visible_hostname        proxy.xxx.com
dns_v4_first on

authenticate_ip_ttl 1 hour

forward_max_tries 25

### negotiate kerberos and ntlm authentication
auth_param negotiate program /usr/local/bin/negotiate_wrapper --ntlm
/usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --domain=xxx
--kerberos /usr/local/bin/squid_kerb_auth -s GSS_C_NO_NAME
auth_param negotiate children 50
auth_param negotiate keep_alive off

### pure ntlm authentication
auth_param ntlm program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-ntlmssp --domain=xxx
auth_param ntlm children 50
auth_param ntlm keep_alive off

### provide basic authentication via ldap for clients not authenticated via
kerberos/ntlm
auth_param basic program /usr/local/squid/libexec/basic_ldap_auth -v 3 -R -b
"dc=xxx,dc=local" -D [hidden email] -W /etc/squid/ldappass.txt -f
sAMAccountName=%s -h srv-dc1.xxx.local

auth_param basic children 50
auth_param basic realm Proxy xxx

### ldap group authorisation
external_acl_type memberof ttl=30 %LOGIN
/usr/local/squid/libexec/ext_ldap_group_acl -v 3 -R -K -b "dc=xxx,dc=local"
-D [hidden email] -W  /etc/squid/ldappass.txt -f
"(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=%g,ou=SQUID,ou=OU
xxx,dc=xxx,dc=local))" -h srv-dc1.xxx.local

### acl for proxy auth and ldap authorizations
acl auth proxy_auth REQUIRED
#   aclname             acltype  typename activedirectorygroup
acl InternetBloccato    external memberof "/etc/squid/Internet_bloccato.txt"
... etc

acl bypass dstdomain somedomains
... etc

# ACL per Windows Update e microsoft
acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain sls.microsoft.com
acl windowsupdate dstdomain .windowsupdate.com
acl windowsupdate dstdomain productactivation.one.microsoft.com
acl windowsupdate dstdomain ntservicepack.microsoft.com
acl windowsupdate dstdomain .delivery.mp.microsoft.com

----a lot more ACL----

# ACL per bloccare per estensione
acl estensionibloccate urlpath_regex -i "/etc/squid/estensionibloccate.txt"

## Disable ssl interception for dropbox.com and hotmail.com (and localhost)
acl no_ssl_interception dstdomain somedomains

ssl_bump none localhost
ssl_bump none no_ssl_interception

ssl_bump stare
ssl_bump bump all


acl SSL_ports port 443
acl SSL_ports port 7071
acl SSL_ports port 10443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http

acl CONNECT method CONNECT
acl wuCONNECT dstdomain www.update.microsoft.com


# Permetti FTP
acl ftp proto FTP
acl ftp_port port 21

# ACL per limiti utenti Internet_limitato
acl giorni time T W F

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
ftp_epsv off

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

http_access allow ftp_port CONNECT
http_access allow ftp

http_access allow CONNECT wuCONNECT
http_access allow windowsupdate

---- a lot more ACL ----

# DO NOT REMOVE THE FOLLOWING LINE
http_access deny all



### logging
logformat useragent  %>a [%tl] "%{User-Agent}>h"

# don't log allowedsites, prioritysites, AnonymousAccess
access_log /var/log/squid/access.log logformat=squid
#!allowedsites !prioritysites !AnonymousAccess
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
cache_swap_log /var/log/squid/swap.log
logfile_rotate 10

# Squid normally listens to port 3128
#http_port 8080
http_port 8080 ssl-bump cert=/etc/squid/proxy.xxx.local.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
options=NO_SSLv3,NO_SSLv2 s$

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320


#
maximum_object_size 3000 KB

#Antivirus ClamAV
icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Authenticated-User
icap_service service_req reqmod_precache bypass=1
icap://127.0.0.1:1344/squidclamav
adaptation_access service_req allow all
icap_service service_resp respmod_precache bypass=1
icap://127.0.0.1:1344/squidclamav
adaptation_access service_resp allow all


------------------------------------------------------

the content of the "/etc/squid/estensionibloccate.txt" file is

------------------------------------------------------

\.exe(\?.*)?$
\.com(\?.*)?$
\.scr(\?.*)?$
\.cmd(\?.*)?$
\.bat(\?.*)?$
\.vbs(\?.*)?$

------------------------------------------------------

Locked for only some users via ACL, the acl is placed at the end, so that
only few users hit this acl

I've already increased the number of vcpu for the machine, but the only
process that i see eating cpu is squid, the helpers aren't eating a lot. I
see only sometimes the clamav service goind high on usage but i think that's
normal.
There is something that i miss or optimize in the config, or simply the
sslbump requires a lot of resources?

Thanks!



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSLBump and squid process CPU usage

Alex Rousskov
On 04/24/2018 06:14 AM, masterx81 wrote:

> I've configured squid with ssl_bump and now the squid process (not the
> helpers) takes quite load. There aren't too much clients on it (max 50).

> I've already increased the number of vcpu for the machine, but the only
> process that i see eating cpu is squid, the helpers aren't eating a lot.
> There is something that i miss or optimize in the config, or simply the
> sslbump requires a lot of resources?

I have not studied your configuration, but doing SSL encryption and/or
decryption (including the SslBump "bump" action) does require a lot of
CPU cycles. Enabling bumping may decrease sustained peak throughput by
70% or more.

If your users are suffering, and your machine has spare physical CPU
cores, consider using SMP Squid:
https://wiki.squid-cache.org/Features/SmpScale

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSLBump and squid process CPU usage

masterx81
Wow, a lot to read (and understand, for a newbie like me :-|)....
From what i've seen it's sufficient to insert the "workers n" directive in
the conf (n number of workers). With some limitations with the features that
support SMP (delay pools, cache, etc - i not think to use any of them)



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSLBump and squid process CPU usage

masterx81
For now i've tried with the "workers 3" directive, i can see 3 squid process,
seem that they span quite evenly the load and the page loading seem better.
Hope that fix the bottlenek...
In any case, i not know if there is somtheing wrong in the config that can
hurt the performance....



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSLBump and squid process CPU usage

Amos Jeffries
Administrator
On 27/04/18 03:19, masterx81 wrote:
> For now i've tried with the "workers 3" directive, i can see 3 squid process,
> seem that they span quite evenly the load and the page loading seem better.
> Hope that fix the bottlenek...
> In any case, i not know if there is somtheing wrong in the config that can
> hurt the performance....

Maybe yes, maybe no. The big performance drags are ICAP with extra TCP
resources requirements and delays, SSL-Bump with the TLS overheads, lots
of complex ACL processing, and regular network delays.

You mention having many ACLs but elided them so we cannot provide any
audit or hints to optimizing that part. The other parts you will have to
yourself test and check for what the actual delays are from each.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSLBump and squid process CPU usage

masterx81
By now i not see anymore the single squid process taking all the resources,
using the multi process the load is spread and all seem work really well. I
see only sometimes the clam-d service hitting 100% for few istants but i
think that is normal, as it's a single process, but not cause any slowdown.
The ACL that i've cut are only big lists of dstdomain (i think that not
require much cpu), and acl for some groups of users (time based ACL).
Nothing really intensive.
The only thing that i think can be intensive is the extension checking for
locking some users, but only few clients hit this ACL.



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users