squid stops replying

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

squid stops replying

Vieri
Hi,

After a long time working correctly, Squid stops working all of a sudden.

No new requests/replies show up in the logs. Complete silence.

If I issue "squid -k reconfigure" I get this message in cache.log:
Set Current Directory to /var/cache/squid

If I set "debug_options rotate=1 ALL,9" and run "squid -k reconfigure" twice then I get this in cache.log:

2017/08/23 07:54:32.676| 21,3| tools.cc(610) enter_suid: enter_suid: PID 17797 taking root privileges
2017/08/23 07:54:32.676| 13,3| mem.cc(473) Report: Memory pools are 'on'; limit: 5.000 MB
2017/08/23 07:54:32.676| Set Current Directory to /var/cache/squid
2017/08/23 07:54:32.676| 21,3| tools.cc(543) leave_suid: leave_suid: PID 17797 called
2017/08/23 07:54:32.676| 21,3| tools.cc(565) leave_suid: leave_suid: PID 17797 giving up root, becoming 'squid'
2017/08/23 07:55:01.605| 21,3| tools.cc(610) enter_suid: enter_suid: PID 17927 taking root privileges
2017/08/23 07:55:01.605| 13,3| mem.cc(473) Report: Memory pools are 'on'; limit: 5.000 MB
2017/08/23 07:55:01.605| Set Current Directory to /var/cache/squid
2017/08/23 07:55:01.605| 21,3| tools.cc(543) leave_suid: leave_suid: PID 17927 called
2017/08/23 07:55:01.605| 21,3| tools.cc(565) leave_suid: leave_suid: PID 17927 giving up root, becoming 'squid'

However, any attempt to browse the web leads to nothing new in the logs.

Finally, stopping the squid service fails. I can list the squid processes:

# ps -ae | grep squid
4439 ?        05:29:57 squid
9059 ?        00:00:00 squid
9160 ?        00:00:00 squid
9162 ?        00:11:42 squid
9206 ?        00:00:00 squid
9208 ?        00:02:04 squid
9254 ?        00:00:00 squid
9257 ?        00:00:28 squid
9313 ?        00:00:00 squid
9315 ?        00:00:55 squid

I have to kill these processes in order to start squid again.

# squid -version
Squid Cache: Version 3.5.26
Service Name: squid

This binary uses OpenSSL 1.0.2k  26 Jan 2017. For legal restrictions on distribution see https://www.openssl.org/source/license.html

configure options:  '--prefix=/usr' '--build=x86_64-pc-linux-gnu' '--host=x86_64-pc-linux-gnu' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--disable-dependency-tracking' '--disable-silent-rules' '--docdir=/usr/share/doc/squid-3.5.26' '--htmldir=/usr/share/doc/squid-3.5.26/html' '--libdir=/usr/lib64' '--sysconfdir=/etc/squid' '--libexecdir=/usr/libexec/squid' '--localstatedir=/var' '--with-pidfile=/run/squid.pid' '--datadir=/usr/share/squid' '--with-logdir=/var/log/squid' '--with-default-user=squid' '--enable-removal-policies=lru,heap' '--enable-storeio=aufs,diskd,rock,ufs' '--enable-disk-io' '--enable-auth-basic=MSNT-multi-domain,NCSA,POP3,getpwnam,SMB,LDAP,PAM,RADIUS' '--enable-auth-digest=file,LDAP,eDirectory' '--enable-auth-ntlm=smb_lm' '--enable-auth-negotiate=kerberos,wrapper' '--enable-external-acl-helpers=file_userip,session,unix_group,wbinfo_group,LDAP_group,eDirectory_userip,kerberos_ldap_group' '--enable-log-daemon-helpers' '--enable-url-rewrite-helpers' '--enable-cache-digests' '--enable-delay-pools' '--enable-eui' '--enable-icmp' '--enable-follow-x-forwarded-for' '--with-large-files' '--disable-strict-error-checking' '--disable-arch-native' '--with-ltdl-includedir=/usr/include' '--with-ltdl-libdir=/usr/lib64' '--with-libcap' '--enable-ipv6' '--disable-snmp' '--with-openssl' '--with-nettle' '--with-gnutls' '--enable-ssl-crtd' '--disable-ecap' '--disable-esi' '--enable-htcp' '--enable-wccp' '--enable-wccpv2' '--enable-linux-netfilter' '--with-mit-krb5' '--without-heimdal-krb5' 'build_alias=x86_64-pc-linux-gnu' 'host_alias=x86_64-pc-linux-gnu' 'CC=x86_64-pc-linux-gnu-gcc' 'CFLAGS=-O2 -pipe' 'LDFLAGS=-Wl,-O1 -Wl,--as-needed' 'CXXFLAGS=-O2 -pipe' 'PKG_CONFIG_PATH=/usr/lib64/pkgconfig'

What can I try if this happens again?

Thanks,

Vieri
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid stops replying

Amos Jeffries
Administrator
On 23/08/17 20:15, Vieri wrote:

> Hi,
>
> After a long time working correctly, Squid stops working all of a sudden.
>
> No new requests/replies show up in the logs. Complete silence.
>
> If I issue "squid -k reconfigure" I get this message in cache.log:
> Set Current Directory to /var/cache/squid
>
> If I set "debug_options rotate=1 ALL,9" and run "squid -k reconfigure" twice then I get this in cache.log:
>
> 2017/08/23 07:54:32.676| 21,3| tools.cc(610) enter_suid: enter_suid: PID 17797 taking root privileges
> 2017/08/23 07:54:32.676| 13,3| mem.cc(473) Report: Memory pools are 'on'; limit: 5.000 MB
> 2017/08/23 07:54:32.676| Set Current Directory to /var/cache/squid
> 2017/08/23 07:54:32.676| 21,3| tools.cc(543) leave_suid: leave_suid: PID 17797 called
> 2017/08/23 07:54:32.676| 21,3| tools.cc(565) leave_suid: leave_suid: PID 17797 giving up root, becoming 'squid'
> 2017/08/23 07:55:01.605| 21,3| tools.cc(610) enter_suid: enter_suid: PID 17927 taking root privileges
> 2017/08/23 07:55:01.605| 13,3| mem.cc(473) Report: Memory pools are 'on'; limit: 5.000 MB
> 2017/08/23 07:55:01.605| Set Current Directory to /var/cache/squid
> 2017/08/23 07:55:01.605| 21,3| tools.cc(543) leave_suid: leave_suid: PID 17927 called
> 2017/08/23 07:55:01.605| 21,3| tools.cc(565) leave_suid: leave_suid: PID 17927 giving up root, becoming 'squid'
>
> However, any attempt to browse the web leads to nothing new in the logs.

At ALL,9 that is a sign of major trouble. The log data is not going
where it should be. Please check your squid.conf that it is not sending
cache_log directive to /dev/null, a pipe or something.


>
> Finally, stopping the squid service fails. I can list the squid processes:
>
> # ps -ae | grep squid
> 4439 ?        05:29:57 squid
> 9059 ?        00:00:00 squid
> 9160 ?        00:00:00 squid
> 9162 ?        00:11:42 squid
> 9206 ?        00:00:00 squid
> 9208 ?        00:02:04 squid
> 9254 ?        00:00:00 squid
> 9257 ?        00:00:28 squid
> 9313 ?        00:00:00 squid
> 9315 ?        00:00:55 squid
>

That seems a lot. What exactly are those processes each doing?
  "ps aux | grep squid" should show the process roles.

If you run "squid -k shutdown ; squid -k shutdown" do they all fully stop?
(exactly that command, shutdown twice in a row)

Once Squid is fully stopped, start it again. Is the problem resolved
when it comes back up?

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid stops replying

Vieri
________________________________
From: Amos Jeffries <[hidden email]>
>
> At ALL,9 that is a sign of major trouble. The log data is not going

> where it should be. Please check your squid.conf that it is not sending
> cache_log directive to /dev/null, a pipe or something.
I do not define cache_log, so it should take the default value of "/var/log/squid/cache.log".

>> # ps -ae | grep squid
>> 4439 ?        05:29:57 squid
>> 9059 ?        00:00:00 squid
>> 9160 ?        00:00:00 squid
>> 9162 ?        00:11:42 squid
>> 9206 ?        00:00:00 squid
>> 9208 ?        00:02:04 squid
>> 9254 ?        00:00:00 squid
>> 9257 ?        00:00:28 squid
>> 9313 ?        00:00:00 squid
>> 9315 ?        00:00:55 squid
>
> That seems a lot. What exactly are those processes each doing?
>   "ps aux | grep squid" should show the process roles.


There are 4 reverse proxies, each one with their own squid conf file, and a caching HTTP/S proxy (tproxy).
The one failing is the latter.

Here's the process list (also includes external processes):

# ps aux | grep squid
squid     7139  7.9  0.6 276064 200472 ?       S    10:10   1:11 (squid-1) -YC -f /etc/squid/squid.conf
squid     7156  0.0  0.0   4012   640 ?        S    10:10   0:00 (unlinkd)
squid     7157  0.0  0.0  13904  3292 ?        S    10:10   0:00 diskd 7310340 7310341 7310342
squid    17857  0.0  0.0   4156   736 ?        S    10:20   0:00 (logfile-daemon) /var/log/squid/access.log
squid    17858  5.2  0.0  41780  7804 ?        S    10:20   0:15 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    17859  0.6  0.0  41792  8000 ?        S    10:20   0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    17860  0.0  0.0  41780  7800 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    17861  0.0  0.0  41776  7608 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    17862  0.0  0.0  41776  7612 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    17863  0.0  0.0  27204  9120 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    17864  0.0  0.0  27140  8872 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    17865  0.0  0.0  27140  8872 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    17866  0.0  0.0  27140  8912 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    17867  0.0  0.0  27140  8912 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    17868  0.8  0.0  68920 15488 ?        S    10:20   0:02 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    17869  0.1  0.0  68916 15380 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    17870  0.0  0.0  68904 15400 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    17871  0.0  0.0  68904 15484 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    17872  0.0  0.0  68792 15292 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    17873  0.0  0.0  19852  1812 ?        S    10:20   0:00 (pinger)
squid    18274  0.0  0.0  68792 15288 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    18405  0.0  0.0  68792 15288 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
root     19737  0.0  0.0  87440  5856 ?        Ss   Aug23   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf
squid    21262  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21263  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21264  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21265  0.0  0.0  68792 15292 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21266  0.0  0.0  68792 15292 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21267  0.0  0.0  68792 15384 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21268  0.0  0.0  68792 15396 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21269  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21270  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    21276  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    22971  0.0  0.0  41772  7604 ?        S    10:24   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
root     23964  0.0  0.0   8968   944 pts/0    S+   10:24   0:00 grep --colour=auto squid
root     27129  0.0  0.0  84856  5140 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf
squid    27132  0.0  0.0 100980 26652 ?        S    Aug24   0:04 (squid-1) -YC -f /etc/squid/squid.http.conf
squid    27134  0.0  0.0  19852  1716 ?        S    Aug24   0:03 (pinger)
root     27281  0.0  0.0  86912  5192 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf
squid    27284  0.0  0.0 106484 31408 ?        S    Aug24   0:13 (squid-1) -YC -f /etc/squid/squid.https.conf
squid    27290  0.0  0.0  19852  1816 ?        S    Aug24   0:03 (pinger)
root     27956  0.0  0.0  86908  5172 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf
squid    27958  0.0  0.0  99560 25016 ?        S    Aug24   1:24 (squid-1) -YC -f /etc/squid/squid.owa.conf
squid    27961  0.0  0.0  19852  1812 ?        S    Aug24   0:04 (pinger)
root     28106  0.0  0.0  86908  5188 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf
squid    28109  0.0  0.2 141024 66372 ?        S    Aug24   0:34 (squid-1) -YC -f /etc/squid/squid.owa2.conf
squid    28113  0.0  0.0  19852  1716 ?        S    Aug24   0:04 (pinger)


> If you run "squid -k shutdown ; squid -k shutdown" do they all fully stop?> (exactly that command, shutdown twice in a row)
>
> Once Squid is fully stopped, start it again. Is the problem resolved
> when it comes back up?


I'd have to wait for Squid to stop replying. That usually takes several days, maybe more than a week.
The failing squid process is currently set up with:
debug_options rotate=1 ALL,1

Should I set a different level BEFORE it "stops working", ie. "now"?
I'm asking because it's going to take a long while to reproduce this issue, and I just want to make sure I'll have enough info when it happens.

Thanks,

Vieri
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid stops replying

Amos Jeffries
Administrator
On 25/08/17 20:34, Vieri wrote:

> ________________________________
> From: Amos Jeffries
>>
>> At ALL,9 that is a sign of major trouble. The log data is not going
>
>> where it should be. Please check your squid.conf that it is not sending
>> cache_log directive to /dev/null, a pipe or something.
> I do not define cache_log, so it should take the default value of "/var/log/squid/cache.log".
>
>>> # ps -ae | grep squid
>>> 4439 ?        05:29:57 squid
>>> 9059 ?        00:00:00 squid
>>> 9160 ?        00:00:00 squid
>>> 9162 ?        00:11:42 squid
>>> 9206 ?        00:00:00 squid
>>> 9208 ?        00:02:04 squid
>>> 9254 ?        00:00:00 squid
>>> 9257 ?        00:00:28 squid
>>> 9313 ?        00:00:00 squid
>>> 9315 ?        00:00:55 squid
>>
>> That seems a lot. What exactly are those processes each doing?
>>    "ps aux | grep squid" should show the process roles.
>
>
> There are 4 reverse proxies, each one with their own squid conf file, and a caching HTTP/S proxy (tproxy).
> The one failing is the latter.

cache_log is one of the Squid directives which require a unique file
path+name per instance when running multiple instances.

As the wiki page <https://wiki.squid-cache.org/MultipleInstances> says,
it *might* work but the result is *probably* a garbled mess. And in your
case I'm relatively sure that some critical information is being
delivered into nul-space.

Also there will be issues with shared memory and such things on current
Squid if you run multiple instances on one machine without proper
instance naming - this will happen for some things even if you are not
using SMP workers.
  Add the -n (lower case) command line option to your startup script(s),
with a unique instance name for each.

First thing to do is fix the above issues. Then whatever the pause is
about will be much easier to track down.


>
> Here's the process list (also includes external processes):
>
> # ps aux | grep squid
> squid     7139  7.9  0.6 276064 200472 ?       S    10:10   1:11 (squid-1) -YC -f /etc/squid/squid.conf
> squid     7156  0.0  0.0   4012   640 ?        S    10:10   0:00 (unlinkd)
> squid     7157  0.0  0.0  13904  3292 ?        S    10:10   0:00 diskd 7310340 7310341 7310342
> squid    17857  0.0  0.0   4156   736 ?        S    10:20   0:00 (logfile-daemon) /var/log/squid/access.log
> squid    17858  5.2  0.0  41780  7804 ?        S    10:20   0:15 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> squid    17859  0.6  0.0  41792  8000 ?        S    10:20   0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> squid    17860  0.0  0.0  41780  7800 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> squid    17861  0.0  0.0  41776  7608 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> squid    17862  0.0  0.0  41776  7612 ?        S    10:20   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> squid    17863  0.0  0.0  27204  9120 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
> squid    17864  0.0  0.0  27140  8872 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
> squid    17865  0.0  0.0  27140  8872 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
> squid    17866  0.0  0.0  27140  8912 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
> squid    17867  0.0  0.0  27140  8912 ?        S    10:20   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
> squid    17868  0.8  0.0  68920 15488 ?        S    10:20   0:02 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    17869  0.1  0.0  68916 15380 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    17870  0.0  0.0  68904 15400 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    17871  0.0  0.0  68904 15484 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    17872  0.0  0.0  68792 15292 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    17873  0.0  0.0  19852  1812 ?        S    10:20   0:00 (pinger)
> squid    18274  0.0  0.0  68792 15288 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    18405  0.0  0.0  68792 15288 ?        S    10:20   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> root     19737  0.0  0.0  87440  5856 ?        Ss   Aug23   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf
> squid    21262  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21263  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21264  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21265  0.0  0.0  68792 15292 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21266  0.0  0.0  68792 15292 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21267  0.0  0.0  68792 15384 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21268  0.0  0.0  68792 15396 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21269  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21270  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    21276  0.0  0.0  68792 15400 ?        S    10:22   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
> squid    22971  0.0  0.0  41772  7604 ?        S    10:24   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
> root     23964  0.0  0.0   8968   944 pts/0    S+   10:24   0:00 grep --colour=auto squid
> root     27129  0.0  0.0  84856  5140 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf
> squid    27132  0.0  0.0 100980 26652 ?        S    Aug24   0:04 (squid-1) -YC -f /etc/squid/squid.http.conf
> squid    27134  0.0  0.0  19852  1716 ?        S    Aug24   0:03 (pinger)
> root     27281  0.0  0.0  86912  5192 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf
> squid    27284  0.0  0.0 106484 31408 ?        S    Aug24   0:13 (squid-1) -YC -f /etc/squid/squid.https.conf
> squid    27290  0.0  0.0  19852  1816 ?        S    Aug24   0:03 (pinger)
> root     27956  0.0  0.0  86908  5172 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf
> squid    27958  0.0  0.0  99560 25016 ?        S    Aug24   1:24 (squid-1) -YC -f /etc/squid/squid.owa.conf
> squid    27961  0.0  0.0  19852  1812 ?        S    Aug24   0:04 (pinger)
> root     28106  0.0  0.0  86908  5188 ?        Ss   Aug24   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf
> squid    28109  0.0  0.2 141024 66372 ?        S    Aug24   0:34 (squid-1) -YC -f /etc/squid/squid.owa2.conf
> squid    28113  0.0  0.0  19852  1716 ?        S    Aug24   0:04 (pinger)
>
>
>> If you run "squid -k shutdown ; squid -k shutdown" do they all fully stop?> (exactly that command, shutdown twice in a row)
>>
>> Once Squid is fully stopped, start it again. Is the problem resolved
>> when it comes back up?
>
>
> I'd have to wait for Squid to stop replying. That usually takes several days, maybe more than a week.

Okay. The above test was to see if the squid processes were all
correctly operating.
  Since it is now clear that you have multiple instances stepping over
each others feet in the background please fix the issues with -n and the
multi-instance critical directives. *Then* do the above to ensure that
all instances are fully using the fixed setup.

If/when the problem occurs again you should be better placed to see what
is going on.

  * crash recovery can be very slow on caching Squid as the cache_dir
require a full disk scan to load all the objects back in.

  * a suddenly overloaded Squid that is using a lot of memory (ie huge
disk cache index) can take a lot of time to startup new dynamic helpers.

Both of the above can result in user-visible service pauses. There is
not much that can be done about it except workaround the cause of the
delay (better peak configuration, or patching the crash not to happen).




> The failing squid process is currently set up with:
> debug_options rotate=1 ALL,1
>
> Should I set a different level BEFORE it "stops working", ie. "now"?
> I'm asking because it's going to take a long while to reproduce this issue, and I just want to make sure I'll have enough info when it happens.
>

Probably not then. ALL,1 is sufficient to show crash issues if they
occur, and if nothing at all appears in the log later you can strace the
paused process to see what it is doing and maybe send it a "squid -n
_instance_name -d 9 " debug level change from the command line (if what
it is doing allows signals to get through before it is finished).


PS; are you using CentOS 6 or an older RHEL with kernel 2.6.x ?
  It has recently just come up that they are still actively using those
old kernels containing broken TPROXY implementations. I vaguely recall
the symptoms are not always immediately visible, but triggered by
certain ICMP or TCP conditions.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid stops replying

Vieri
In reply to this post by Amos Jeffries

________________________________
From: Amos Jeffries <[hidden email]>

>

> cache_log is one of the Squid directives which require a unique file path+name per instance when running multiple instances.


I defined a unique name for this directive for each squid service (squid.conf used by the failing caching proxy: not defined -> default value):

# grep cache_log squid.*.conf
squid.http.conf:cache_log /var/log/squid/cache.http.log
squid.https.conf:cache_log /var/log/squid/cache.https.log
squid.owa.conf:cache_log /var/log/squid/cache.owa.log
squid.owa2.conf:cache_log /var/log/squid/cache.owa2.log

# grep pid_filename squid.*.conf
squid.http.conf:pid_filename /run/squid.http.pid
squid.https.conf:pid_filename /run/squid.https.pid
squid.owa.conf:pid_filename /run/squid.owa.pid
squid.owa2.conf:pid_filename /run/squid.owa2.pid

# grep access_log squid.*.conf
squid.http.conf:access_log none
squid.https.conf:access_log none
squid.owa.conf:access_log none
squid.owa2.conf:access_log none

I also just added unique_hostname for all reverse proxies, but did not define it for the "failing" proxy cache:
unique_hostname rev_https
unique_hostname rev_http
unique_hostname rev_owa
unique_hostname rev_owa2

The man page states:

-n name    Specify Windows Service name to use for service operations,
default is: Squid

I didn't know it could be used in Linux.
So I set it up as you suggested (thanks for pointing me to the Multiple Instances wiki page).

I modified my init scripts with the -n option. I also checked the squid logs to confirm that the service names were properly set.
eg. for one of the reverse proxies: kid1| Service Name: squidhttp

So, after applying everything:

# ps aux | grep squid
root     19635  0.0  0.0  84856  5116 ?        Ss   14:41   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf -n squidhttp
squid    19639  0.0  0.0  93452 19184 ?        S    14:41   0:00 (squid-1) -YC -f /etc/squid/squid.http.conf -n squidhttp
squid    19641  0.0  0.0  19852  1812 ?        S    14:41   0:00 (pinger)
root     20019  0.0  0.0  86912  5168 ?        Ss   14:42   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf -n squidhttps
squid    20022  0.0  0.0  95372 19588 ?        S    14:42   0:00 (squid-1) -YC -f /etc/squid/squid.https.conf -n squidhttps
squid    20024  0.0  0.0  19852  1716 ?        S    14:42   0:00 (pinger)
root     20615  0.0  0.0  86908  5164 ?        Ss   14:44   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf -n squidowa
squid    20617  0.1  0.0  98076 23000 ?        S    14:44   0:06 (squid-1) -YC -f /etc/squid/squid.owa.conf -n squidowa
squid    20620  0.0  0.0  19852  1720 ?        S    14:44   0:00 (pinger)
root     20978  0.0  0.0  86908  5180 ?        Ss   14:45   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid    20981  0.0  0.0  98420 22852 ?        S    14:45   0:01 (squid-1) -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid    20983  0.0  0.0  19852  1812 ?        S    14:45   0:00 (pinger)
root     22620  0.0  0.0  87440  5160 ?        Ss   Aug25   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid
squid    28916  0.0  0.0   4156   644 ?        S    16:25   0:00 (logfile-daemon) /var/log/squid/access.log
squid    28917  1.5  0.0  41788  7996 ?        S    16:25   0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    28918  0.1  0.0  41772  7596 ?        S    16:25   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    28919  0.0  0.0  41772  7604 ?        S    16:25   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    28920  0.0  0.0  40444  5176 ?        S    16:25   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    28921  0.0  0.0  40444  5364 ?        S    16:25   0:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    28922  0.0  0.0  27204  9156 ?        S    16:25   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    28923  0.0  0.0  27140  8912 ?        S    16:25   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    28924  0.0  0.0  27140  8872 ?        S    16:25   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    28925  0.0  0.0  27140  8912 ?        S    16:25   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    28926  0.0  0.0  27140  8872 ?        S    16:25   0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    28927  0.4  0.0  68916 15468 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28928  0.1  0.0  68792 15384 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28929  0.0  0.0  68792 15292 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28930  0.0  0.0  68792 15400 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28931  0.0  0.0  68792 15284 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28932  0.0  0.0  19852  1716 ?        S    16:25   0:00 (pinger)
squid    28939  0.0  0.0  68792 15396 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    28942  0.0  0.0  68792 15400 ?        S    16:25   0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    29505  1.2  0.7 326680 251396 ?       S    10:50   4:10 (squid-1) -YC -f /etc/squid/squid.conf -n squid
squid    29522  0.0  0.0   4008   632 ?        S    10:50   0:00 (unlinkd)
squid    29523  0.0  0.0  13904  3292 ?        S    10:50   0:02 diskd 30213124 30213125 30213126

If the issue comes back up again I will run the following:

squid -n squid -d 9
strace -o squid.strace /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid

...followed by
squid -n squid -k shutdown
several times until I desperately kill -s 9.

I'm not using CentOS.
I'm using Gentoo with kernel 4.9.34.


Well, I'm going to sit and wait now... Thanks.


Vieri
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid stops replying

Vieri
In reply to this post by Amos Jeffries
Good news. My Squid process "failed" earlier than expected.

This is one of the first warning messages I see in the log when it happens:
WARNING! Your cache is running out of filedescriptors

# squidclient mgr:info | grep 'file descri'
Maximum number of file descriptors:  1024
Available number of file descriptors:  255
Reserved number of file descriptors:  100

# ulimit -a
core file size          (blocks, -c) 0
data seg size          (kbytes, -d) unlimited
scheduling priority            (-e) 0
file size              (blocks, -f) unlimited
pending signals                (-i) 127521
max locked memory      (kbytes, -l) 64
max memory size        (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues    (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time              (seconds, -t) unlimited
max user processes              (-u) 127521
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

I'm not sure whether this is the root cause, and if I need to change it with something like:

echo '*              hard    nofile          8192' >> /etc/security/limits.conf
echo '*              soft    nofile          4096' >> /etc/security/limits.conf

I'm not even sure how to determine which values I need.

I'm getting high CPU usage during the "failure" such as:
PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM    TIME+ COMMAND
20010 squid    20  0 1050808 878424  13152 R 100.0  2.7  80:06.23 squid

8430 root      20  0  76524  50096  3632 R 100.0  0.2  0:34.85 perl
20010 squid    20  0 1052948 900864  13152 R 100.0  2.7  81:22.71 squid

However, I don't think it's too high because a reading with top + 1 shows values such as:

load average: 1.59, 1.73, 1.94
Tasks: 342 total,  3 running, 339 sleeping,  0 stopped,  0 zombie
%Cpu0  : 58.3 us,  1.0 sy,  0.0 ni, 39.1 id,  0.0 wa,  0.0 hi,  1.7 si,  0.0 st
%Cpu1  :  4.3 us,  0.0 sy,  2.3 ni, 90.7 id,  0.0 wa,  0.0 hi,  2.6 si,  0.0 st
%Cpu2  : 10.9 us,  0.3 sy,  4.3 ni, 82.1 id,  0.0 wa,  0.0 hi,  2.3 si,  0.0 st
%Cpu3  :  5.3 us,  0.7 sy,  0.7 ni, 91.1 id,  0.0 wa,  0.0 hi,  2.3 si,  0.0 st
%Cpu4  :  6.6 us,  0.3 sy,  0.7 ni, 88.4 id,  0.0 wa,  0.0 hi,  4.0 si,  0.0 st
%Cpu5  :  3.3 us,  0.3 sy,  0.0 ni, 95.0 id,  0.0 wa,  0.0 hi,  1.3 si,  0.0 st
%Cpu6  : 10.6 us,  2.0 sy,  1.0 ni, 85.1 id,  0.0 wa,  0.0 hi,  1.3 si,  0.0 st
%Cpu7  :  6.3 us,  0.7 sy,  0.7 ni, 80.5 id,  0.3 wa,  0.0 hi, 11.6 si,  0.0 st
KiB Mem : 32865056 total, 12159836 free,  5675908 used, 15029312 buff/cache
KiB Swap: 37036988 total, 36947156 free,    89832 used. 26689368 avail Mem

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM    TIME+ COMMAND
21118 squid    20  0  253520 166112  12856 R  80.8  0.5  2:19.69 squid

Anyway, here's a process list during the "failure":

# ps aux | grep squid
squid    1882  0.0  0.0  41796  7880 ?        S    10:23  0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    1883  0.0  0.0  41804  8016 ?        S    10:23  0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    1884  0.0  0.0  41792  7876 ?        S    10:23  0:01 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    2043  0.0  0.0  41788  7964 ?        S    09:28  0:02 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
root      9100  0.0  0.0  8968  844 pts/0    S+  12:26  0:00 grep --colour=auto squid
squid    10375  0.0  0.0  4156  740 ?        S    08:50  0:07 (logfile-daemon) /var/log/squid/access.log
squid    10376  2.4  0.0  41788  7996 ?        S    08:50  5:22 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    10377  0.4  0.0  41780  7992 ?        S    08:50  1:00 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    10378  0.1  0.0  41800  8012 ?        S    08:50  0:22 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    10379  0.0  0.0  41788  7872 ?        S    08:50  0:11 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    10380  0.0  0.0  41788  7876 ?        S    08:50  0:06 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB
squid    10381  0.0  0.0  27140  8912 ?        S    08:50  0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    10382  0.0  0.0  27140  8872 ?        S    08:50  0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    10383  0.0  0.0  27204  8904 ?        S    08:50  0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    10384  0.0  0.0  27204  9156 ?        S    08:50  0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    10385  0.0  0.0  27204  9112 ?        S    08:50  0:00 /usr/bin/perl -w /usr/libexec/squid/ext_wbinfo_group_acl -K
squid    10386  0.2  0.0  68904 15368 ?        S    08:50  0:36 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    10387  0.0  0.0  68912 15400 ?        S    08:50  0:05 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    10388  0.0  0.0  68920 15396 ?        S    08:50  0:02 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    10389  0.0  0.0  68904 15400 ?        S    08:50  0:01 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    10390  0.0  0.0  68920 15292 ?        S    08:50  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    10391  0.0  0.0  19852  1816 ?        S    08:50  0:01 (pinger)
squid    10406  0.0  0.0  68920 15292 ?        S    08:50  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12021  0.0  0.0  68920 15292 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12022  0.0  0.0  68904 15396 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12023  0.0  0.0  68792 15396 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12024  0.0  0.0  68908 15292 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12025  0.0  0.0  68792 15400 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    12026  0.0  0.0  68792 15288 ?        S    08:53  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    13535  0.0  0.0  68792 15120 ?        S    11:30  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    13536  0.0  0.0  68792 15104 ?        S    11:30  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    13537  0.0  0.0  68792 15140 ?        S    11:30  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    13538  0.0  0.0  68792 15072 ?        S    11:30  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    13539  0.0  0.0  68792 15064 ?        S    11:30  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
root    19635  0.0  0.0  84856  5116 ?        Ss  Aug27  0:00 /usr/sbin/squid -YC -f /etc/squid/squid.http.conf -n squidhttp
squid    19639  0.0  0.0 100572 26420 ?        S    Aug27  0:03 (squid-1) -YC -f /etc/squid/squid.http.conf -n squidhttp
squid    19641  0.0  0.0  19852  1812 ?        S    Aug27  0:02 (pinger)
squid    20010 32.0  2.7 1052948 900864 ?      R    08:10  81:59 (squid-1) -YC -f /etc/squid/squid.conf -n squid
root    20019  0.0  0.0  86912  5168 ?        Ss  Aug27  0:00 /usr/sbin/squid -YC -f /etc/squid/squid.https.conf -n squidhttps
squid    20022  0.0  0.0  99100 23484 ?        S    Aug27  0:06 (squid-1) -YC -f /etc/squid/squid.https.conf -n squidhttps
squid    20024  0.0  0.0  19852  1716 ?        S    Aug27  0:02 (pinger)
squid    20030  0.0  0.0  4012  732 ?        S    08:10  0:00 (unlinkd)
squid    20035  0.0  0.0  13904  3080 ?        S    08:10  0:05 diskd 20490244 20490245 20490246
root    20615  0.0  0.0  86908  5164 ?        Ss  Aug27  0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa.conf -n squidowa
squid    20617  0.1  0.1 118488 43868 ?        S    Aug27  1:20 (squid-1) -YC -f /etc/squid/squid.owa.conf -n squidowa
squid    20620  0.0  0.0  19852  1720 ?        S    Aug27  0:02 (pinger)
root    20978  0.0  0.0  86908  5180 ?        Ss  Aug27  0:00 /usr/sbin/squid -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid    20981  0.0  0.1 123380 48140 ?        S    Aug27  0:15 (squid-1) -YC -f /etc/squid/squid.owa2.conf -n squidowa2
squid    20983  0.0  0.0  19852  1812 ?        S    Aug27  0:02 (pinger)
root    22620  0.0  0.0  87440  5160 ?        Ss  Aug25  0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid
squid    23415  0.0  0.0  68792 15288 ?        S    10:58  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    23416  0.0  0.0  68792 15384 ?        S    10:58  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    23417  0.0  0.0  68792 15280 ?        S    10:58  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    23418  0.0  0.0  68792 15288 ?        S    10:58  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    23419  0.0  0.0  68792 15292 ?        S    10:58  0:00 /usr/bin/perl -w -s /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl -tbl_name=shallalist_bl adv,aggressive,alcohol,anonvpn,automobile_bikes,automobile_boats,automobile_cars,automobile_planes,chat,costtraps,dating,drugs,dynamic,finance_insurance,finance_moneylending,finance_other,finance_realestate,finance_trading,fortunetelling,forum,gamble,hacking,hobby_cooking,hobby_games-misc,hobby_games-online,hobby_gardening,hobby_pets,homestyle,imagehosting,isp,jobsearch,military,models,movies,music,podcasts,politics,porn,radiotv,recreation_humor,recreation_martialarts,recreation_restaurants,recreation_sports,recreation_travel,recreation_wellness,redirector,religion,remotecontrol,ringtones,science_astronomy,science_chemistry,sex_education,sex_lingerie,shopping,socialnet,spyware,tracker,updatesites,urlshortener,violence,warez,weapons,webphone,webradio,webtv
squid    29575  0.0  0.0  41800  7884 ?        S    09:18  0:04 (ssl_crtd) -s /var/lib/squid/ssl_db -M 16MB

I saved the output of 'strace /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid' if you require it.

The command "squid -n squid -d 9" doesn't seem to change the logging output, at least not immediately.

I see lines such as the following in the log:

kid1| BUG: Unexpected state while connecting to a cache_peer or origin server
kid1| WARNING! Your cache is running out of filedescriptors
kid1| WARNING: 10 pending requests queued
kid1| Queue overload, rejecting
kid1| WARNING: All 10/10 ssl_crtd processes are busy.
kid1| WARNING: Consider increasing the number of ssl_crtd processes in your config file.
kid1| comm_open: socket failure: (24) Too many open files
kid1| helperDispatch: invalid callback data

This is the sslcrtd_program I'm using:

sslcrtd_program /usr/libexec/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 16MB
sslcrtd_children 10

I don't remember why I set it to 10. Probably a bad idea since the default is "sslcrtd_children 32 startup=5 idle=1".

So, I'm going to either comment out sslcrtd_children, or change it to:

sslcrtd_children 64 startup=10 idle=2

Could this alone explain the issue I'm seeing?

Vieri
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users