Squid File descriptors warning

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid File descriptors warning

Alex K
Hi all,

I observed the following warning at squid cache logs:

WARNING! Your cache is running out of filedescriptors

Googling around I tried to increase the default file descriptors of the system (I am runnign Debian9 x64 bit), by setting at /etc/sysctl.conf:

fs.file-max=802762

Restarted system. Still was receiving the warnings.

When checking further I observed that I have the following default limits:


ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 15338
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 15338
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited



Where the "open files" seems to be the related one.

I set also the following at squid conf:

max_filedescriptors 65535

I am running a compiled version 3.5.23.

I am not sure I have done the correct steps or if I need to tweak the ulimits also.
Any experience from your side?

Thanx,
alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid File descriptors warning

Amos Jeffries
Administrator
On 07/08/18 19:45, Alex K wrote:
> Hi all,
>
> I observed the following warning at squid cache logs:
>
> WARNING! Your cache is running out of filedescriptors
>
> Googling around I tried to increase the default file descriptors of the
> system (I am runnign Debian9 x64 bit), by setting at /etc/sysctl.conf:
>
...
>
> I am running a compiled version 3.5.23.
>
> I am not sure I have done the correct steps or if I need to tweak the
> ulimits also.
> Any experience from your side?

The init script installed with Squid 3.x on Debian sets the limit to 64K
file descriptors.

What is Squid startup logging for file descriptors available?
 Probably in the system /var/log/messages log, if not cache.log.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid File descriptors warning

L.P.H. van Belle
In reply to this post by Alex K
I do know there is/was a bug the systemd isnt picking up the filedescriptors with systemd, you might have hit it.
Im suspecting your start script is a sysv script invoked by systemd.
 
Try to set the limits within the start script (sysv) so the correct users ( running squid ) gets the filedescriptors.
 
I run Debian 9, with a recompiled squid from debian sid and that work fine for me atm.
If you use the 4.1 from sid, add the following changes also.

/etc/logrotate.d/squid
        postrotate
            if [ -d /run/systemd/system ] && command systemctl >/dev/null 2>&1 && systemctl is-active --quiet squid.service; then
                systemctl restart squid.service
            elif [ -f /var/run/squid.pid ]; then
                test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || /usr/sbin/squid -k rotate
            fi
        endscript
 

Try the following and the below the command created the file /etc/systemd/system/squid.service.d/override.conf
Add there the following.
 
systemctl edit squid
[Unit]
After=network.target network-online.target nss-lookup.target
Wants=network-online.target
 
[Service]
LimitNOFILE=8192:65535
User=proxy
Group=proxy
 
 
Greetz,
 
Louis
 
 


Van: squid-users [mailto:[hidden email]] Namens Alex K
Verzonden: dinsdag 7 augustus 2018 9:46
Aan: [hidden email]
Onderwerp: [squid-users] Squid File descriptors warning

Hi all,

I observed the following warning at squid cache logs:

WARNING! Your cache is running out of filedescriptors

Googling around I tried to increase the default file descriptors of the system (I am runnign Debian9 x64 bit), by setting at /etc/sysctl.conf:

fs.file-max=802762

Restarted system. Still was receiving the warnings.

When checking further I observed that I have the following default limits:


ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 15338
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 15338
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited



Where the "open files" seems to be the related one.

I set also the following at squid conf:

max_filedescriptors 65535

I am running a compiled version 3.5.23.

I am not sure I have done the correct steps or if I need to tweak the ulimits also.
Any experience from your side?

Thanx,
alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid File descriptors warning

Amos Jeffries
Administrator
On 07/08/18 20:44, L.P.H. van Belle wrote:
> I do know there is/was a bug the systemd isnt picking up the
> filedescriptors with systemd, you might have hit it.
> Im suspecting your start script is a sysv script invoked by systemd.
>  

squid-3.5 builds are using sysV init.d scripts.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid File descriptors warning

Alex K
In reply to this post by Amos Jeffries


On Tue, Aug 7, 2018 at 11:26 AM, Amos Jeffries <[hidden email]> wrote:
On 07/08/18 19:45, Alex K wrote:
> Hi all,
>
> I observed the following warning at squid cache logs:
>
> WARNING! Your cache is running out of filedescriptors
>
> Googling around I tried to increase the default file descriptors of the
> system (I am runnign Debian9 x64 bit), by setting at /etc/sysctl.conf:
>
...
>
> I am running a compiled version 3.5.23.
>
> I am not sure I have done the correct steps or if I need to tweak the
> ulimits also.
> Any experience from your side?

The init script installed with Squid 3.x on Debian sets the limit to 64K
file descriptors. 

What is Squid startup logging for file descriptors available?
 Probably in the system /var/log/messages log, if not cache.log.
 
After having set the parameter at squid.conf I get:

With 65535 file descriptors available

Without settings it I get:

With 4096 file descriptors available

Which seems to be the value I have set at /etc/systemd/system.conf and /etc/systemd/user.conf:

DefaultLimitNOFILE=4096

Setting such values at /etc/security/limits.conf as below, doesn't seem to affect anything:

proxy            soft    nofile          4096
proxy            hard    nofile          1048576


Without settings any limits I get:

With 1024 file descriptors available

which seems to be reflecting the default value reported from ulimit -a.



Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid File descriptors warning

L.P.H. van Belle
In reply to this post by Amos Jeffries
Hai Amos,

Yes, true, but if systemd-sysv is installed, and it probley is, you might hit this bug.
Systemd is calling the sysv script.

I tried to find it, the bug report but ive to much things here trown at my head atm, sorry.

My production squid = 3.5.27 and no problems.  ( also rebuilded from sid at the time.)
The squid 4.1 from debian sid runs fine also.

My best advice, checkout that 4.1 example for systemd and start using that.


Greetz,

Louis



> -----Oorspronkelijk bericht-----
> Van: squid-users
> [mailto:[hidden email]] Namens
> Amos Jeffries
> Verzonden: dinsdag 7 augustus 2018 11:05
> Aan: [hidden email]
> Onderwerp: Re: [squid-users] Squid File descriptors warning
>
> On 07/08/18 20:44, L.P.H. van Belle wrote:
> > I do know there is/was a bug the systemd isnt picking up the
> > filedescriptors with systemd, you might have hit it.
> > Im suspecting your start script is a sysv script invoked by systemd.
> >  
>
> squid-3.5 builds are using sysV init.d scripts.
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users