Cache is running out of filedescriptors

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Cache is running out of filedescriptors

Sathyan, Arjonan

Hi,

I am receiving more warnings on my cache.log

"2007/07/20 10:26:18| WARNING! Your cache is running out of
filedescriptors"

We used to increase the Number File descriptor value whenever we receive
such errors.

Current File descriptor value,

"max_open_disk_fds 32768"

[root@ussvlpxy02 root]# ulimit -a
core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) 4
max memory size       (kbytes, -m) unlimited
open files                    (-n) 32768
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) 10240
cpu time             (seconds, -t) unlimited
max user processes            (-u) 7168
virtual memory        (kbytes, -v) unlimited

My squid version,

[root@ussvlpxy02 root]# /usr/local/squid/sbin/squid -v
Squid Cache: Version 2.5.STABLE13
configure options:  --enable-smartfilter

But even after increasing these values I am getting the same error.
Please let me know what I should do to avoid these warnings.

Regards,
Sathyan Arjunan
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Tek Bahadur Limbu
Sathyan, Arjonan wrote:

> Hi,
>
> I am receiving more warnings on my cache.log
>
> "2007/07/20 10:26:18| WARNING! Your cache is running out of
> filedescriptors"
>
> We used to increase the Number File descriptor value whenever we receive
> such errors.
>
> Current File descriptor value,
>
> "max_open_disk_fds 32768"
>
> [root@ussvlpxy02 root]# ulimit -a
> core file size        (blocks, -c) 0
> data seg size         (kbytes, -d) unlimited
> file size             (blocks, -f) unlimited
> max locked memory     (kbytes, -l) 4
> max memory size       (kbytes, -m) unlimited
> open files                    (-n) 32768
> pipe size          (512 bytes, -p) 8
> stack size            (kbytes, -s) 10240
> cpu time             (seconds, -t) unlimited
> max user processes            (-u) 7168
> virtual memory        (kbytes, -v) unlimited
>
> My squid version,
>
> [root@ussvlpxy02 root]# /usr/local/squid/sbin/squid -v
> Squid Cache: Version 2.5.STABLE13
> configure options:  --enable-smartfilter
>
> But even after increasing these values I am getting the same error.
> Please let me know what I should do to avoid these warnings.

Hi Sathyan,

I am just pasting from my earlier post. By the way, which OS are you
running Squid?

You need to increase the size of _FD_SETSIZE in your
/usr/include/bits/typesizes.h file.

After increasing the above value to 8192, you type ulimit -HSn 8192 and
compile Squid.


I suggest you to upgrade to the latest version of Squid for many
benefits regarding performance among others?

Please see the links below and see if it helps.


http://wiki.squid-cache.org/SquidFaq/CompleteFaq#head-eb3240fe8e61368056af86138a2b5dcbc9781a54

http://www.mail-archive.com/squid-users@.../msg35858.html



Thanking you...

>
> Regards,
> Sathyan Arjunan
>
>
>


--

With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Henrik Nordström
On lör, 2007-07-21 at 00:13 +0545, Tek Bahadur Limbu wrote:

> You need to increase the size of _FD_SETSIZE in your
> /usr/include/bits/typesizes.h file.

No you don't. Hasn't been needed since Squid-2.5.something (don't
remember exact version, many many years ago).

> After increasing the above value to 8192, you type ulimit -HSn 8192 and
> compile Squid.

This, or --max-fd=NN configure option is needed however. And you also
need to raise the ulimit when Squid is started.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Cache is running out of filedescriptors

Sathyan, Arjonan



>> After increasing the above value to 8192, you type ulimit -HSn 8192
and
>> compile Squid.

> This, or --max-fd=NN configure option is needed however. And you also
> need to raise the ulimit when Squid is started.

So I need to recompile my Squid like,

Step 1) Start the recompilation procedures,
                ./configure --with-maxfd=8192 --enable-smartfilter

Step 2) Edit squid.conf, max_open_disk_fds=8192

Step 3) ulimit -HSn 8192

Kindly correct me if I am wrong...

Regards,
Sathyan Arjunan
Reply | Threaded
Open this post in threaded view
|

RE: Cache is running out of filedescriptors

Henrik Nordström
On fre, 2007-07-20 at 14:52 -0700, Sathyan, Arjonan wrote:

> So I need to recompile my Squid like,
>
> Step 1) Start the recompilation procedures,
> ./configure --with-maxfd=8192 --enable-smartfilter

yes, but first you should verify with Smartfilter that they support
this.. No support for Smartfilter is provided on this list as it's a
closed source extension to Squid and we can't tell what it's doing or
how it will react to changes in Squid.

> Step 2) Edit squid.conf, max_open_disk_fds=8192

There is no such squid.conf parameter..

> Step 3) ulimit -HSn 8192

Yes.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Cache is running out of filedescriptors

Sathyan, Arjonan


>> So I need to recompile my Squid like,
>>
>> Step 1) Start the recompilation procedures,
>> ./configure --with-maxfd=8192 --enable-smartfilter

> yes, but first you should verify with Smartfilter that they support
> this.. No support for Smartfilter is provided on this list as it's a
> closed source extension to Squid and we can't tell what it's doing or
> how it will react to changes in Squid.

Yes... I am sure this version supports Smartfilter because we have a
identical server running on the same version of squid + smartfilter

>> Step 2) Edit squid.conf, max_open_disk_fds=8192

> There is no such squid.conf parameter..

Yes... It is there in the configuration file.

My Squid version is "Squid Cache: Version 2.5.STABLE13"

Regards,
--Sathyan Arjunan
Reply | Threaded
Open this post in threaded view
|

RE: Cache is running out of filedescriptors

Henrik Nordström
On fre, 2007-07-20 at 17:43 -0700, Sathyan, Arjonan wrote:

> Yes... I am sure this version supports Smartfilter because we have a
> identical server running on the same version of squid + smartfilter

What I meant was that you must verify with Smartfilter that they support
the use or more than 1024 filedescriptors.

> >> Step 2) Edit squid.conf, max_open_disk_fds=8192
>
> > There is no such squid.conf parameter..
>
> Yes... It is there in the configuration file.

Ah, yes indeed there is. You can ignore this parameter. Not relevant to
what you are doing. Best left unset at it's default 0 value.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Tek Bahadur Limbu
In reply to this post by Henrik Nordström
Henrik Nordstrom wrote:

> On lör, 2007-07-21 at 00:13 +0545, Tek Bahadur Limbu wrote:
>
>> You need to increase the size of _FD_SETSIZE in your
>> /usr/include/bits/typesizes.h file.
>
> No you don't. Hasn't been needed since Squid-2.5.something (don't
> remember exact version, many many years ago).
>
>> After increasing the above value to 8192, you type ulimit -HSn 8192 and
>> compile Squid.
>
> This, or --max-fd=NN configure option is needed however. And you also
> need to raise the ulimit when Squid is started.

Hi Henrik,

Thanks for correcting me. By the way, I don't see the "--max-fd=NN"
option with "./configure  --help".

I will use this option in my Squid boxes running on Linux in the future.

Thanking you...

>
> Regards
> Henrik


--

With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Henrik Nordström
On lör, 2007-07-21 at 17:35 +0545, Tek Bahadur Limbu wrote:

> Henrik Nordstrom wrote:
> > On lör, 2007-07-21 at 00:13 +0545, Tek Bahadur Limbu wrote:
> >
> >> You need to increase the size of _FD_SETSIZE in your
> >> /usr/include/bits/typesizes.h file.
> >
> > No you don't. Hasn't been needed since Squid-2.5.something (don't
> > remember exact version, many many years ago).
> >
> >> After increasing the above value to 8192, you type ulimit -HSn 8192 and
> >> compile Squid.
> >
> > This, or --max-fd=NN configure option is needed however. And you also
> > need to raise the ulimit when Squid is started.
>
> Hi Henrik,
>
> Thanks for correcting me. By the way, I don't see the "--max-fd=NN"
> option with "./configure  --help".
My error, it's --with-maxfd=NN. Don't type it often as it's buried in
the build script..

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Manoj Rajkarnikar
In reply to this post by Tek Bahadur Limbu
On Sat, 21 Jul 2007, Tek Bahadur Limbu wrote:

> Hi Henrik,
>
> Thanks for correcting me. By the way, I don't see the "--max-fd=NN" option
> with "./configure  --help".

there is an option --with-maxfd=N in 2.6.

>
> I will use this option in my Squid boxes running on Linux in the future.
>
> Thanking you...
>

Manoj.
--
Reply | Threaded
Open this post in threaded view
|

Re: Cache is running out of filedescriptors

Tek Bahadur Limbu
Manoj_Rajkarnikar wrote:
> On Sat, 21 Jul 2007, Tek Bahadur Limbu wrote:
>
>> Hi Henrik,
>>
>> Thanks for correcting me. By the way, I don't see the "--max-fd=NN"
>> option with "./configure  --help".
>
> there is an option --with-maxfd=N in 2.6.
>

Hi Manoj,

Thanks. I completely missed the "--with-maxfd=N" option with
"./configure --help". After carefully inspection, I finally noticed it!!!

However, using the previous methods, I am currently running Squid in my
Linux boxes with 8192 file descriptors. I will compile
Squid with the "--with-maxfd=8192" option without modifying
/usr/include/bits/typesizes.h file in the near future.

Thanking you...

>>
>> I will use this option in my Squid boxes running on Linux in the future.
>>
>> Thanking you...
>>
>
> Manoj.


--

With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np