Recommended Cache Settings for cache_mem

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Recommended Cache Settings for cache_mem

Adam Parsons
Hi,  I would like advise on what the best settings would be for cache_mem on a SquidNT (Note: SquidNT) box that we will be putting out into a number of schools.  The specs of the workstation are Pentium Duo Core 1.86, 1 GB Ram, 160GB Hard drive.  We have set aside 60GB of space for the cache (though we could go to 80GB?, as we have the space).  The box is mostly going to be used for a local squid caching box, but it is also going to be used as a McAfee repository (which will only take up 150MB of space).  What would be the best setting for cache_mem and would setting the cache_dir to 80GB be beneficial, seeing as most of these sites would use less than 10GB of internet a month?

Thanks in advance - Adam

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Snow Wolf-2
From my experience,when squid has used full of cache_dir close to
20-30G,the OS would become very high-load due to the disk cache
swap.So I think even you have 160G disk,you may not set the cache_dir
bigger than 20G.


2007/7/4, Adam Parsons <[hidden email]>:
> Hi,  I would like advise on what the best settings would be for cache_mem on a SquidNT (Note: SquidNT) box that we will be putting out into a number of schools.  The specs of the workstation are Pentium Duo Core 1.86, 1 GB Ram, 160GB Hard drive.  We have set aside 60GB of space for the cache (though we could go to 80GB?, as we have the space).  The box is mostly going to be used for a local squid caching box, but it is also going to be used as a McAfee repository (which will only take up 150MB of space).  What would be the best setting for cache_mem and would setting the cache_dir to 80GB be beneficial, seeing as most of these sites would use less than 10GB of internet a month?
>
> Thanks in advance - Adam
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Michel (M)

Snow Wolf disse na ultima mensagem:
>>From my experience,when squid has used full of cache_dir close to
> 20-30G,the OS would become very high-load due to the disk cache
> swap.So I think even you have 160G disk,you may not set the cache_dir
> bigger than 20G.
>

ahem ... so I would need 8 x 160G disks in order getting 160G of cache_dir
or 4 x 250 also would do it? And the rest of each disk is used by this
"disk cache swap" thing?

Michel


>
> 2007/7/4, Adam Parsons <[hidden email]>:
>> Hi,  I would like advise on what the best settings would be for
>> cache_mem on a SquidNT (Note: SquidNT) box that we will be putting out
>> into a number of schools.  The specs of the workstation are Pentium Duo
>> Core 1.86, 1 GB Ram, 160GB Hard drive.  We have set aside 60GB of space
>> for the cache (though we could go to 80GB?, as we have the space).  The
>> box is mostly going to be used for a local squid caching box, but it is
>> also going to be used as a McAfee repository (which will only take up
>> 150MB of space).  What would be the best setting for cache_mem and would
>> setting the cache_dir to 80GB be beneficial, seeing as most of these
>> sites would use less than 10GB of internet a month?
>>
>> Thanks in advance - Adam
>>
>>
>
>
>
>
>
>
>
> A mensagem foi scaneada pelo sistema de e-mail e pode ser considerada
> segura.
> Service fornecido pelo Datacenter Matik  https://datacenter.matik.com.br
>




...




****************************************************
Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.
****************************************************

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Jeff Pang-6
2007/7/4, Michel Santos <[hidden email]>:

>
> Snow Wolf disse na ultima mensagem:
> >>From my experience,when squid has used full of cache_dir close to
> > 20-30G,the OS would become very high-load due to the disk cache
> > swap.So I think even you have 160G disk,you may not set the cache_dir
> > bigger than 20G.
> >
>
> ahem ... so I would need 8 x 160G disks in order getting 160G of cache_dir
> or 4 x 250 also would do it? And the rest of each disk is used by this
> "disk cache swap" thing?
>

Hmm,you can try it.I mean no more than 20G totally for use.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Amos Jeffries
Administrator
Jeff Pang wrote:

> 2007/7/4, Michel Santos <[hidden email]>:
>>
>> Snow Wolf disse na ultima mensagem:
>> >>From my experience,when squid has used full of cache_dir close to
>> > 20-30G,the OS would become very high-load due to the disk cache
>> > swap.So I think even you have 160G disk,you may not set the cache_dir
>> > bigger than 20G.
>> >
>>
>> ahem ... so I would need 8 x 160G disks in order getting 160G of
>> cache_dir
>> or 4 x 250 also would do it? And the rest of each disk is used by this
>> "disk cache swap" thing?
>>

I would think, one drive with several cache_dir inside would suffice ;-)
Though I think there is something screwey with a system that can't
handle disk-cache for any disk it accepts.

>
> Hmm,you can try it.I mean no more than 20G totally for use.

For the record which cache_dir filesystem (diskd, ufs, aufs, coss, null)
do you use that only gives you 20Gb ??

I have had aufs here and at one point a 30Gb cache running happily
without stressing anything (even on RAID drives at the time).

Amos
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Tek Bahadur Limbu
In reply to this post by Adam Parsons
Adam Parsons wrote:
> Hi,  I would like advise on what the best settings would be for cache_mem on a SquidNT (Note: SquidNT) box that we will be putting out into a number of schools.  The specs of the workstation are Pentium Duo Core 1.86, 1 GB Ram, 160GB Hard drive.  We have set aside 60GB of space for the cache (though we could go to 80GB?, as we have the space).  The box is mostly going to be used for a local squid caching box, but it is also going to be used as a McAfee repository (which will only take up 150MB of space).  What would be the best setting for cache_mem and would setting the cache_dir to 80GB be beneficial, seeing as most of these sites would use less than 10GB of internet a month?
>

Hi Adam,

For cache_mem, you could use:

cache_mem 32 MB

Regarding cache_dir, you can try and use 60GB if your internet usage is
high.

Or you could use something like:

cache_dir ufs /cache1 20480 24 256


I suppose you will only be using 1 partition (/cache1) in each of your
SquidNT boxes.

Also you can use several other storage schemes instead of ufs. They
include aufs, diskd and coss. However if your SquidNT box will only be
serving between, say, 1 to 40 requests per second, then ufs should serve
you fine.

If you want to use or experiment with other storage schemes, then you
need to instruct Squid to use them during your compilation with
something like the following parameter:

'--enable-storeio=aufs,diskd,coss,ufs,null'

Hope it helps.

Thanking you...

> Thanks in advance - Adam
>
>
>
>


--

With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Joel Jaeggli-3
In reply to this post by Jeff Pang-6
Jeff Pang wrote:
> 2007/7/4, Michel Santos <[hidden email]>:
>>
>> Snow Wolf disse na ultima mensagem:
>> >>From my experience,when squid has used full of cache_dir close to
>> > 20-30G,the OS would become very high-load due to the disk cache
>> > swap.So I think even you have 160G disk,you may not set the cache_dir
>> > bigger than 20G.
>> >

The real measure here is number of i/o's per second per spindle...
you'll something on the order of 50-100 from 7200 rpm disks (read
service time is around 20ms) given that not all activities (writes)
require an immediate seek. In the context of squid, fast disks or lots
of disks are more important than large disks because scaling comes from
the number of requests that can be served....

>> ahem ... so I would need 8 x 160G disks in order getting 160G of
>> cache_dir
>> or 4 x 250 also would do it? And the rest of each disk is used by this
>> "disk cache swap" thing?
>>

the rest of the disk is empty.

> Hmm,you can try it.I mean no more than 20G totally for use.
>

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Henrik Nordström
In reply to this post by Snow Wolf-2
On ons, 2007-07-04 at 09:31 +0800, Snow Wolf wrote:
> From my experience,when squid has used full of cache_dir close to
> 20-30G,the OS would become very high-load due to the disk cache
> swap.So I think even you have 160G disk,you may not set the cache_dir
> bigger than 20G.

Then you probably ran out of memory.. See FAQ section on memory use.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Michel (M)
In reply to this post by Joel Jaeggli-3

Joel Jaeggli disse na ultima mensagem:

> Jeff Pang wrote:
>> 2007/7/4, Michel Santos <[hidden email]>:
>>>
>>> Snow Wolf disse na ultima mensagem:
>>> >>From my experience,when squid has used full of cache_dir close to
>>> > 20-30G,the OS would become very high-load due to the disk cache
>>> > swap.So I think even you have 160G disk,you may not set the cache_dir
>>> > bigger than 20G.
>>> >
>
> The real measure here is number of i/o's per second per spindle...
> you'll something on the order of 50-100 from 7200 rpm disks (read
> service time is around 20ms) given that not all activities (writes)
> require an immediate seek. In the context of squid, fast disks or lots
> of disks are more important than large disks because scaling comes from
> the number of requests that can be served....
>

I better hold my peace here ...


>>> ahem ... so I would need 8 x 160G disks in order getting 160G of
>>> cache_dir
>>> or 4 x 250 also would do it? And the rest of each disk is used by this
>>> "disk cache swap" thing?
>>>
>
> the rest of the disk is empty.
>


as long as they are new this empty disks are really cool but I, may be I
am kind of wierd, buy them to fill them up  ;)



Michel
...




****************************************************
Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.
****************************************************

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Michel (M)
In reply to this post by Jeff Pang-6

Jeff Pang disse na ultima mensagem:

> 2007/7/4, Michel Santos <[hidden email]>:
>>
>> Snow Wolf disse na ultima mensagem:
>> >>From my experience,when squid has used full of cache_dir close to
>> > 20-30G,the OS would become very high-load due to the disk cache
>> > swap.So I think even you have 160G disk,you may not set the cache_dir
>> > bigger than 20G.
>> >
>>
>> ahem ... so I would need 8 x 160G disks in order getting 160G of
>> cache_dir
>> or 4 x 250 also would do it? And the rest of each disk is used by this
>> "disk cache swap" thing?
>>
>
> Hmm,you can try it.I mean no more than 20G totally for use.
>

I thought cache_dir is for caching so I for my case I have some servers
having more than 1TB overall cache space and I am pretty happy with it.
20G or so might be enough for a not dedicated server or a small business
or home proxy. Anyway using for above mentioned reasons 20G of a 160g disk
seems kind of awkward to me

Michel


...




****************************************************
Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.
****************************************************

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Alexandre Correa
i have 1 dedicated server for squid serving about 700 users simultaneous and
80 req/s !!

server is dual opteron dual core and 4gb of ram..

this cache_mem is fine ?

cache_mem 256 MB

or is best decrease ou increase this ?



Thanks,


AlexandrE

--

Sds.
Alexandre J. Correa
Onda Internet / OPinguim.net
http://www.ondainternet.com.br
http://www.opinguim.net
Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Adrian Chadd
On Thu, Jul 05, 2007, Alexandre Correa wrote:
> i have 1 dedicated server for squid serving about 700 users simultaneous and
> 80 req/s !!

Smallfry. :)

> server is dual opteron dual core and 4gb of ram..
>
> this cache_mem is fine ?
>
> cache_mem 256 MB
>
> or is best decrease ou increase this ?

Thats fine. Just make sure you've either compiled Squid 64-bit or, if its
compiled 32-bit, you don't exceed whatever the system-wide process max size
is (2 or 3 gig, its system and kernel dependant.)

The rest of your RAM will be used for kernel disk/socket buffers and such;
that'll be good enough under your proposed load. You only need to up
cache_mem if your workload thrashes the memory cache.



Adrian

Reply | Threaded
Open this post in threaded view
|

Re: Recommended Cache Settings for cache_mem

Michel (M)
In reply to this post by Alexandre Correa

Alexandre Correa disse na ultima mensagem:

> i have 1 dedicated server for squid serving about 700 users simultaneous
> and
> 80 req/s !!
>
> server is dual opteron dual core and 4gb of ram..
>
> this cache_mem is fine ?
>
> cache_mem 256 MB
>
> or is best decrease ou increase this ?

how much you need depends in parts on maximum_object_size_in_memory also

you might like to monitor squid's memory usage and increase cache_mem to a
value which just not cause swap usage but I would say you could set it
initially to 350-450MB or something


Michel


...




****************************************************
Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.
****************************************************