Quantcast

DiskThreadsDiskFile::openDone squid 3.5.0.4

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

DiskThreadsDiskFile::openDone squid 3.5.0.4

HackXBack
Hello squid ,
after using 3.5.0.4 on fresh debian system
i see many errors in cache.log

2014/12/26 07:21:39 kid1|       /cache03/2/00/31/00003123
2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:39 kid1|       /cache04/1/00/5F/00005F16
2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:39 kid1|       /cache03/3/00/29/0000291F
2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:39 kid1|       /cache05/1/00/11/000011F6
2014/12/26 07:21:45 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:45 kid1|       /cache03/3/00/17/0000176C
2014/12/26 07:21:46 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:46 kid1|       /cache02/6/00/15/000015CE
2014/12/26 07:21:47 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:47 kid1|       /cache02/6/00/0B/00000B07
2014/12/26 07:21:47 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:21:47 kid1|       /cache02/2/00/02/000002B4
2014/12/26 07:22:09 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:09 kid1|       /cache03/4/00/03/00000365
2014/12/26 07:22:12 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:12 kid1|       /cache03/2/00/1F/00001F26
2014/12/26 07:22:12 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:12 kid1|       /cache03/6/00/1F/00001F25
2014/12/26 07:22:13 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:13 kid1|       /cache04/6/00/1F/00001F21
2014/12/26 07:22:15 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:15 kid1|       /cache05/2/00/1F/00001F30
2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:21 kid1|       /cache02/6/00/1D/00001D5A
2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:21 kid1|       /cache02/2/00/0C/00000CB5
2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:21 kid1|       /cache03/5/00/01/00000144
2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:31 kid1|       /cache02/2/00/25/00002504
2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2014/12/26 07:22:31 kid1|       /cache04/5/00/24/0000244D
2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

Kinkie
Nothing to worry about. The files were removed by some outside
software and were not found. Squid will manage the error and carry on.

On Fri, Dec 26, 2014 at 1:22 PM, HackXBack <[hidden email]> wrote:

> Hello squid ,
> after using 3.5.0.4 on fresh debian system
> i see many errors in cache.log
>
> 2014/12/26 07:21:39 kid1|       /cache03/2/00/31/00003123
> 2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:39 kid1|       /cache04/1/00/5F/00005F16
> 2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:39 kid1|       /cache03/3/00/29/0000291F
> 2014/12/26 07:21:39 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:39 kid1|       /cache05/1/00/11/000011F6
> 2014/12/26 07:21:45 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:45 kid1|       /cache03/3/00/17/0000176C
> 2014/12/26 07:21:46 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:46 kid1|       /cache02/6/00/15/000015CE
> 2014/12/26 07:21:47 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:47 kid1|       /cache02/6/00/0B/00000B07
> 2014/12/26 07:21:47 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:21:47 kid1|       /cache02/2/00/02/000002B4
> 2014/12/26 07:22:09 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:09 kid1|       /cache03/4/00/03/00000365
> 2014/12/26 07:22:12 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:12 kid1|       /cache03/2/00/1F/00001F26
> 2014/12/26 07:22:12 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:12 kid1|       /cache03/6/00/1F/00001F25
> 2014/12/26 07:22:13 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:13 kid1|       /cache04/6/00/1F/00001F21
> 2014/12/26 07:22:15 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:15 kid1|       /cache05/2/00/1F/00001F30
> 2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:21 kid1|       /cache02/6/00/1D/00001D5A
> 2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:21 kid1|       /cache02/2/00/0C/00000CB5
> 2014/12/26 07:22:21 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:21 kid1|       /cache03/5/00/01/00000144
> 2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:31 kid1|       /cache02/2/00/25/00002504
> 2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2014/12/26 07:22:31 kid1|       /cache04/5/00/24/0000244D
> 2014/12/26 07:22:31 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
>
>
>
>
> --
> View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/DiskThreadsDiskFile-openDone-squid-3-5-0-4-tp4668840.html
> Sent from the Squid - Users mailing list archive at Nabble.com.
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users



--
    Francesco
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

elicro
In reply to this post by HackXBack
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/26/2014 02:22 PM, HackXBack wrote:
> Hello squid , after using 3.5.0.4 on fresh debian system i see many
> errors in cache.log

Hey There,

(leaving aside these errors)
As a part of a cache proxy integration I am generally recommending to
do it in steps by first using a proxy with ram only cache while later
adding the disk cache to make sure the machine can hold the basic
environment loads.

Eliezer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJUnwV1AAoJENxnfXtQ8ZQU7+kIAJoKcCbkRdgndSTsZiHyYMY8
oTJWAafR094ThJp6LFurkDaAZqaZhpa8ncBFlKaB9MQOUt5ylDGnFfGm9CNcdHOx
dtNIJl/FHxEylw4YABp89A8DuoepRRMDa6ACrtW74JV1UB7t6kYJzZIkZYYymn39
cTS5HeJlTc1aG/UJXtXDwkZnQ2ZQLIGhshfgCuuI7PN6hgljY2nrsmgbWsL1Jc3j
EjejRi4KdJQujlGyQJ0fvQYAxqWYKHYPJFnFYApT2Glqb+KHdMhKwBbsBpuqhUQM
loPc05vgcxqvR/RwiyYWcQDL7X4wLMEDn+NFu3MPnJ4O9jQLcqD8l9X2TMjOco8=
=KGXb
-----END PGP SIGNATURE-----
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

asnani_satish
This post has NOT been accepted by the mailing list yet.
This post was updated on .
This happens when size specified in cache_mem >= cache_dir
Example:
cache_dir aufs /var/spool/squid 1000 32 512
   implies 1000 MB physical disk space allotted for cache in specified directory
cache_mem 900 MB
   cache size to be used by squid which must be less than the size specified in cache_dir directive.

Dont forget to restart squid

We cannot just ignore the error because if this error keeps on occurring the performance of squid degrades.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

tmblue@gmail.com
asnani_satish wrote
This happens when size specified in cache_mem >= cache_dir
Example:
cache_dir aufs /var/spool/squid 1000 32 512
   implies 1000 MB physical disk space allotted for cache in specified directory
cache_mem 900 MB
   cache size to be used by squid which must be less than the size specified in cache_dir directive.

Dont forget to restart squid

We cannot just ignore the error because if this error keeps on occurring the performance of squid degrades.
Are you sure???

i'm getting this error when turning back on a squid system that I had down (out of production for 2 days). My config does not indicate an issue that you advise to be the cause of the error.

cache_mem 1 GB
cache_dir aufs /cache 65000 16 256

I'm not seeing any issue but the error is rampant at this point

2017/02/02 10:58:59 kid1| /cache/07/49/003749FE
2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2017/02/02 10:59:00 kid1| /cache/05/3E/00053EA0
2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2017/02/02 10:59:00 kid1| /cache/0D/EB/005DEB19
2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or directory
2017/02/02 10:59:00 kid1| /cache/02/27/0002273A
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

joseph
In reply to this post by asnani_satish
lol
mostly of this ar wen vary changed lol
wen they work on vary as it should be  then most of those msg will go away
not 100% but most of them
im not going into detail but its the vary in header

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

Amos Jeffries
Administrator
In reply to this post by tmblue@gmail.com
On 3/02/2017 7:56 a.m., [hidden email] wrote:

> asnani_satish wrote
>> This happens when size specified in cache_mem >= cache_dir
>> Example:
>> cache_dir aufs /var/spool/squid 1000 32 512
>>    implies 1000 MB physical disk space allotted for cache in specified
>> directory
>> cache_mem 900 MB
>>    cache size to be used by squid which must be less than the size
>> specified in cache_dir directive.
>>
>> Dont forget to restart squid
>>
>> We cannot just ignore the error because if this error keeps on occurring
>> the performance of squid degrades.
>
> Are you sure???
>
> i'm getting this error when turning back on a squid system that I had down
> (out of production for 2 days). My config does not indicate an issue that
> you advise to be the cause of the error.
>
> cache_mem 1 GB
> cache_dir aufs /cache 65000 16 256
>
> I'm not seeing any issue but the error is rampant at this point
>
> 2017/02/02 10:58:59 kid1| /cache/07/49/003749FE
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1| /cache/05/3E/00053EA0
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1| /cache/0D/EB/005DEB19
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1| /cache/02/27/0002273A
>

Squid uses a journal file swap.state to store changes to the cache. On
restart it is used to reconstruct the cache index fast. If you cause
Squid to shutdown very fast (kill -9, systemd crashing it, or
shutdown_lifetime too short) then it does not have time to fully update
the journal entries.

When it gets restarted after one of those abrupt shutdowns one of these
messages (not errors!) gets logged for each entry which the journal
indicates are present - but are not really there because their removal
was not recorded on shutdown.

NP: There are also things in the cache being orphaned then just
overwritten because the journal does not record them as existing. But no
messages about that because its not detected.

These messages should decrease exponentially proportional to your
traffics normal HIT rate. But in a very large cache like yours it still
may take many hours to become noticably slower and days to disappear
entirely. That is of course assuming your Squid does not have another
restart event to begin the process all over again.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

tmblue@gmail.com


On Thu, Feb 2, 2017 at 3:51 PM, Amos Jeffries <[hidden email]> wrote:
On 3/02/2017 7:56 a.m., [hidden email] wrote:
> asnani_satish wrote
>> This happens when size specified in cache_mem >= cache_dir
>> Example:
>> cache_dir aufs /var/spool/squid 1000 32 512
>>    implies 1000 MB physical disk space allotted for cache in specified
>> directory
>> cache_mem 900 MB
>>    cache size to be used by squid which must be less than the size
>> specified in cache_dir directive.
>>
>> Dont forget to restart squid
>>
>> We cannot just ignore the error because if this error keeps on occurring
>> the performance of squid degrades.
>
> Are you sure???
>
> i'm getting this error when turning back on a squid system that I had down
> (out of production for 2 days). My config does not indicate an issue that
> you advise to be the cause of the error.
>
> cache_mem 1 GB
> cache_dir aufs /cache 65000 16 256
>
> I'm not seeing any issue but the error is rampant at this point
>
> 2017/02/02 10:58:59 kid1|     /cache/07/49/003749FE
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1|     /cache/05/3E/00053EA0
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1|     /cache/0D/EB/005DEB19
> 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such file or
> directory
> 2017/02/02 10:59:00 kid1|     /cache/02/27/0002273A
>

Squid uses a journal file swap.state to store changes to the cache. On
restart it is used to reconstruct the cache index fast. If you cause
Squid to shutdown very fast (kill -9, systemd crashing it, or
shutdown_lifetime too short) then it does not have time to fully update
the journal entries.

When it gets restarted after one of those abrupt shutdowns one of these
messages (not errors!) gets logged for each entry which the journal
indicates are present - but are not really there because their removal
was not recorded on shutdown.

NP: There are also things in the cache being orphaned then just
overwritten because the journal does not record them as existing. But no
messages about that because its not detected.

These messages should decrease exponentially proportional to your
traffics normal HIT rate. But in a very large cache like yours it still
may take many hours to become noticably slower and days to disappear
entirely. That is of course assuming your Squid does not have another
restart event to begin the process all over again.

Amos

As always thanks Amos

These are systemctl stop squid and systemctl start squid, so nothing dramatic or nasty, should be shutting down cleanly

Tory 

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: DiskThreadsDiskFile::openDone squid 3.5.0.4

Matus UHLAR - fantomas
On 02.02.17 15:54, Tory M Blue wrote:

>> > I'm not seeing any issue but the error is rampant at this point
>> >
>> > 2017/02/02 10:58:59 kid1|     /cache/07/49/003749FE
>> > 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such
>> file or
>> > directory
>> > 2017/02/02 10:59:00 kid1|     /cache/05/3E/00053EA0
>> > 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such
>> file or
>> > directory
>> > 2017/02/02 10:59:00 kid1|     /cache/0D/EB/005DEB19
>> > 2017/02/02 10:59:00 kid1| DiskThreadsDiskFile::openDone: (2) No such
>> file or
>> > directory
>> > 2017/02/02 10:59:00 kid1|     /cache/02/27/0002273A

>> Squid uses a journal file swap.state to store changes to the cache. On
>> restart it is used to reconstruct the cache index fast. If you cause
>> Squid to shutdown very fast (kill -9, systemd crashing it, or
>> shutdown_lifetime too short) then it does not have time to fully update
>> the journal entries.

>These are systemctl stop squid and systemctl start squid, so nothing
>dramatic or nasty, should be shutting down cleanly

"should be" ... still depends on shutdown_lifetime and the systemd can still
be instructed to kill squid, look at squid.service

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The 3 biggets disasters: Hiroshima 45, Tschernobyl 86, Windows 95
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...