Questions regarding "store-id" made by helper and log system

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions regarding "store-id" made by helper and log system

Thomas PALFRAY

Hi everyone !

 

I’m trying to use a helper to make custom « store-id » for some requests.

The idea is to increase HIT on theses request types.

 

Here is some example of my helper input/output :

 

http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?succinct=true&streamId=207698&nvltToken=27348686-4a4e-4d1e-b3b4-62a7243f5207&publicCache=1296000&cmisselector=content&objectId=146662

 

OK store-id=http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?cmisselector=content&objectId=146662&streamId=207698&succinct=true

 

http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?succinct=true&streamId=207698&publicCache=1296000&cmisselector=content&objectId=146662

 

OK store-id=http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?cmisselector=content&objectId=146662&streamId=207698&succinct=true

 

I update the squid.conf like this :

store_id_program /usr/local/bin/urlSort2

store_id_children 25 startup=15 idle=5 concurrency=0

 

And restart the service.

 

I see urlSort2 in ram using ps so i guess it’s called by squid and returning modified store-id :

[root@xxx ~]# ps -eaf | grep urlSort2

squid     15788  15749  0 08:58 ?        00:00:00 (urlSort2)

 

 

 

When i look into access.log and store.log, i see this :

 

Access.log :

1607599918.911   1245 10.101.20.53 TCP_MISS/200 128832 GET http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?succinct=true&streamId=207698&nvltToken=27348686-4a4e-4d1e-b3b4-62a7243f5207&publicCache=1296000&cmisselector=content&objectId=146662 user HIER_DIRECT/xxx.xxx.xxx.xxx image/jpeg

 

Store.log :

1607666419.148 SWAPOUT 00 00000543 791B7B07B8004584E9432DA2444EED74  200 1607666417        -1 1608962417 image/jpeg 97165/97165 GET http://myproject-prod.myhost.com:8080/current/cmis/browser/myproject_prod/root?succinct=true&streamId=194324&nvltToken=72f33f50-f267-4025-a373-7b1a00cfe500&publicCache=1296000&cmisselector=content&objectId=129520

 

When i call the first request of my example of output help, then the second, i get a MISS.

I suppose that there’s not using the same store-id.

 

So i have two questions :

  1. Is the log system of squid display (or can display) store-id, or only url request ? I didn’t found information about this on the documentation.
  2. Why i get only MISS, and how be sure that the helper is correctly use ?

 

BR,

 

Thomas PALFRAY

 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding "store-id" made by helper and log system

Alex Rousskov
On 12/11/20 1:42 AM, Thomas PALFRAY wrote:

> When i call the first request of my example of output help, then the
> second, i get a MISS.

For a more complete picture, I would test with four sequential requests,
two with one URL and two with another.


> I suppose that there’s not using the same store-id.

That is one of many possibilities. Your best bet may be to examine Squid
debugging log to understand whether the right Store ID is computed for
each request and, if yes, why the second request misses. See
debug_options in squid.conf.documented.

If setting debug_options to "ALL,7" overwhelms you with noise, try
something like "ALL,3 33,7 47,7 61,7 85,7". If you cannot figure it out,
then use "ALL,9" and share a pointer to compressed cache.log containing
just the two transactions (to start with).


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding "store-id" made by helper and log system

Thomas PALFRAY
Hi Alex,

Thanks for your quick answer.

> For a more complete picture, I would test with four sequential requests, two with one URL and two with another.

I get a HIT for the two request when i retry each one which let me think than the store-id computed is not the one from the helper but the plain url.

I also tried to force each request to go through my helper with " store_id_bypass off"
When I do that performance go dramatically down (i have around 400 users behind squid).

>That is one of many possibilities. Your best bet may be to examine Squid debugging log to understand whether the right Store ID is computed for each request and, if yes, why the second request misses. See debug_options in squid.conf.documented.
>If setting debug_options to "ALL,7" overwhelms you with noise, try something like "ALL,3 33,7 47,7 61,7 85,7". If you cannot figure it out, then use "ALL,9" and share a pointer to compressed cache.log containing just the two transactions (to start with).

Ok i will try these settings, i let you know.

BR,

Thomas PALFRAY

-----Message d'origine-----
De : Alex Rousskov <[hidden email]>
Envoyé : vendredi 11 décembre 2020 18:01
À : Thomas PALFRAY <[hidden email]>; [hidden email]
Objet : Re: [squid-users] Questions regarding "store-id" made by helper and log system

On 12/11/20 1:42 AM, Thomas PALFRAY wrote:

> When i call the first request of my example of output help, then the
> second, i get a MISS.

For a more complete picture, I would test with four sequential requests, two with one URL and two with another.


> I suppose that there's not using the same store-id.

That is one of many possibilities. Your best bet may be to examine Squid debugging log to understand whether the right Store ID is computed for each request and, if yes, why the second request misses. See debug_options in squid.conf.documented.

If setting debug_options to "ALL,7" overwhelms you with noise, try something like "ALL,3 33,7 47,7 61,7 85,7". If you cannot figure it out, then use "ALL,9" and share a pointer to compressed cache.log containing just the two transactions (to start with).


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users