icap result caching in squid

Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

icap result caching in squid

Darren Breeze-2

Hi all.

Some quick question about icap result caching in squid.

Does the returned Expires header control how long squid will cache the result (for both a req and resp mod)?

Are the values that are cached keyed to the queried URL or is it cached per user / url?

thanks

Darren B.




This email and any files transmitted with
it are confidential and intended solely for the use of the individual
or entity to whom they are addressed. If you have received this email
in error please notify the system manager. This message contains
confidential information and is intended only for the individual named.
If you are not the named addressee you should not disseminate,
distribute or copy this e-mail. Please notify the sender immediately by
e-mail if you have received this e-mail by mistake and delete this
e-mail from your system. If you are not the intended recipient you are
notified that disclosing, copying, distributing or taking any action in
reliance on the contents of this information is strictly prohibited.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Alex Rousskov
On 11/29/19 12:20 PM, Darren Breeze wrote:

> Some quick question about icap result caching in squid.
>
> Does the returned Expires header control how long squid will cache the
> result (for both a req and resp mod)?
>
> Are the values that are cached keyed to the queried URL or is it cached
> per user / url?

Squid only supports pre-cache vectoring points. Thus, bugs
notwithstanding, post-ICAP headers should be treated (for caching
purposes) as if Squid received the same adjusted HTTP message directly
from an HTTP agent, and there were no ICAP modifications at all.

The above statement does not answer your question, but it changes that
question from "How ICAP-set X affects caching?" to "How X affects
caching?" -- a question that you may already know the answer to or, if
you do not, a question that others on the list may be able to answer
better or faster than I currently can.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Darren Breeze-2
Thanks Alex.

That has given me the perspective I need.

I can modulate the response header Expires value via icap to get the job done when squid goes to cache it.


Darren B.

*Darren Breeze*
Director
*KSN Systems NZ Limited*
**E: *[hidden email]*
**M:* +64 274 666 017*
**S:* dbinhk*

On Sun, Dec 1, 2019, at 6:22 AM, Alex Rousskov wrote:

> On 11/29/19 12:20 PM, Darren Breeze wrote:
>
> > Some quick question about icap result caching in squid.
> >
> > Does the returned Expires header control how long squid will cache the
> > result (for both a req and resp mod)?
> >
> > Are the values that are cached keyed to the queried URL or is it cached
> > per user / url?
>
> Squid only supports pre-cache vectoring points. Thus, bugs
> notwithstanding, post-ICAP headers should be treated (for caching
> purposes) as if Squid received the same adjusted HTTP message directly
> from an HTTP agent, and there were no ICAP modifications at all.
>
> The above statement does not answer your question, but it changes that
> question from "How ICAP-set X affects caching?" to "How X affects
> caching?" -- a question that you may already know the answer to or, if
> you do not, a question that others on the list may be able to answer
> better or faster than I currently can.
>
>
> HTH,
>
> Alex.
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Amos Jeffries
Administrator
On 3/12/19 12:17 pm, Darren Breeze wrote:
> Thanks Alex.
>
> That has given me the perspective I need.
>
> I can modulate the response header Expires value via icap to get the job done when squid goes to cache it.
>

Why are you needing to do this at all?

NP: please be aware that the changes you make to the HTTP headers affect
*all* downstream caches, no just your Squid.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Darren Breeze-2
Hi Amos

The Icap service is doing redirects based on client permissions (that may change). What I am doing is just setting the Expires value in the RESP_MOD response when I return a 307 redirect so I can control how long the caches (both Squid and the browser) hang on to it.



Darren B.





On Tue, Dec 3, 2019, at 6:17 PM, Amos Jeffries wrote:

> On 3/12/19 12:17 pm, Darren Breeze wrote:
> > Thanks Alex.
> >
> > That has given me the perspective I need.
> >
> > I can modulate the response header Expires value via icap to get the job done when squid goes to cache it.
> >
>
> Why are you needing to do this at all?
>
> NP: please be aware that the changes you make to the HTTP headers affect
> *all* downstream caches, no just your Squid.
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Amos Jeffries
Administrator
On 4/12/19 6:05 am, Darren Breeze wrote:
> Hi Amos
>
> The Icap service is doing redirects based on client permissions (that may change). What I am doing is just setting the Expires value in the RESP_MOD response when I return a 307 redirect so I can control how long the caches (both Squid and the browser) hang on to it.
>

In that case you should be using Cache-Control:max-age=NN instead where
the NN being your desired TTL in seconds. This is a *lot* simpler and
faster to deliver than calculating the timestamps needed for valid
Expires header.

Also, are you aware that current Squid versions can generate redirects
(custom headers included) based on output from an external_acl_type helper?
 A helper to lookup your permissions system plus a few extra squid.conf
settings would be a lot simpler in terms of traffic processing and
bandwidth consumption than sending everything through an external ICAP
service.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Darren Breeze-2
Thanks Amos

The Icap also does url filtering so it's a one stop shop for everything. I have to stay with V3.5 for the moment but will move up to 4.x later and re-examine my approach based on the newer features available.

Cache-Control is much cleaner and it actually is named as per what I want to do so it makes my intent clearer to others reading the code in the future.

Thanks again.

Darren B.

On Wed, Dec 4, 2019, at 6:26 PM, Amos Jeffries wrote:

> On 4/12/19 6:05 am, Darren Breeze wrote:
> > Hi Amos
> >
> > The Icap service is doing redirects based on client permissions (that may change). What I am doing is just setting the Expires value in the RESP_MOD response when I return a 307 redirect so I can control how long the caches (both Squid and the browser) hang on to it.
> >
>
> In that case you should be using Cache-Control:max-age=NN instead where
> the NN being your desired TTL in seconds. This is a *lot* simpler and
> faster to deliver than calculating the timestamps needed for valid
> Expires header.
>
> Also, are you aware that current Squid versions can generate redirects
> (custom headers included) based on output from an external_acl_type helper?
>  A helper to lookup your permissions system plus a few extra squid.conf
> settings would be a lot simpler in terms of traffic processing and
> bandwidth consumption than sending everything through an external ICAP
> service.
>
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: icap result caching in squid

Darren Breeze-2
In reply to this post by Amos Jeffries
Hi Amos

I have done some digging into your suggestion, and it looks like a better way to go for what I need to do.

> Also, are you aware that current Squid versions can generate redirects
> (custom headers included) based on output from an external_acl_type helper?
>  A helper to lookup your permissions system plus a few extra squid.conf
> settings would be a lot simpler in terms of traffic processing and
> bandwidth consumption than sending everything through an external ICAP
> service.

Are you able to point me in the direction of some sample config fragments that show how this can be done please.

I am guessing that I would need to return some custom keyword / value pairs from the external_acl and set other acls based on the values.

What I am aiming at is to selectively bump or redirect (or both) based on the client status and the site being requested.

Thanks again.

Darren B.



On Wed, Dec 4, 2019, at 6:26 PM, Amos Jeffries wrote:

> On 4/12/19 6:05 am, Darren Breeze wrote:
> > Hi Amos
> >
> > The Icap service is doing redirects based on client permissions (that may change). What I am doing is just setting the Expires value in the RESP_MOD response when I return a 307 redirect so I can control how long the caches (both Squid and the browser) hang on to it.
> >
>
> In that case you should be using Cache-Control:max-age=NN instead where
> the NN being your desired TTL in seconds. This is a *lot* simpler and
> faster to deliver than calculating the timestamps needed for valid
> Expires header.
>
> Also, are you aware that current Squid versions can generate redirects
> (custom headers included) based on output from an external_acl_type helper?
>  A helper to lookup your permissions system plus a few extra squid.conf
> settings would be a lot simpler in terms of traffic processing and
> bandwidth consumption than sending everything through an external ICAP
> service.
>
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users