When the redirect [301, 302, 307] is cached by Squid?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

When the redirect [301, 302, 307] is cached by Squid?

kAja Ziegler
Hi all,

  I want to ask why my Squid does not cache redirects 301, 302 and 307. See anomised example below. Even if I call the URL more times or I open it in the browser, I always get MISS independently of the return code 301, 302 or 307.

$ curl -v test.example.com/img307.jpg

> GET /img307.jpg HTTP/1.1
> Host:
test.example.com
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 307 Temporary Redirect
< Date: Tue, 19 Sep 2017 12:27:50 GMT
< Server: Apache
< Location: http://
test.example.com/img.svg
< Content-Length: 249
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128
< Connection: keep-alive
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>307 Temporary Redirect</title>
</head><body>
<h1>Temporary Redirect</h1>
<p>The document has moved <a href="http://
test.example.com/img.svg">here</a>.</p>
</body></html>


My anomised squid.conf is attached.

Thanks in advance for clarification

  zigi



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

squid.conf (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

Eliezer Croitoru
As you can see in the response headers there are no rules for caching:
< HTTP/1.1 307 Temporary Redirect
< Date: Tue, 19 Sep 2017 12:27:50 GMT
< Server: Apache
< Location: http://http://test.example.com/img.svg
< Content-Length: 249
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128
< Connection: keep-alive

If you have a specific service try to use redbot to analyze the response:
https://redbot.org/

It might give you what you need.

All The Bests,
Eliezer

* I do not know if it should be this way or not since I am missing couple things from the setup such as squid.conf..

----
http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


From: squid-users [mailto:[hidden email]] On Behalf Of kAja Ziegler
Sent: Tuesday, September 19, 2017 17:00
To: [hidden email]
Subject: [squid-users] When the redirect [301, 302, 307] is cached by Squid?

Hi all,

  I want to ask why my Squid does not cache redirects 301, 302 and 307. See anomised example below. Even if I call the URL more times or I open it in the browser, I always get MISS independently of the return code 301, 302 or 307.

$ curl -v http://test.example.com/img307.jpg

> GET /img307.jpg HTTP/1.1
> Host: http://test.example.com
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 307 Temporary Redirect
< Date: Tue, 19 Sep 2017 12:27:50 GMT
< Server: Apache
< Location: http://http://test.example.com/img.svg
< Content-Length: 249
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128
< Connection: keep-alive
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>307 Temporary Redirect</title>
</head><body>
<h1>Temporary Redirect</h1>
<p>The document has moved <a href="http://http://test.example.com/img.svg">here</a>.</p>
</body></html>

My anomised squid.conf is attached.
Thanks in advance for clarification

  zigi




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

Amos Jeffries
Administrator
In reply to this post by kAja Ziegler
On 20/09/17 02:00, kAja Ziegler wrote:
> Hi all,
>
>    I want to ask why my Squid does not cache redirects 301, 302 and 307.
> See anomised example below. Even if I call the URL more times or I open
> it in the browser, I always get MISS independently of the return code
> 301, 302 or 307.

302 and 307 are not because as their status description indicates they
are *temporary* results. They can only be cached if there are explicit
details from the server indicating for how long.

301 should be cached unless the object would need revalidation immediately.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

kAja Ziegler
In reply to this post by Eliezer Croitoru
Thank you to pointing me on such nice tool.

zigi

On Tue, Sep 19, 2017 at 4:54 PM, Eliezer Croitoru <[hidden email]> wrote:
As you can see in the response headers there are no rules for caching:
< HTTP/1.1 307 Temporary Redirect
< Date: Tue, 19 Sep 2017 12:27:50 GMT
< Server: Apache
< Location: http://http://test.example.com/img.svg
< Content-Length: 249
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128
< Connection: keep-alive

If you have a specific service try to use redbot to analyze the response:
https://redbot.org/

It might give you what you need.

All The Bests,
Eliezer

* I do not know if it should be this way or not since I am missing couple things from the setup such as squid.conf..

----
http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: <a href="tel:%2B972-5-28704261" value="+972528704261">+972-5-28704261
Email: [hidden email]


From: squid-users [mailto:[hidden email]] On Behalf Of kAja Ziegler
Sent: Tuesday, September 19, 2017 17:00
To: [hidden email]
Subject: [squid-users] When the redirect [301, 302, 307] is cached by Squid?

Hi all,

  I want to ask why my Squid does not cache redirects 301, 302 and 307. See anomised example below. Even if I call the URL more times or I open it in the browser, I always get MISS independently of the return code 301, 302 or 307.

$ curl -v http://test.example.com/img307.jpg

> GET /img307.jpg HTTP/1.1
> Host: http://test.example.com
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 307 Temporary Redirect
< Date: Tue, 19 Sep 2017 12:27:50 GMT
< Server: Apache
< Location: http://http://test.example.com/img.svg
< Content-Length: 249
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128
< Connection: keep-alive
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>307 Temporary Redirect</title>
</head><body>
<h1>Temporary Redirect</h1>
<p>The document has moved <a href="http://http://test.example.com/img.svg">here</a>.</p>
</body></html>

My anomised squid.conf is attached.
Thanks in advance for clarification

  zigi






_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

kAja Ziegler
In reply to this post by Amos Jeffries
Hi Amos,

302 and 307 are not because as their status description indicates they are *temporary* results. They can only be cached if there are explicit details from the server indicating for how long.

You were right. After I added header Cache-Control "max-age=60, must-revalidate" to the redirects, then the response was cached. Thank you for clarification.

301 should be cached unless the object would need revalidation immediately.

But for 301 I always get MISS - with and without a cache-control header:

$ curl -v http:/test.example.com/img301.jpg

> GET /img301.jpg HTTP/1.1
> Host:
test.example.com
> User-Agent: curl/7.50.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Thu, 21 Sep 2017 06:44:41 GMT
< Server: Apache
< Cache-Control: max-age=60, must-revalidate
< Location: http://
test.example.com/img.svg
< Content-Length: 247
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from
<squid-proxy>
< X-Cache-Lookup: MISS from
<squid-proxy>:3128

I can't find any information on Squid wiki or via Google if the object need revalidation immediately.


I use Squid 3.4.14 from CentOS 6 package squid34-3.4.14-15.el6.x86_64.

With best regards,

zigi

On Tue, Sep 19, 2017 at 5:17 PM, Amos Jeffries <[hidden email]> wrote:
On 20/09/17 02:00, kAja Ziegler wrote:
Hi all,

   I want to ask why my Squid does not cache redirects 301, 302 and 307. See anomised example below. Even if I call the URL more times or I open it in the browser, I always get MISS independently of the return code 301, 302 or 307.

302 and 307 are not because as their status description indicates they are *temporary* results. They can only be cached if there are explicit details from the server indicating for how long.

301 should be cached unless the object would need revalidation immediately.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

Amos Jeffries
Administrator
On 21/09/17 20:36, kAja Ziegler wrote:

> Hi Amos,
>
>     302 and 307 are not because as their status description indicates
>     they are *temporary* results. They can only be cached if there are
>     explicit details from the server indicating for how long.
>
>
> You were right. After I added header Cache-Control "max-age=60,
> must-revalidate" to the redirects, then the response was cached. Thank
> you for clarification.
>
>     301 should be cached unless the object would need revalidation
>     immediately.
>
>
> But for 301 I always get MISS - with and without a cache-control header:
>
> $ curl -v http:/test.example.com/img301.jpg
> <http://test.example.com/img301.jpg>
>
>> GET /img301.jpg HTTP/1.1
>> Host: test.example.com <http://test.example.com>
>> User-Agent: curl/7.50.1
>> Accept: */*
>>
> < HTTP/1.1 301 Moved Permanently
> < Date: Thu, 21 Sep 2017 06:44:41 GMT
> < Server: Apache
> < Cache-Control: max-age=60, must-revalidate
> < Location: http://test.example.com <http://test.example.com>/img.svg
> < Content-Length: 247
> < Content-Type: text/html; charset=iso-8859-1
> < X-Cache: MISS from <squid-proxy>
> < X-Cache-Lookup: MISS from <squid-proxy>:3128
>
> I can't find any information on Squid wiki or via Google if the object
> need revalidation immediately.

Thats what the "must-revalidate" means. It should work better with just
max-age or Expires header - and with a longer value than 60 sec since
this is supposed to be a *permanent* situation.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

kAja Ziegler
Thats what the "must-revalidate" means. It should work better with just max-age or Expires header - and with a longer value than 60 sec since this is supposed to be a *permanent* situation.

As I know "must-revalidate" mean "refuse to return stale responses to the user even if they say that stale responses are acceptable" - cached object must be revalidated.

Max-age=60 was only used for testing.

The results of my testing:

- redirect 302 or 307 - to be cached needs Cache-Control max-age > 0 or Expires "access plus 1 seconds"
- redirect 301 - to be cached needs Cache-Control max-age > 60 or Expires "access plus 61 seconds"

This is strange because I thought that 301 is always cached without Cache-Control or Expires headers. And I can't find any information in the documentation which describes such behaviour.

zigi

On Thu, Sep 21, 2017 at 11:11 AM, Amos Jeffries <[hidden email]> wrote:
On 21/09/17 20:36, kAja Ziegler wrote:
Hi Amos,

    302 and 307 are not because as their status description indicates
    they are *temporary* results. They can only be cached if there are
    explicit details from the server indicating for how long.


You were right. After I added header Cache-Control "max-age=60, must-revalidate" to the redirects, then the response was cached. Thank you for clarification.

    301 should be cached unless the object would need revalidation
    immediately.


But for 301 I always get MISS - with and without a cache-control header:

$ curl -v http:/test.example.com/img301.jpg <http://test.example.com/img301.jpg>

GET /img301.jpg HTTP/1.1
Host: test.example.com <http://test.example.com>
User-Agent: curl/7.50.1
Accept: */*

< HTTP/1.1 301 Moved Permanently
< Date: Thu, 21 Sep 2017 06:44:41 GMT
< Server: Apache
< Cache-Control: max-age=60, must-revalidate
< Location: http://test.example.com <http://test.example.com>/img.svg
< Content-Length: 247
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from <squid-proxy>
< X-Cache-Lookup: MISS from <squid-proxy>:3128

I can't find any information on Squid wiki or via Google if the object need revalidation immediately.

Thats what the "must-revalidate" means. It should work better with just max-age or Expires header - and with a longer value than 60 sec since this is supposed to be a *permanent* situation.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

Amos Jeffries
Administrator
On 21/09/17 22:16, kAja Ziegler wrote:

>     Thats what the "must-revalidate" means. It should work better with
>     just max-age or Expires header - and with a longer value than 60 sec
>     since this is supposed to be a *permanent* situation.
>
>
> As I know "must-revalidate" mean "refuse to return stale responses to
> the user even if they say that stale responses are acceptable" - cached
> object must be revalidated.
>
> Max-age=60 was only used for testing.
>
> The results of my testing:
>
> - redirect 302 or 307 - to be cached needs Cache-Control max-age > 0 or
> Expires "access plus 1 seconds"
> - redirect 301 - to be cached needs Cache-Control max-age > 60 or
> Expires "access plus 61 seconds"
>

There is no >60 need on the 301. Just me suggesting that 60sec is too
short caching time for a _permanent_ thing.

> This is strange because I thought that 301 is always cached without
> Cache-Control or Expires headers. And I can't find any information in
> the documentation which describes such behaviour.

It should be, so long as it is fresh so that means it does depend on
refresh_pattern saying it is fresh when no controls are present.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: When the redirect [301, 302, 307] is cached by Squid?

kAja Ziegler
There is no >60 need on the 301. Just me suggesting that 60sec is too short caching time for a _permanent_ thing.

There is - may be some misconfiguration in my squid.conf.

If I set max-age to values from interval <1;60> only 302 and 307 redirects were cached (HIT) and no 301. When I increased the max-age value to 61 then the 301 redirect was cached too.


But you are right, that the cause of no-caching the 301 redirect is the min value of:
refresh_pattern .		0	20%	4320

But for redirects 302 and 307 the above-mentioned refresh_pattern is ignored.


My source server is Apache and the corresponding configuration is:

RewriteRule /img301.jpg /img.svg [R=301,L,E=rcache:1]
RewriteRule /img302.jpg /img.svg [R=302,L,E=rcache:1]
RewriteRule /img307.jpg /img.svg [R=307,L,E=rcache:1]

Header always set Cache-Control "max-age=3" env=rcache


zigi

On Thu, Sep 21, 2017 at 2:24 PM, Amos Jeffries <[hidden email]> wrote:
On 21/09/17 22:16, kAja Ziegler wrote:
    Thats what the "must-revalidate" means. It should work better with
    just max-age or Expires header - and with a longer value than 60 sec
    since this is supposed to be a *permanent* situation.


As I know "must-revalidate" mean "refuse to return stale responses to the user even if they say that stale responses are acceptable" - cached object must be revalidated.

Max-age=60 was only used for testing.

The results of my testing:

- redirect 302 or 307 - to be cached needs Cache-Control max-age > 0 or Expires "access plus 1 seconds"
- redirect 301 - to be cached needs Cache-Control max-age > 60 or Expires "access plus 61 seconds"


There is no >60 need on the 301. Just me suggesting that 60sec is too short caching time for a _permanent_ thing.

This is strange because I thought that 301 is always cached without Cache-Control or Expires headers. And I can't find any information in the documentation which describes such behaviour.

It should be, so long as it is fresh so that means it does depend on refresh_pattern saying it is fresh when no controls are present.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users