flickr.com redirect error

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

flickr.com redirect error

Ozgur Batur
I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen. 

I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.

I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.


Ozgur

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Yuri Voinov

Try to do something like:


# 301 loop
acl text_mime rep_mime_type text/html text/plain

acl http301 http_status 301

store_miss deny text_mime http301
send_hit deny text_mime http301


24.06.2016 18:14, Ozgur Batur пишет:
I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen. 

I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.

I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.


Ozgur


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Ozgur Batur
Hi Yuri,

Thank you. I put the #301 loop directives and restarted squid unfortunately result is the same. Here is the access logs:

1466777191.791    235 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html
1466777192.031    237 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html
1466777192.386    352 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html
1466777192.612    223 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html
...

As I understand all responses are from origin server, there is no cache hit with or without store_miss and send_hit. Confusing part is when directly connected to server without proxy, flickr server does not send 301 response. When squid sends the same request somehow flickr server returns 301 with same URL. 

Ozgur


On Fri, Jun 24, 2016 at 3:50 PM, Yuri <[hidden email]> wrote:

Try to do something like:


# 301 loop
acl text_mime rep_mime_type text/html text/plain

acl http301 http_status 301

store_miss deny text_mime http301
send_hit deny text_mime http301


24.06.2016 18:14, Ozgur Batur пишет:
I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen. 

I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.

I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.


Ozgur


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
Hm. My opinion is the same - this is redirection loop. Just need to localize it.


24.06.2016 20:23, Ozgur Batur пишет:
> Hi Yuri,
>
> Thank you. I put the #301 loop directives and restarted squid unfortunately result is the same. Here is the access logs:
>
> 1466777191.791    235 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
> 1466777192.031    237 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
> 1466777192.386    352 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
> 1466777192.612    223 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
> ...
>
> As I understand all responses are from origin server, there is no cache hit with or without store_miss and send_hit. Confusing part is when directly connected to server without proxy, flickr server does not send 301 response. When squid sends the same request somehow flickr server returns 301 with same URL.
>
> Ozgur
>
>
> On Fri, Jun 24, 2016 at 3:50 PM, Yuri <[hidden email] [hidden email]> wrote:
>
>     Try to do something like:
>
>
>     # 301 loop
>     acl text_mime rep_mime_type text/html text/plain
>
>     acl http301 http_status 301
>
>     store_miss deny text_mime http301
>     send_hit deny text_mime http301
>
>
>     24.06.2016 18:14, Ozgur Batur пишет:
>>     I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen.
>>
>>     I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.
>>
>>     I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.
>>
>>
>>     Ozgur
>>
>>
>>     _______________________________________________
>>     squid-users mailing list
>>     [hidden email] [hidden email]
>>     http://lists.squid-cache.org/listinfo/squid-users
>
>
>     _______________________________________________
>     squid-users mailing list
>     [hidden email] [hidden email]
>     http://lists.squid-cache.org/listinfo/squid-users
>
>


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXbUaJAAoJENNXIZxhPexGGpwIAK4mYSAoZbIU96VbS3L/Xq+f
6taPqkZrvy9JPU3aS92qE0bSuJFjtQrJ9lz8W8zAygeljyhCgwct9/9qBCy1gX25
7Z6qJj4UTfS7dIxb5NnAq2CHovuKiqvv6HThBqQ9J8/bq3jYk7u3rNK60ZEMK2Wg
sHaVLDiJMVu9gFCiYWlaPnBpFvse20gqybwhrhysjdM94HWAGOT9Oe+YWxIdB+Fj
lq1Udt3i4EvHrz4tOOgf5gggUVTBk7VttcKhgko9hI+KnfL3S2Yk2phzWX4apVt4
aDV/LKzb8vU33jOR9fV/sIOS0TyeBcIm3lokDWNfjB1SEjxQxXNPI1iOVggQv0Q=
=Sr78
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Rafael Akchurin
In reply to this post by Ozgur Batur

Hello Ozgur, Yuri,

 

I also see this error. Actually it is even present on videos.yahoo.com if I am not mistaken.

The reason for this is unclear for me (incorrect handling of “Via” header by some of back office servers of Yahoo???)

 

I was able to fix it by setting “via off” in squid.conf. I am not sure if this is the recommended way ( I presume not) and how to disable Via only for yahoo servers. Hopefully Amos has better answers.

 

Via looks like:

 

Via:"http/1.1 fts110.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"

 

Best regards,

Rafael Akchurin

Diladele B.V.

 

From: squid-users [mailto:[hidden email]] On Behalf Of Ozgur Batur
Sent: Friday, June 24, 2016 4:23 PM
To: Yuri <[hidden email]>
Cc: [hidden email]
Subject: Re: [squid-users] flickr.com redirect error

 

Hi Yuri,

 

Thank you. I put the #301 loop directives and restarted squid unfortunately result is the same. Here is the access logs:

 

1466777191.791    235 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.031    237 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.386    352 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.612    223 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

...

 

As I understand all responses are from origin server, there is no cache hit with or without store_miss and send_hit. Confusing part is when directly connected to server without proxy, flickr server does not send 301 response. When squid sends the same request somehow flickr server returns 301 with same URL. 

 

Ozgur

 

 

On Fri, Jun 24, 2016 at 3:50 PM, Yuri <[hidden email]> wrote:

Try to do something like:

 

# 301 loop
acl text_mime rep_mime_type text/html text/plain

acl http301 http_status 301

store_miss deny text_mime http301
send_hit deny text_mime http301

 

24.06.2016 18:14, Ozgur Batur пишет:

I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen. 
 
I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.
 
I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.

 

 

Ozgur

 

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Ozgur Batur
Hi Rafael, Yuri,

Thank you very much, "via off" did the trick. It is probably a server specific issue as you said.

Best Regards,

On Fri, Jun 24, 2016 at 6:29 PM, Rafael Akchurin <[hidden email]> wrote:

Hello Ozgur, Yuri,

 

I also see this error. Actually it is even present on videos.yahoo.com if I am not mistaken.

The reason for this is unclear for me (incorrect handling of “Via” header by some of back office servers of Yahoo???)

 

I was able to fix it by setting “via off” in squid.conf. I am not sure if this is the recommended way ( I presume not) and how to disable Via only for yahoo servers. Hopefully Amos has better answers.

 

Via looks like:

 

Via:"http/1.1 fts110.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"

 

Best regards,

Rafael Akchurin

Diladele B.V.

 

From: squid-users [mailto:[hidden email]] On Behalf Of Ozgur Batur
Sent: Friday, June 24, 2016 4:23 PM
To: Yuri <[hidden email]>
Cc: [hidden email]
Subject: Re: [squid-users] flickr.com redirect error

 

Hi Yuri,

 

Thank you. I put the #301 loop directives and restarted squid unfortunately result is the same. Here is the access logs:

 

1466777191.791    235 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.031    237 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.386    352 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

1466777192.612    223 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 text/html

...

 

As I understand all responses are from origin server, there is no cache hit with or without store_miss and send_hit. Confusing part is when directly connected to server without proxy, flickr server does not send 301 response. When squid sends the same request somehow flickr server returns 301 with same URL. 

 

Ozgur

 

 

On Fri, Jun 24, 2016 at 3:50 PM, Yuri <[hidden email]> wrote:

Try to do something like:

 

# 301 loop
acl text_mime rep_mime_type text/html text/plain

acl http301 http_status 301

store_miss deny text_mime http301
send_hit deny text_mime http301

 

24.06.2016 18:14, Ozgur Batur пишет:

I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen. 
 
I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.
 
I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.

 

 

Ozgur

 

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

 




--
H Özgür Batur

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
Be careful, guys. Via is reauired to HTTP by RFC.


24.06.2016 21:40, Ozgur Batur пишет:
> Hi Rafael, Yuri,
>
> Thank you very much, "via off" did the trick. It is probably a server specific issue as you said.
>
> Best Regards,
>
> On Fri, Jun 24, 2016 at 6:29 PM, Rafael Akchurin <[hidden email] [hidden email]> wrote:
>
>     Hello Ozgur, Yuri,
>
>     
>
>     I also see this error. Actually it is even present on videos.yahoo.com <http://videos.yahoo.com> if I am not mistaken.
>
>     The reason for this is unclear for me (incorrect handling of “Via” header by some of back office servers of Yahoo???)
>
>     
>
>     I was able to fix it by setting “via off” in squid.conf. I am not sure if this is the recommended way ( I presume not) and how to disable Via only for yahoo servers. Hopefully Amos has better answers.
>
>     
>
>     Via looks like:
>
>     
>
>     Via:"http/1.1 fts110.flickr.bf1.yahoo.com <http://fts110.flickr.bf1.yahoo.com> (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net <http://r02.ycpi.ams.yahoo.net> (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"
>
>     
>
>     Best regards,
>
>     Rafael Akchurin
>
>     Diladele B.V.
>
>     
>
>     *From:*squid-users [[hidden email] [hidden email]] *On Behalf Of *Ozgur Batur
>     *Sent:* Friday, June 24, 2016 4:23 PM
>     *To:* Yuri <[hidden email] [hidden email]>
>     *Cc:* [hidden email] [hidden email]
>     *Subject:* Re: [squid-users] flickr.com <http://flickr.com> redirect error
>
>     
>
>     Hi Yuri,
>
>     
>
>     Thank you. I put the #301 loop directives and restarted squid unfortunately result is the same. Here is the access logs:
>
>     
>
>     1466777191.791    235 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
>
>     1466777192.031    237 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
>
>     1466777192.386    352 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
>
>     1466777192.612    223 ::1 TCP_MISS/301 987 GET https://www.flickr.com/ - HIER_DIRECT/188.125.93.100 <http://188.125.93.100> text/html
>
>     ...
>
>     
>
>     As I understand all responses are from origin server, there is no cache hit with or without store_miss and send_hit. Confusing part is when directly connected to server without proxy, flickr server does not send 301 response. When squid sends the same request somehow flickr server returns 301 with same URL.
>
>     
>
>     Ozgur
>
>     
>
>     
>
>     On Fri, Jun 24, 2016 at 3:50 PM, Yuri <[hidden email] [hidden email]> wrote:
>
>         Try to do something like:
>
>         
>
>         # 301 loop
>         acl text_mime rep_mime_type text/html text/plain
>
>         acl http301 http_status 301
>
>         store_miss deny text_mime http301
>         send_hit deny text_mime http301
>
>         
>
>         24.06.2016 18:14, Ozgur Batur пишет:
>
>             I receive too many redirects(301 responses with same page URL) error on browser when opening https://www.flickr.com via Squid 3.5 proxy with SSL interception. If I connect to flickr website directly without Squid error does not happen.
>
>             
>
>             I tested it on two different systems one is Centos other is Ubuntu. There is no acl, redirect or any other configuration in squid.conf except enabling SSL interception.
>
>             
>
>             I opened http://bugs.squid-cache.org/show_bug.cgi?id=4537 for this issue but later thought it is better to ask if you also experience the same issue.
>
>             
>
>             
>
>             Ozgur
>
>             
>
>             _______________________________________________
>
>             squid-users mailing list
>
>             [hidden email] [hidden email]
>
>             http://lists.squid-cache.org/listinfo/squid-users
>
>         
>
>
>         _______________________________________________
>         squid-users mailing list
>         [hidden email] [hidden email]
>         http://lists.squid-cache.org/listinfo/squid-users
>
>     
>
>
>
>
> --
> H Özgür Batur


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXbVmoAAoJENNXIZxhPexGPFgH/ib6RKjQ/JhhnvTtBQnM6euV
+F6e/rrf6B295OpsrUgqFdogmCshJZGivdSBd8266KPOlvxE3I0F01SNBtAt96wC
1pL3Sam+TmFwbOGa5vYStQ+ZAkn5ReiSHppKVdeR1lXxBlMuhcDJovIxDtXvVV5G
SZcmJWT1q+LS8vcS+mGybXOt0H7J32sSUyor+qJ0CZEfG5HEPb1XKjave1mJNxUj
JEwsL0/B5zVw8LtL2yOzZY7E3ERY0r2ieGqQ4GpzYUVoDwoc5q8xwKaU08j5qyrP
iS2fW8wbAZ2RoZmvJRxnFpFKel0NgzwrAOUeSAs8hPONUUpWaklFTL55lezNY+A=
=t07f
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Amos Jeffries
Administrator
In reply to this post by Ozgur Batur
On 25/06/2016 3:40 a.m., Ozgur Batur wrote:
> Hi Rafael, Yuri,
>
> Thank you very much, "via off" did the trick. It is probably a server
> specific issue as you said.
>

Hmm. What was the Via header emitted by your proxy?

There are some common misconfigurations that can lead to a broken Via
being sent and various resulting strange behaviour.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Amos Jeffries
Administrator
In reply to this post by Yuri Voinov
On 25/06/2016 4:02 a.m., Yuri Voinov wrote:
>
> Be careful, guys. Via is reauired to HTTP by RFC.
>

As of RFC 7230 et al, it is officially now optional. Yay!

As of Squid-3.2 emitting HTTP/1.1, its use in preventing 1.1<->1.0
translation errors is greatly reduced. Yay!

It is still important to avoid forwarding loops though. So interceptors
and complex hierarchy installations are advised to enable it where
possible. Just for safety though, not RFC compliance.

[somewhere down on my to-do list is making Squid be a bit more flexible
that on vs off for that header].

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Rafael Akchurin
In reply to this post by Amos Jeffries
Hello Amos,

The Via from mine is:

Via:"http/1.1 fts110.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"

Might it be the error when constructing via contents in squid? As it starts with 1.1 while other constructed by Yahoo all start with http/1.1 ?

Best regards,
Rafael

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Amos Jeffries
Sent: Saturday, June 25, 2016 8:05 AM
To: [hidden email]
Subject: Re: [squid-users] flickr.com redirect error

On 25/06/2016 3:40 a.m., Ozgur Batur wrote:
> Hi Rafael, Yuri,
>
> Thank you very much, "via off" did the trick. It is probably a server
> specific issue as you said.
>

Hmm. What was the Via header emitted by your proxy?

There are some common misconfigurations that can lead to a broken Via being sent and various resulting strange behaviour.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Amos Jeffries
Administrator
On 25/06/2016 6:14 p.m., Rafael Akchurin wrote:
> Hello Amos,
>
> The Via from mine is:
>
> Via:"http/1.1 fts110.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"
>
> Might it be the error when constructing via contents in squid? As it starts with 1.1 while other constructed by Yahoo all start with http/1.1 ?
>

I think thats the Via on the reply coming back, not the request going out.

If that is actually your outgoing Via header *to* Yahoo. Then it says
the message has already been through their service. Thus a loop.

If Yahoo have any machine whose private hostname is "qlproxy" then your
Via header will match that machine (or qlproxy.*.yahoo.com) and again
they will detect a loop.

==> this will be true on whatever the outgoing Via really is from your
"qlproxy" proxies.

==> This is one of several reasons why I keep saying the
visible_hostname is *required* to be a FQDN, not a local one-label name.
And why Squid attempts to validate any auto-detected value in DNS before
using them.


What I'm expecting to see in Ozgur's header is either "localhost" or a
simple one-label name like yours which might match something inside the
private portion of the recipients CDN network.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Ozgur Batur
Hello Amos,

This is the via header sent by my local proxy as part of the request. 
Via: 1.1 ubuntuozgen (squid/3.5.19)

It is not fqdn but ubuntu concatanated with a Turkish name so it is highly unlikely that yahoo have such named reverse proxy. I could not decrypt the squid <--> flicker traffic yet this is from pcap output from another http site but i think it should be same right?

Thanks.

On Sat, Jun 25, 2016 at 3:10 PM, Amos Jeffries <[hidden email]> wrote:
On 25/06/2016 6:14 p.m., Rafael Akchurin wrote:
> Hello Amos,
>
> The Via from mine is:
>
> Via:"http/1.1 fts110.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r02.ycpi.ams.yahoo.net (ApacheTrafficServer [cMsSf ]), 1.1 qlproxy (squid/3.3.8)"
>
> Might it be the error when constructing via contents in squid? As it starts with 1.1 while other constructed by Yahoo all start with http/1.1 ?
>

I think thats the Via on the reply coming back, not the request going out.

If that is actually your outgoing Via header *to* Yahoo. Then it says
the message has already been through their service. Thus a loop.

If Yahoo have any machine whose private hostname is "qlproxy" then your
Via header will match that machine (or qlproxy.*.yahoo.com) and again
they will detect a loop.

==> this will be true on whatever the outgoing Via really is from your
"qlproxy" proxies.

==> This is one of several reasons why I keep saying the
visible_hostname is *required* to be a FQDN, not a local one-label name.
And why Squid attempts to validate any auto-detected value in DNS before
using them.


What I'm expecting to see in Ozgur's header is either "localhost" or a
simple one-label name like yours which might match something inside the
private portion of the recipients CDN network.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users



--
H Özgür Batur

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Amos Jeffries
Administrator
On 27/06/2016 9:04 p.m., Ozgur Batur wrote:
> Hello Amos,
>
> This is the via header sent by my local proxy as part of the request.
> *Via: 1.1 ubuntuozgen (squid/3.5.19)*
>
> It is not fqdn but ubuntu concatanated with a Turkish name so it is highly
> unlikely that yahoo have such named reverse proxy. I could not decrypt the
> squid <--> flicker traffic yet this is from pcap output from another http
> site but i think it should be same right?

Yes pcap (with full packet data) should contain the same needed details
yes. cache.log with debug level 11,2 is the easier way to get the
headers though since the crypto is removed by Squid.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Ozgur Batur
Yes that is much easier, thank you. 

Rafaels line is response header, I received the same. Here is the related cachelog:

2016/06/27 13:52:49.194 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server REQUEST:
GET / HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: tr,en-US;q=0.8,en;q=0.6
...
Via: 1.1 ubuntuozgen (squid/3.5.19)
Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"
X-Forwarded-For: ::1
Cache-Control: max-age=0
Connection: keep-alive

..
2016/06/27 13:52:49.477 kid1| 11,2| http.cc(751) processReplyHeader: HTTP Server REPLY:
---------
HTTP/1.1 301 Moved Permanently
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, max-age=0, must-revalidate, no-store
Pragma: no-cache
X-Request-Id: 36e709a2
Vary: Accept
Content-Type: text/html; charset=utf-8
Content-Length: 102
Server: ATS
Date: Mon, 27 Jun 2016 10:52:40 GMT
Age: 0
Via: http/1.1 fts111.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]), http/1.1 r11.ycpi.dea.yahoo.net (ApacheTrafficServer [cMs f ])
Connection: keep-alive
..

And this repeats on and on. As I understand disabling Via header is an acceptable solution. If I could disable the header only for problematic domains that would be better of course. 

Thank you all. 

On Mon, Jun 27, 2016 at 1:39 PM, Amos Jeffries <[hidden email]> wrote:
On 27/06/2016 9:04 p.m., Ozgur Batur wrote:
> Hello Amos,
>
> This is the via header sent by my local proxy as part of the request.
> *Via: 1.1 ubuntuozgen (squid/3.5.19)*
>
> It is not fqdn but ubuntu concatanated with a Turkish name so it is highly
> unlikely that yahoo have such named reverse proxy. I could not decrypt the
> squid <--> flicker traffic yet this is from pcap output from another http
> site but i think it should be same right?

Yes pcap (with full packet data) should contain the same needed details
yes. cache.log with debug level 11,2 is the easier way to get the
headers though since the crypto is removed by Squid.

Amos




--
H Özgür Batur

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Amos Jeffries
Administrator
On 27/06/2016 11:01 p.m., Ozgur Batur wrote:
> Yes that is much easier, thank you.
>
> Rafaels line is response header, I received the same. Here is the related
> cachelog:
>

What is the content of the line above this one. With the IP:port details ?

> 2016/06/27 13:52:49.194 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server
> REQUEST:
> GET / HTTP/1.1
> Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36
> Accept-Encoding: gzip, deflate, sdch
> Accept-Language: tr,en-US;q=0.8,en;q=0.6
> ...
> Host: www.flickr.com
> Via: 1.1 ubuntuozgen (squid/3.5.19)
> Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"
> X-Forwarded-For: ::1

You said this was using interception. But Squid XFF is telling Yahoo
that its receiving localhost traffic.

Try "forwarded_for transparent" in your squid.conf, and find out why
that ::1 is happening on an intercepted proxy. There may be a bug in
your NAT or routing configuration.


> Cache-Control: max-age=0
> Connection: keep-alive
>
> ..
> 2016/06/27 13:52:49.477 kid1| 11,2| http.cc(751) processReplyHeader: HTTP
> Server REPLY:
> ---------
> HTTP/1.1 301 Moved Permanently
> X-Frame-Options: SAMEORIGIN
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> X-Served-By: pprd1-node552-lh1.manhattan.bf1.yahoo.com
> X-Instance: flickr.v1.production.manhattan.bf1.yahoo.com
> Cache-Control: no-cache, max-age=0, must-revalidate, no-store
> Pragma: no-cache
> X-Request-Id: 36e709a2
> Location: https://www.flickr.com/
> Vary: Accept
> Content-Type: text/html; charset=utf-8
> Content-Length: 102
> Server: ATS
> Date: Mon, 27 Jun 2016 10:52:40 GMT
> Age: 0
> Via: http/1.1 fts111.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]),
> http/1.1 r11.ycpi.dea.yahoo.net (ApacheTrafficServer [cMs f ])
> Connection: keep-alive
> ..
>
> And this repeats on and on. As I understand disabling Via header is an
> acceptable solution. If I could disable the header only for problematic
> domains that would be better of course.

Okay. Unfortunately not possible. If that forwarded_for change works it
would be better than disabling Via.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Ozgur Batur

Browser i used to test runs on same machine with squid,  i changed it to explicit mode(no intercept - I set proxy ip in browser) during my attempts for ssl interception. Sorry I forgot to mention that in my last post of logs. So xff localhost is normal I guess. Here is the request log with  port info:

----------

2016/06/27 15:49:40.909 kid1| 11,2| http.cc(2234) sendRequest: HTTP Server local=10.100.136.56:47772 remote=188.125.93.100:443 FD 47 flags=1

2016/06/27 15:49:40.909 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server REQUEST:

---------

GET / HTTP/1.1

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8

Upgrade-Insecure-Requests: 1

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36

Accept-Encoding: gzip, deflate, sdch

Accept-Language: tr,en-US;q=0.8,en;q=0.6

..

Host: www.flickr.com

Via: 1.1 ubuntuozgen (squid/3.5.19)

Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"

X-Forwarded-For: ::1

Cache-Control: max-age=259200

Connection: keep-alive



On Mon, Jun 27, 2016 at 2:27 PM, Amos Jeffries <[hidden email]> wrote:
On 27/06/2016 11:01 p.m., Ozgur Batur wrote:
> Yes that is much easier, thank you.
>
> Rafaels line is response header, I received the same. Here is the related
> cachelog:
>

What is the content of the line above this one. With the IP:port details ?

> 2016/06/27 13:52:49.194 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server
> REQUEST:
> GET / HTTP/1.1
> Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36
> Accept-Encoding: gzip, deflate, sdch
> Accept-Language: tr,en-US;q=0.8,en;q=0.6
> ...
> Host: www.flickr.com
> Via: 1.1 ubuntuozgen (squid/3.5.19)
> Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"
> X-Forwarded-For: ::1

You said this was using interception. But Squid XFF is telling Yahoo
that its receiving localhost traffic.

Try "forwarded_for transparent" in your squid.conf, and find out why
that ::1 is happening on an intercepted proxy. There may be a bug in
your NAT or routing configuration.


> Cache-Control: max-age=0
> Connection: keep-alive
>
> ..
> 2016/06/27 13:52:49.477 kid1| 11,2| http.cc(751) processReplyHeader: HTTP
> Server REPLY:
> ---------
> HTTP/1.1 301 Moved Permanently
> X-Frame-Options: SAMEORIGIN
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> X-Served-By: pprd1-node552-lh1.manhattan.bf1.yahoo.com
> X-Instance: flickr.v1.production.manhattan.bf1.yahoo.com
> Cache-Control: no-cache, max-age=0, must-revalidate, no-store
> Pragma: no-cache
> X-Request-Id: 36e709a2
> Location: https://www.flickr.com/
> Vary: Accept
> Content-Type: text/html; charset=utf-8
> Content-Length: 102
> Server: ATS
> Date: Mon, 27 Jun 2016 10:52:40 GMT
> Age: 0
> Via: http/1.1 fts111.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]),
> http/1.1 r11.ycpi.dea.yahoo.net (ApacheTrafficServer [cMs f ])
> Connection: keep-alive
> ..
>
> And this repeats on and on. As I understand disabling Via header is an
> acceptable solution. If I could disable the header only for problematic
> domains that would be better of course.

Okay. Unfortunately not possible. If that forwarded_for change works it
would be better than disabling Via.

Amos




--
H Özgür Batur

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: flickr.com redirect error

Eliezer Croitoru

Hey,

 

Can you test if the details at bug 4253:

http://bugs.squid-cache.org/show_bug.cgi?id=4253#c13

 

Helps you to resolve the issue?


Eliezer

 

----

Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]

 

From: squid-users [mailto:[hidden email]] On Behalf Of Ozgur Batur
Sent: Monday, June 27, 2016 6:02 PM
To: Amos Jeffries
Cc: [hidden email]
Subject: Re: [squid-users] flickr.com redirect error

 

Browser i used to test runs on same machine with squid,  i changed it to explicit mode(no intercept - I set proxy ip in browser) during my attempts for ssl interception. Sorry I forgot to mention that in my last post of logs. So xff localhost is normal I guess. Here is the request log with  port info:

----------

2016/06/27 15:49:40.909 kid1| 11,2| http.cc(2234) sendRequest: HTTP Server local=10.100.136.56:47772 remote=188.125.93.100:443 FD 47 flags=1

2016/06/27 15:49:40.909 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server REQUEST:

---------

GET / HTTP/1.1

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8

Upgrade-Insecure-Requests: 1

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36

Accept-Encoding: gzip, deflate, sdch

Accept-Language: tr,en-US;q=0.8,en;q=0.6

..

Host: www.flickr.com

Via: 1.1 ubuntuozgen (squid/3.5.19)

Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"

X-Forwarded-For: ::1

Cache-Control: max-age=259200

Connection: keep-alive

 

 

On Mon, Jun 27, 2016 at 2:27 PM, Amos Jeffries <[hidden email]> wrote:

On 27/06/2016 11:01 p.m., Ozgur Batur wrote:
> Yes that is much easier, thank you.
>
> Rafaels line is response header, I received the same. Here is the related
> cachelog:
>

What is the content of the line above this one. With the IP:port details ?


> 2016/06/27 13:52:49.194 kid1| 11,2| http.cc(2235) sendRequest: HTTP Server
> REQUEST:
> GET / HTTP/1.1
> Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> Upgrade-Insecure-Requests: 1
> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Ubuntu Chromium/50.0.2661.102 Chrome/50.0.2661.102 Safari/537.36
> Accept-Encoding: gzip, deflate, sdch
> Accept-Language: tr,en-US;q=0.8,en;q=0.6
> ...
> Host: www.flickr.com
> Via: 1.1 ubuntuozgen (squid/3.5.19)
> Surrogate-Capability: ubuntuozgen="Surrogate/1.0 ESI/1.0"
> X-Forwarded-For: ::1

You said this was using interception. But Squid XFF is telling Yahoo
that its receiving localhost traffic.

Try "forwarded_for transparent" in your squid.conf, and find out why
that ::1 is happening on an intercepted proxy. There may be a bug in
your NAT or routing configuration.



> Cache-Control: max-age=0
> Connection: keep-alive
>
> ..
> 2016/06/27 13:52:49.477 kid1| 11,2| http.cc(751) processReplyHeader: HTTP
> Server REPLY:
> ---------
> HTTP/1.1 301 Moved Permanently
> X-Frame-Options: SAMEORIGIN
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> X-Served-By: pprd1-node552-lh1.manhattan.bf1.yahoo.com
> X-Instance: flickr.v1.production.manhattan.bf1.yahoo.com
> Cache-Control: no-cache, max-age=0, must-revalidate, no-store
> Pragma: no-cache
> X-Request-Id: 36e709a2
> Location: https://www.flickr.com/
> Vary: Accept
> Content-Type: text/html; charset=utf-8
> Content-Length: 102
> Server: ATS
> Date: Mon, 27 Jun 2016 10:52:40 GMT
> Age: 0
> Via: http/1.1 fts111.flickr.bf1.yahoo.com (ApacheTrafficServer [cMs f ]),
> http/1.1 r11.ycpi.dea.yahoo.net (ApacheTrafficServer [cMs f ])
> Connection: keep-alive
> ..
>
> And this repeats on and on. As I understand disabling Via header is an
> acceptable solution. If I could disable the header only for problematic
> domains that would be better of course.

Okay. Unfortunately not possible. If that forwarded_for change works it

would be better than disabling Via.

Amos



 

--

H Özgür Batur


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

flickr.com redirect error

Garri Djavadyan
>Can you test if the details at bug 4253:
>
>http://bugs.squid-cache.org/show_bug.cgi?id=4253#c13
>
>Helps you to resolve the issue?
>
>Eliezer

The above bug is not related to the issue.

The issue is actually on origin servers side. Details can be found
here:

http://bugs.squid-cache.org/show_bug.cgi?id=4537#c3

Garri
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users