ICAP and 403 Encapsulated answers (SSL denied domains)

Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ICAP and 403 Encapsulated answers (SSL denied domains)

FredB-2
Hello all,

I'm playing with Squid4 and e2guardian as ICAP server.

I'm seeing something I misunderstand, when a SSL website is blocked
e2guardian returns a encapsulated "HTTP/1.1 403 Forbidden" header this
part seems good to me with an encrypted website a denied or redirection
page can't be added

But unfortunately Squid adds a "Connection: keep-alive" header and if I
just reload the page I'm waiting a timeout a long moment, (and there is
no ICAP request between squid and e2) it's like the previous connection
still opened.

So the first request is well denied, but the second is without answer

I tried to add "Connection: close" in encapsulated header from
e2guardian without more success, but anyway "Connection: close" value is
removed by squid

I'm doing something wrong ? This wastes connections and from user point
of view the proxy is (very) slow, for example with ADS filtering some
websites freezes

FI the request is well denied in squid and E2 logs

Maybe this is a bug, but I don't known if the issue is from Squid or E2
? What is the correct response from an ICAP server with a denied SSL
website request ?

Thank you

Fred



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

Alex Rousskov
On 1/21/19 3:35 AM, FredB wrote:

> I'm playing with Squid4 and e2guardian as ICAP server.
>
> I'm seeing something I misunderstand, when a SSL website is blocked
> e2guardian returns a encapsulated "HTTP/1.1 403 Forbidden" header this
> part seems good to me with an encrypted website a denied or redirection
> page can't be added

> But unfortunately Squid adds a "Connection: keep-alive" header

It is not clear _why_ you consider that header "unfortunate" and the
connection "wasted". That header may or may not be wrong, and the
connection may or may not be reusable, depending on many factors (that
you have not shared).


> and if I
> just reload the page I'm waiting a timeout a long moment, (and there is
> no ICAP request between squid and e2) it's like the previous connection
> still opened.
>
> So the first request is well denied, but the second is without answer

Can the browser reuse the connection after receiving the HTTP 403
(Forbidden) response? Does it? If you provide a sample of client-Squid
request and response headers (including CONNECT messages, if any), and
specify whether they were all sharing the same TCP connection, then we
may be able to assign the blame for the "timeout".

If (some of) the messages are encrypted, providing ALL,2 cache.log may
work. Otherwise, a packet capture (in pcap format) is probably the
easiest sharing method.


> I tried to add "Connection: close" in encapsulated header from
> e2guardian without more success, but anyway "Connection: close" value is
> removed by squid

Yes, by ICAP design, an ICAP service does not have direct control over
HTTP connections maintained by the host application (e.g., Squid).

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

FredB-2
Hello Alex


>> But unfortunately Squid adds a "Connection: keep-alive" header
> It is not clear _why_ you consider that header "unfortunate" and the
> connection "wasted". That header may or may not be wrong, and the
> connection may or may not be reusable, depending on many factors (that
> you have not shared).
>
Your are right, it's not clear for me too, the only thing I'm seeing
it's that a keep-alive is not present in my answer from ICAP but well
added in header to client, after that if there is a refresh the browser
waits for the page a long time

But perhaps this is not related to my issue


>
> work. Otherwise, a packet capture (in pcap format) is probably the
> easiest sharing method.
>

Here a short tcpdump trace
https://nas.traceroot.fr:8081/owncloud/index.php/s/egrcXnU3lxiU0mi

   1 - I'm surfing to the website https://www.toto.fr

   2 - I receive a 403 (blank page)

   3 - I refresh the page, and I wait a long time before timeout

A real issue is filtering ADS, surf to www.aaa.com and block www.bbb.com
(ads), there are multiple links to bbb in aaa, in this case www.aaa.com
never appears completely (or after a long time) the browser freeze and
still waiting bbb  (the name appears in bottom: waiting for bbb)


>
> Yes, by ICAP design, an ICAP service does not have direct control over
> HTTP connections maintained by the host application (e.g., Squid).

Yes it's what I saw and read in the rfc

Thank you

Fred

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

Alex Rousskov
On 1/22/19 1:22 AM, FredB wrote:

> Here a short tcpdump trace
> https://nas.traceroot.fr:8081/owncloud/index.php/s/egrcXnU3lxiU0mi
>
>   1 - I'm surfing to the website https://www.toto.fr

Yes (tcp.stream eq 30).


>   2 - I receive a 403 (blank page)

> HTTP/1.1 403 Forbidden
> Server: e2guardian
> Date: Mon, 21 Jan 2019 10:06:54 GMT
> X-Cache: MISS from proxyorion_test
> X-Cache-Lookup: NONE from proxyorion_test:3128
> Transfer-Encoding: chunked
> Via: 1.1 proxyorion_test (squid/4.5)
> Connection: keep-alive
>
> 0

Agreed. Frame 99 contains a well-formed HTTP 403 response with an empty
body. IIRC, popular browsers refuse to display 403 responses to CONNECT
requests. There is also nothing to display in your specific case because
the 403 response body is empty, but that is irrelevant.


> 3 - I refresh the page, and I wait a long time before timeout

The trace you posted does not seem to show this part AFAICT. Perhaps
your "refresh" was not forceful enough for the browser to open a new
connection. I do not know whether that part is important.


> A real issue is filtering ADS

Please note that it is your responsibility to reproduce the
real/relevant problem. Your current test case may be sufficient -- I do
not know -- but if it is _not_ sufficient, we may not be able to tell
that it is insufficient or irrelevant, and will chase ghosts.


In summary, the trace you posted does not seem to indicate a Squid
problem. That does not mean there is no problem. It only means this use
case does not seem to expose that problem from Squid point of view.

If you believe that the browser is waiting for Squid to send something
after those HTTP 403 response bytes, then it sounds like there is a
browser bug -- Squid sent a full/complete response AFAICT. You may be
able to learn more about browser needs by debugging the browser.


As a workaround, you can try disabling client-to-Squid persistent
connections (client_persistent_connections off) or changing your ICAP
service to produce a response with a non-empty 403 body.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

FredB-2

>
> As a workaround, you can try disabling client-to-Squid persistent
> connections (client_persistent_connections off) or changing your ICAP
> service to produce a response with a non-empty 403 body.


You are right this is a browser bug (firefox at least recent versions)
and this issue can be resolved by client_persistent_connections off
unfortunately non-empty body is not enough

I will post a bug report to firefox

I found nothing in documentation about client_persistent_connections off
impact, do you think this can be problematic with high load ?

Fred
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

Alex Rousskov
On 1/23/19 3:17 AM, FredB wrote:

> I found nothing in documentation about client_persistent_connections off
> impact, do you think this can be problematic with high load ?

Yes, disabling client-to-Squid persistent connections can increase load
on the proxy server. In SslBump environments that bump many connections,
such an increase can be drastic. IIRC, it may also break some
authenticaiton mechanisms that rely on connection persistency.

Alex.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

FredB-2
In reply to this post by FredB-2

Amos, Alex

I thought you might be interested, there was a bug in Firefox with huge impact for some configurations

https://bugzilla.mozilla.org/show_bug.cgi?id=1522093


Regards

Fredb



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: ICAP and 403 Encapsulated answers (SSL denied domains)

Alex Rousskov
On 2/19/19 1:40 AM, FredB wrote:
> there was a bug in Firefox with huge impact for some configurations
> https://bugzilla.mozilla.org/show_bug.cgi?id=1522093

Congratulations on getting that Firefox bug fixed!

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users