debug headers between squid --> website

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

debug headers between squid --> website

Ahmad Alzaeem
Hello Tem ,

How can i debug Headers that is between squid——> website request made 

say we have this simple topology 

pc ——squid —— website


—> As an example if i run curl  some website   from my device connecting to squid proxy .


$ curl -x  x.x.8.187:xx433 -U abc:abc ifconfig.io/ip  -vv
*   Trying 108.61.8.187...
* TCP_NODELAY set
* Connected to x.x.8.187 (x.x.8.187) port xx433 (#0)
* Proxy auth using Basic with user 'ben'
> GET http://ifconfig.io/ip HTTP/1.1
> Host: ifconfig.io
> Proxy-Authorization: Basic YmVuOmJlbg==
> User-Agent: curl/7.54.0
> Accept: */*
> Proxy-Connection: Keep-Alive
< HTTP/1.1 200 OK
< Date: Mon, 02 Dec 2019 17:30:42 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 40
< Set-Cookie: __cfduid=d639c4bd01a9f8c32f0de7cb09f40671575307842; expires=Wed, 01-Jan-20 17:30:42 GMT; path=/; domain=.ifconfig.io; HttpOnly
< CF-Cache-Status: DYNAMIC
< Alt-Svc: h3-23=":443"; ma=86400
< Server: cloudflare
< CF-RAY: 53ef07bd8d28efed-EWR
< X-Cache: MISS from squid
< Via: 1.1 xyz (squid)
< Connection: keep-alive
11.22.33.44
* Connection #0 to host x.x.8.187 left intact


i believe this is negotiation  above is from  pc <—> squid .


How can i see this kind of debug or header in case of squid— website level ?

i need to see what squid send headers to website 
and what website reply o squid .



Thanks 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Antony Stone
On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:

> Hello Tem ,
>
> How can i debug Headers that is between squid——> website request made

Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid server,
looking at the external interface (ie: the one pointing to the website/s).

> i need to see what squid send headers to website
> and what website reply o squid .

So long as you're doing HTTP (as per your example) and not HTTPS, any packet
sniffer and protocol analyser (wireshark is *very* good at this) will show you
this quite easily.


Antony.

--
Atheism is a non-prophet-making organisation.

                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Ahmad Alzaeem
Thank you for that .

Is it possible to run it from squid ?

Thanks

Sent from my iPhone

> On Dec 2, 2019, at 8:58 PM, Antony Stone <[hidden email]> wrote:
>
> On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:
>
>> Hello Tem ,
>>
>> How can i debug Headers that is between squid——> website request made
>
> Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid server,
> looking at the external interface (ie: the one pointing to the website/s).
>
>> i need to see what squid send headers to website
>> and what website reply o squid .
>
> So long as you're doing HTTP (as per your example) and not HTTPS, any packet
> sniffer and protocol analyser (wireshark is *very* good at this) will show you
> this quite easily.
>
>
> Antony.
>
> --
> Atheism is a non-prophet-making organisation.
>
>                                                   Please reply to the list;
>                                                         please *don't* CC me.
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Antony Stone
On Monday 02 December 2019 at 19:31:43, Ahmad Alzaeem wrote:

> Thank you for that .
>
> Is it possible to run it from squid ?

I don't understand that question.

You start Squid; it listens for incoming connections and sends them on to the
external servers (and gets the responses etc, etc...)

At the same time, you run the packet sniffer on the machine where Squid is
running, and it collects all the traffic passing between Squid and the rest of
the Internet.

Then you make your request/s with a browser (or wget, curl, as you wish), and
let Squid do its thing, and let the packet sniffer capture what happened.

After it's all over, you then have a packet capture which you can analyse (eg:
using wireshark) to find out what Squid sent to the server/s, and what came
back again.


Antony.

> > On Dec 2, 2019, at 8:58 PM, Antony Stone
> > <[hidden email]> wrote:
> >
> > On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:
> >> Hello Tem ,
> >>
> >> How can i debug Headers that is between squid——> website request made
> >
> > Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid server,
> > looking at the external interface (ie: the one pointing to the
> > website/s).
> >
> >> i need to see what squid send headers to website
> >> and what website reply o squid .
> >
> > So long as you're doing HTTP (as per your example) and not HTTPS, any
> > packet sniffer and protocol analyser (wireshark is *very* good at this)
> > will show you this quite easily.
> >
> >
> > Antony.

--
"It wouldn't be a good idea to talk about him behind his back in front of
him."

 - murble

                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Alex Rousskov
In reply to this post by Ahmad Alzaeem
On 12/2/19 1:31 PM, Ahmad Alzaeem wrote:

> Is it possible to run it from squid ?

Packet catpure is usually better, especially for plain HTTP traffic, but
you can also get raw HTTP headers in cache.log if you set debug_options
in squid.conf to ALL,2

Alex.


>> On Dec 2, 2019, at 8:58 PM, Antony Stone <[hidden email]> wrote:
>>
>> On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:
>>
>>> Hello Tem ,
>>>
>>> How can i debug Headers that is between squid——> website request made
>>
>> Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid server,
>> looking at the external interface (ie: the one pointing to the website/s).
>>
>>> i need to see what squid send headers to website
>>> and what website reply o squid .
>>
>> So long as you're doing HTTP (as per your example) and not HTTPS, any packet
>> sniffer and protocol analyser (wireshark is *very* good at this) will show you
>> this quite easily.
>>
>>
>> Antony.
>>
>> --
>> Atheism is a non-prophet-making organisation.
>>
>>                                                   Please reply to the list;
>>                                                         please *don't* CC me.
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Ahmad Alzaeem
Can I do same  thing for https ?

Thanks 

Sent from my iPhone

On Dec 2, 2019, at 10:03 PM, Alex Rousskov <[hidden email]> wrote:

On 12/2/19 1:31 PM, Ahmad Alzaeem wrote:

Is it possible to run it from squid ?

Packet catpure is usually better, especially for plain HTTP traffic, but
you can also get raw HTTP headers in cache.log if you set debug_options
in squid.conf to ALL,2

Alex.


On Dec 2, 2019, at 8:58 PM, Antony Stone <[hidden email]> wrote:

On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:

Hello Tem ,

How can i debug Headers that is between squid——> website request made

Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid server,
looking at the external interface (ie: the one pointing to the website/s).

i need to see what squid send headers to website
and what website reply o squid .

So long as you're doing HTTP (as per your example) and not HTTPS, any packet
sniffer and protocol analyser (wireshark is *very* good at this) will show you
this quite easily.


Antony.

--
Atheism is a non-prophet-making organisation.

                                                 Please reply to the list;
                                                       please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Alex Rousskov
On 12/2/19 2:19 PM, Ahmad Alzaeem wrote:
> Can I do same  thing for https ?

Yes, you can. Squid logs CONNECT headers and also HTTP headers of
incoming and outgoing decrypted HTTPS requests. Squid does not see (and
cannot log) HTTP headers of encrypted traffic inside CONNECT tunnels
that are not bumped using the SslBump feature, of course.

Wireshark is often a better tool for header analysis because it makes it
easier to associate headers with connections and HTTP request-reply
exchanges. Wireshark can even handle encrypted-by-Squid traffic, but
that requires connection master keys that are not trivial to obtain.

Alex.


>> On Dec 2, 2019, at 10:03 PM, Alex Rousskov wrote:
>>
>> On 12/2/19 1:31 PM, Ahmad Alzaeem wrote:
>>
>>> Is it possible to run it from squid ?
>>
>> Packet catpure is usually better, especially for plain HTTP traffic, but
>> you can also get raw HTTP headers in cache.log if you set debug_options
>> in squid.conf to ALL,2
>>
>> Alex.
>>
>>
>>>> On Dec 2, 2019, at 8:58 PM, Antony Stone
>>>> <[hidden email]> wrote:
>>>>
>>>> On Monday 02 December 2019 at 18:34:31, Ahmad Alzaeem wrote:
>>>>
>>>>> Hello Tem ,
>>>>>
>>>>> How can i debug Headers that is between squid——> website request made
>>>>
>>>> Run a packet sniffer (tcpdump, wireshark, tshark...) on the Squid
>>>> server,
>>>> looking at the external interface (ie: the one pointing to the
>>>> website/s).
>>>>
>>>>> i need to see what squid send headers to website
>>>>> and what website reply o squid .
>>>>
>>>> So long as you're doing HTTP (as per your example) and not HTTPS,
>>>> any packet
>>>> sniffer and protocol analyser (wireshark is *very* good at this)
>>>> will show you
>>>> this quite easily.
>>>>
>>>>
>>>> Antony.
>>>>
>>>> --
>>>> Atheism is a non-prophet-making organisation.
>>>>
>>>>                                                  Please reply to the
>>>> list;
>>>>                                                        please
>>>> *don't* CC me.
>>>> _______________________________________________
>>>> squid-users mailing list
>>>> [hidden email]
>>>> http://lists.squid-cache.org/listinfo/squid-users
>>> _______________________________________________
>>> squid-users mailing list
>>> [hidden email]
>>> http://lists.squid-cache.org/listinfo/squid-users
>>>
>>
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

--Ahmad--
Hi Alex ,

Thank you for your precious info .


You Said 
“”
Yes, you can. Squid logs CONNECT headers and also HTTP headers of
incoming and outgoing decrypted HTTPS requests. Squid does not see (and
cannot log) HTTP headers of encrypted traffic inside CONNECT tunnels
that are not bumped using the SslBump feature, of course.
“”


Can you tell me example of headers of “Connect headers” and headers inside “ connect Tunnel “ ?



On Dec 2, 2019, at 10:31 PM, Alex Rousskov <[hidden email]> wrote:

Yes, you can. Squid logs CONNECT headers and also HTTP headers of
incoming and outgoing decrypted HTTPS requests. Squid does not see (and
cannot log) HTTP headers of encrypted traffic inside CONNECT tunnels
that are not bumped using the SslBump feature, of course.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: debug headers between squid --> website

Alex Rousskov
On 12/2/19 5:22 PM, --Ahmad-- wrote:
> You Said 
> “”
> Yes, you can. Squid logs CONNECT headers and also HTTP headers of
> incoming and outgoing decrypted HTTPS requests. Squid does not see (and
> cannot log) HTTP headers of encrypted traffic inside CONNECT tunnels
> that are not bumped using the SslBump feature, of course.
> “”


> Can you tell me example of headers of “Connect headers” and headers
> inside “ connect Tunnel “ ?

CONNECT requests are described, with examples, at
https://tools.ietf.org/html/rfc7231#section-4.3.6

Any HTTP message (both headers and body) can be sent inside a CONNECT
tunnel.

Please note that when Squid is configured to intercept HTTPS/TLS
connections, it treats the intercepted TCP connection as if that
intercepted traffic was inside a CONNECT tunnel. Squid even fakes the
CONNECT request in that case as if the TLS client sent a CONNECT request
before securing the connection.

Alex.


>> On Dec 2, 2019, at 10:31 PM, Alex Rousskov wrote:
>>
>> Yes, you can. Squid logs CONNECT headers and also HTTP headers of
>> incoming and outgoing decrypted HTTPS requests. Squid does not see (and
>> cannot log) HTTP headers of encrypted traffic inside CONNECT tunnels
>> that are not bumped using the SslBump feature, of course.
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users