Squid log details - HTTPS tunnel detection

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid log details - HTTPS tunnel detection

Markus Moeller
Is it possible to log the bytes in and out of a connection made with the
CONNECT method. ? I am looking at identifying users misusing the SSL
connection as a "remote access" solution and was wondering if byte in/byte
out ratios could be used to identify the misuse without decrypting the
session.

Are there other known ways besides IP-address/hostname blacklisting to
identify HTTPS tunnels ?

Thank you
Markus



Reply | Threaded
Open this post in threaded view
|

Re: Squid log details - HTTPS tunnel detection

Henrik Nordström
ons 2007-05-23 klockan 17:46 +0100 skrev Markus Moeller:
> Is it possible to log the bytes in and out of a connection made with the
> CONNECT method. ? I am looking at identifying users misusing the SSL
> connection as a "remote access" solution and was wondering if byte in/byte
> out ratios could be used to identify the misuse without decrypting the
> session.

Squid only keeps a single total counter for CONNECT requests. To get
them split you need to extend the code to keep two counters.

> Are there other known ways besides IP-address/hostname blacklisting to
> identify HTTPS tunnels ?

Most isn't actually using SSL, so a IDS system looking for odd traffic
in CONNECT requests will trap many of them (but not all).
 
Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid log details - HTTPS tunnel detection

Markus Moeller

"Henrik Nordstrom" <[hidden email]> wrote in message
news:[hidden email]...

>ons 2007-05-23 klockan 17:46 +0100 skrev Markus Moeller:
>> Is it possible to log the bytes in and out of a connection made with the
>> CONNECT method. ? I am looking at identifying users misusing the SSL
>> connection as a "remote access" solution and was wondering if byte
>> in/byte
>> out ratios could be used to identify the misuse without decrypting the
>> session.
>
>Squid only keeps a single total counter for CONNECT requests. To get
>them split you need to extend the code to keep two counters.

Do you have a pointer where in the code I have to look for it ?

>
>> Are there other known ways besides IP-address/hostname blacklisting to
>> identify HTTPS tunnels ?
>
>Most isn't actually using SSL, so a IDS system looking for odd traffic
>in CONNECT requests will trap many of them (but not all).

Correct. But I am specifically interested in the bad guys which use SSL.

>
>Regards
>Henrik

Thank you
Markus



Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

Henrik Nordström
ons 2007-05-23 klockan 19:25 +0100 skrev Markus Moeller:

> >Squid only keeps a single total counter for CONNECT requests. To get
> >them split you need to extend the code to keep two counters.
>
> Do you have a pointer where in the code I have to look for it ?

There is a couple of different places..

The CONNECT traffic is all processed in ssl.c. The counter is updated in
sslWriteClient & sslWriteServer.

                *sslState->size_ptr += len;

This size_ptr is given to ssl.c as part of the sslStart() call from
client_side.c.

Finally client_size.c also hands the counters down to the access logging
code in the call to accessLogLog().

Regards
Henrik

signature.asc (316 bytes) Download Attachment
K K
Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

K K
In reply to this post by Markus Moeller
On 5/23/07, Markus Moeller <[hidden email]> wrote:
> "Henrik Nordstrom" <[hidden email]> wrote in message
> news:[hidden email]...
> >Most isn't actually using SSL, so a IDS system looking for odd traffic
> >in CONNECT requests will trap many of them (but not all).

Any chance of implementing basic "Is this CONNECT session really SSL?"
functionality in Squid?


> Correct. But I am specifically interested in the bad guys which use SSL.

I recall some (recent?) research on using Netflow and/or Argus to
identify unusual patterns of traffic flow.  Normal HTTP inside of SSL
produces a different pattern of query->response packets than does a
remote access tunnel, this can be detected by old school "traffic
analysis".

Another option is to route SSL through a commercial product which does
true SSL/TLS "interception", terminating the crypto in the analysis
box and then re-establishing a new SSL session to the Internet.  This
has *huge* implications for privacy, HIPAA, etc.

I've spoken with Bluecoat, Radware, Checkpoint, and others about
products in this space, but the whole idea gives me the willies.

Kevin
Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

Henrik Nordström
ons 2007-05-23 klockan 16:00 -0500 skrev K K:

> Another option is to route SSL through a commercial product which does
> true SSL/TLS "interception", terminating the crypto in the analysis
> box and then re-establishing a new SSL session to the Internet.  This
> has *huge* implications for privacy, HIPAA, etc.

Or hire a developer to add this to Squid. Not much missing to be honest.

> I've spoken with Bluecoat, Radware, Checkpoint, and others about
> products in this space, but the whole idea gives me the willies.

Privacy is a luxury. In some environments it's not something you are
allowed to have and in such environments these decrypting proxies makes
sense.

Have seen a number of large corporations where their security policy do
not allow encrypted communication between the internal LAN and Internet,
absolutely requiring the ability to inspect the traffic. Naturally these
also have enforced policies defining how Internet may be used at the
office, only allowing it to be used as part of the work and not for
private purposes.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid log details - HTTPS tunnel detection

Markus Moeller
In reply to this post by Henrik Nordström
FYI

With a modified squid (at the source Henrik pointed to) I get

Outgoing ssh (only command was ls and then exit)

1180183741.678   6328 127.0.0.1 TCP_MISS/200 7432 5036 2396 CONNECT
opensuse.suse.home:22 - DIRECT/192.168.1.7 -

5036 = Bytes written to client     (Inbound)
2396 = Bytes written to server    (Outbound)

Ratio 2.10 like normal surfing  which has a ratio > 1

The same as above only tunneled via stunnel to have real SSL connection
1180188642.907   7747 192.168.1.7 TCP_MISS/200 13380 9102 4278 CONNECT
opensuse.suse.home:443 - DIRECT/192.168.1.7 -

9102 = Bytes written to client     (Inbound)
4278 = Bytes written to server    (Outbound)

Ratio 2.13 like normal surfing  which has a ratio > 1


Normal HTTPS traffic looks like:

 1180183683.128    405 192.168.1.10 TCP_MISS/200 11177 9824 1353 CONNECT
www.hsbc.co.uk:443 - DIRECT/193.108.74.209 -
 1180183683.197    468 192.168.1.10 TCP_MISS/200 7561 6197 1364 CONNECT
www.hsbc.co.uk:443 - DIRECT/193.108.74.209 -

Ratio 7.26 and 4.54


Outgoing ssh with remote port forwarding and incoming ssh connection (only
command was ls and then exit). THIS IS A MISUSE EXAMPLE

1180183763.638  13448 127.0.0.1 TCP_MISS/200 15352 6076 9276 CONNECT
opensuse.suse.home:22 - DIRECT/192.168.1.7 -


6076 = Bytes written to client    (Inbound)
9276 = Bytes written to server   (Outbound)

Ratio 0.655

As expected a ratio < 1

The same as above only tunneled via stunnel to have real SSL connection
 1180188667.142  16940 192.168.1.7 TCP_MISS/200 22664 8863 13801 CONNECT
opensuse.suse.home:443 - DIRECT/192.168.1.7 -


8863 = Bytes written to client    (Inbound)
13801 = Bytes written to server   (Outbound)

Ratio 0.642

As expected a ratio < 1

So it looks like it could help determining malicious use of proxies even if
only few shell commands are executed.

Regards
Markus

"Henrik Nordstrom" <[hidden email]> wrote in message
news:[hidden email]...

>ons 2007-05-23 klockan 19:25 +0100 skrev Markus Moeller:
>
>> >Squid only keeps a single total counter for CONNECT requests. To get
>> >them split you need to extend the code to keep two counters.
>>
>> Do you have a pointer where in the code I have to look for it ?
>
>There is a couple of different places..
>
>The CONNECT traffic is all processed in ssl.c. The counter is updated in
>sslWriteClient & sslWriteServer.
>
>                *sslState->size_ptr += len;
>
>This size_ptr is given to ssl.c as part of the sslStart() call from
>client_side.c.
>
>Finally client_size.c also hands the counters down to the access logging
>code in the call to accessLogLog().
>
>Regards
>Henrik



Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

Adrian Chadd
You might want to include mean/median/distribution of read/write IO
sizes on SSL connections; you might find 'normal' SSL accesses
(even with AJAXed stuff?) has different access patterns versus command-line
SSL.

Are there any fingerprint bits in the SSL exchange which would tell
you its at least SSL encrypted traffic, versus just traffic not tunneled
inside SSL? Thats probably a good starting point.



Adrian

Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

Henrik Nordström
In reply to this post by Markus Moeller
mån 2007-05-28 klockan 14:44 +0100 skrev Markus Moeller:

> So it looks like it could help determining malicious use of proxies even if
> only few shell commands are executed.

Don't forget POST requests, which may give any ratio <> 1 depending on
the use..

Someone POST:ing a large file to a simple page (or smaller than the
POST:ed data): < 1

Someone POST:ing small amount to a large page: > 1


And with all the Web2.0 stuff being done these days you'll never really
know..

A packet size distribution might work more reliably. ssh, imap, pop etc
has a lot of very small command packets, while HTTP with it's larger
syntax nearly always has quite big packets..


Another question: Would you be interested in contributing your code
changes? Others might be interested in this for statistics purposes.

Regards
Henrik

signature.asc (316 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Re: Squid log details - HTTPS tunnel detection

Henrik Nordström
In reply to this post by Adrian Chadd
tis 2007-05-29 klockan 00:18 +0800 skrev Adrian Chadd:

> Are there any fingerprint bits in the SSL exchange which would tell
> you its at least SSL encrypted traffic, versus just traffic not tunneled
> inside SSL? Thats probably a good starting point.

The initial hello message exchange isn't too hard to identify. But there
is a couple different ones (SSLv2, SSLv3, TLS), and who knows what the
future revisions will look like..

One very trivial thing which doesn't require any payload inspection byt
yet would block at least SSH, SMTP, POP and IMAP is to require the
client to send the first packet. The SSH protocols all start with the
client sending a hello message, while in most Internet application
protocols it's the server which sends the hello message..

Regards
Henrik

signature.asc (316 bytes) Download Attachment