squid upgrade issue and tunnelled ssh connections

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

squid upgrade issue and tunnelled ssh connections

Simon Beale
Hi

I'm trying to upgrade our squid proxies from 3.1.19 to 3.4.2, and have hit
a problem where I can no longer proxy ssh/sftp connections through after
the upgrade.

For testing, I've heavily cut down my squid.conf, to the following
configuration on 3.1.19, 3.3.11 and 3.4.2:

=============================
http_access allow all
http_port 3128
cache_mem 2 GB
maximum_object_size_in_memory 4 MB
cache_dir ufs /var/cache/squid 10240 16 256
maximum_object_size 1 MB
cache_swap_low 80
refresh_pattern .               0       20%     4320
=============================

If I then try run the following ssh command:

ssh -oProxyCommand='nc -v -X connect -x SQUIDHOST:3128 %h %p' github.com

With squid 3.1.19, I log in straight away.
With squid 3.3.11 and 3.4.2, I get the error:

nc: Proxy error: "HTTP/1.1 200 Connection established"
ssh_exchange_identification: Connection closed by remote host

Looking in the logfiles, it's logged:

1389375458.633     89 10.147.82.2 TCP_MISS/200 0 CONNECT github.com:22 -
HIER_DIRECT/192.30.252.131 -

Is there some option I'm overlooking to enable me to do these tunnelled
SSH/SFTP connections, that was introduced after 3.1.19?

Thanks

Simon

Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Eliezer Croitoru
Hey Simon,

What is the output of "squid -v"?
It can be related to squid and not..
Can you ssh from the proxy machine?

Eliezer

On 10/01/14 19:45, Simon Beale wrote:
> Hi
>
> I'm trying to upgrade our squid proxies from 3.1.19 to 3.4.2, and have hit
> a problem where I can no longer proxy ssh/sftp connections through after
> the upgrade.
>
> For testing, I've heavily cut down my squid.conf, to the following
> configuration on 3.1.19, 3.3.11 and 3.4.2:


Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Amos Jeffries
Administrator
In reply to this post by Simon Beale
On 11/01/2014 6:45 a.m., Simon Beale wrote:

> Hi
>
> I'm trying to upgrade our squid proxies from 3.1.19 to 3.4.2, and have hit
> a problem where I can no longer proxy ssh/sftp connections through after
> the upgrade.
>
> For testing, I've heavily cut down my squid.conf, to the following
> configuration on 3.1.19, 3.3.11 and 3.4.2:
>
> =============================
> http_access allow all
> http_port 3128
> cache_mem 2 GB
> maximum_object_size_in_memory 4 MB
> cache_dir ufs /var/cache/squid 10240 16 256
> maximum_object_size 1 MB
> cache_swap_low 80
> refresh_pattern .               0       20%     4320
> =============================
>
> If I then try run the following ssh command:
>
> ssh -oProxyCommand='nc -v -X connect -x SQUIDHOST:3128 %h %p' github.com
>
> With squid 3.1.19, I log in straight away.
> With squid 3.3.11 and 3.4.2, I get the error:
>
> nc: Proxy error: "HTTP/1.1 200 Connection established"
> ssh_exchange_identification: Connection closed by remote host
>
> Looking in the logfiles, it's logged:
>
> 1389375458.633     89 10.147.82.2 TCP_MISS/200 0 CONNECT github.com:22 -
> HIER_DIRECT/192.30.252.131 -
>
> Is there some option I'm overlooking to enable me to do these tunnelled
> SSH/SFTP connections, that was introduced after 3.1.19?

That "HTTP/1.1 200 Connection established" is the HTTP response produced
by Squid after successfully opening the tunnel.
Is nc tool getting confused over the HTTP/1.1 version? (3.1 would emit
HTTP/1.0 label with the same message.)

The "ssh_exchange_identification: Connection closed by remote host"
seems to be the issue.


Amos
Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Simon Beale
In reply to this post by Eliezer Croitoru
Heya

Squid Cache: Version 3.4.2
configure options:  '--build=x86_64-unknown-linux-gnu' '--host=x86_64-unknown-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-eui' '--enable-follow-x-forwarded-for' '--enable-auth' '--enable-auth-basic=DB,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,getpwnam' '--enable-auth-ntlm=smb_lm,fake' '--enable-auth-digest=file,LDAP,eDirectory' '--enable-auth-negotiate=kerberos,wrapper' '--enable-external-acl-helpers=wbinfo_group,LDAP_group,AD_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,ufs,rock' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=squid' '--with-filedescriptors=32768' '--with-dl' '--with-openssl' '--with-pthreads' '--disable-arch-native' 'build_alias=x86_64-unknown-linux-gnu' 'host_alias=x86_64-unknown-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'LDFLAGS=-pie' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'PKG_CONFIG_PATH=/usr/lib64/pkgconfig:/usr/share/pkgconfig’

Yes, I can ssh from the proxy machine. I’ve also run the test trying to connect via squid to another host on the same vlan as the squid host, to ensure that there’s no random firewall/router/switch oddness going on, with the same results.

Thanks

Simon

On 11 Jan 2014, at 02:28, Eliezer Croitoru <[hidden email]> wrote:

> Hey Simon,
>
> What is the output of "squid -v"?
> It can be related to squid and not..
> Can you ssh from the proxy machine?
>
> Eliezer
>
> On 10/01/14 19:45, Simon Beale wrote:
>> Hi
>>
>> I'm trying to upgrade our squid proxies from 3.1.19 to 3.4.2, and have hit
>> a problem where I can no longer proxy ssh/sftp connections through after
>> the upgrade.
>>
>> For testing, I've heavily cut down my squid.conf, to the following
>> configuration on 3.1.19, 3.3.11 and 3.4.2:
>
>

Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Simon Beale
In reply to this post by Amos Jeffries

On 11 Jan 2014, at 02:54, Amos Jeffries <[hidden email]> wrote:

> On 11/01/2014 6:45 a.m., Simon Beale wrote:
>> Hi
>>
>> I'm trying to upgrade our squid proxies from 3.1.19 to 3.4.2, and have hit
>> a problem where I can no longer proxy ssh/sftp connections through after
>> the upgrade.
>>
>> For testing, I've heavily cut down my squid.conf, to the following
>> configuration on 3.1.19, 3.3.11 and 3.4.2:
>>
>> =============================
>> http_access allow all
>> http_port 3128
>> cache_mem 2 GB
>> maximum_object_size_in_memory 4 MB
>> cache_dir ufs /var/cache/squid 10240 16 256
>> maximum_object_size 1 MB
>> cache_swap_low 80
>> refresh_pattern .               0       20%     4320
>> =============================
>>
>> If I then try run the following ssh command:
>>
>> ssh -oProxyCommand='nc -v -X connect -x SQUIDHOST:3128 %h %p' github.com
>>
>> With squid 3.1.19, I log in straight away.
>> With squid 3.3.11 and 3.4.2, I get the error:
>>
>> nc: Proxy error: "HTTP/1.1 200 Connection established"
>> ssh_exchange_identification: Connection closed by remote host
>>
>> Looking in the logfiles, it's logged:
>>
>> 1389375458.633     89 10.147.82.2 TCP_MISS/200 0 CONNECT github.com:22 -
>> HIER_DIRECT/192.30.252.131 -
>>
>> Is there some option I'm overlooking to enable me to do these tunnelled
>> SSH/SFTP connections, that was introduced after 3.1.19?
>
> That "HTTP/1.1 200 Connection established" is the HTTP response produced
> by Squid after successfully opening the tunnel.
> Is nc tool getting confused over the HTTP/1.1 version? (3.1 would emit
> HTTP/1.0 label with the same message.)
>

Ahah! Yes, you’re right.

I’ve pulled down the source for nc and found that in HTTP proxy mode, it explicitly looks for the string "HTTP/1.0 200” in the response. Patching it to accept HTTP/1.1 as an alternative, it now will successfully make the ssh connection.

Cheers for that!

Simon
Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Eliezer Croitoru
In reply to this post by Amos Jeffries
So the issue is:
# nc -v -x192.168.10.1:3128 -Xconnect 213.151.33.10 22
nc: Proxy error: "HTTP/1.1 403 Forbidden"
# nc -v -x192.168.10.1:3128 -Xconnect 213.151.33.10 22
nc: Proxy error: "HTTP/1.1 200 Connection established"

So as far as I understand the nc tool is compatible with 1.0 while not
built for 1.1.

I discovered another issue while testing this bug(above 403).
I will open another thread after testing more.

Eliezer
Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Simon Beale
On 11 Jan 2014, at 18:48, Eliezer Croitoru <[hidden email]> wrote:

> So the issue is:
> # nc -v -x192.168.10.1:3128 -Xconnect 213.151.33.10 22
> nc: Proxy error: "HTTP/1.1 403 Forbidden"
> # nc -v -x192.168.10.1:3128 -Xconnect 213.151.33.10 22
> nc: Proxy error: "HTTP/1.1 200 Connection established"
>
> So as far as I understand the nc tool is compatible with 1.0 while not built for 1.1.

The version of nc/netcat that is shipped with RHEL5 and RHEL6 (and based on the OpenBSD implementation of netcat) has this problem with HTTP/1.1. The upstream OpenBSD implementation fixed this problem in September 2006, but it never made it in to the RHEL releases. I’ve discovered there was a redhat bug opened recently to get this fixed.

I haven’t explicitly tested it, but I think the version of nc included in OSX Mavericks may also suffer the same issue, based purely on running the ‘strings’ command against the nc binary.

Debian/Ubuntu appear to use the NMap implementation of netcat, which doesn’t appear to offer proxy support, so technically they aren’t affected :)

Simon

Reply | Threaded
Open this post in threaded view
|

Re: squid upgrade issue and tunnelled ssh connections

Eliezer Croitoru
Hey Simon,

The above is Debian\Ubuntu version of nc..
It seems like a regular netcat to me:
$ apt-cache showsrc netcat
Package: netcat
Binary: netcat, netcat-traditional
Version: 1.10-40
Priority: optional
Section: universe/net
Maintainer: Ruben Molina <[hidden email]>
Build-Depends: quilt, debhelper (>= 9)
Architecture: any all
Standards-Version: 3.9.3
Format: 3.0 (quilt)
Directory: pool/universe/n/netcat
Files:
  1c98a2db12fa49b1c2affb778524650e 1765 netcat_1.10-40.dsc
  7dc5c7450e708796395ffd746c197234 64363 netcat_1.10.orig.tar.bz2
  0c1cf52aa4159338295c1dd66f90281c 20390 netcat_1.10-40.debian.tar.bz2
Uploaders: Anibal Monsalve Salazar <[hidden email]>
Package-List:
  netcat deb net extra
  netcat-traditional deb net important
Checksums-Sha1:
  3dfa812efb7248188153a1465fefc8ae7ac95ed8 1765 netcat_1.10-40.dsc
  d46f89c122fbe9f7c383cebbc1c7c97b8d0e682c 64363 netcat_1.10.orig.tar.bz2
  7742b2276887ca83b55f16cd5909106805f1fcf7 20390
netcat_1.10-40.debian.tar.bz2
Checksums-Sha256:
  75fc93a94a3c2dbfcf9ae3e03345ad55d7cc4a80897f00d75a72aa2541ec826e 1765
netcat_1.10-40.dsc
  64913dc3f0b4a96c3ab04d062d84f28ba6854152c94344e3985458b2aebca3d5 64363
netcat_1.10.orig.tar.bz2
  c9c09d510d9e5fd34fa15c47b0b14fe1ed66cc7525576b2fec8318a72c3aa0d3 20390
netcat_1.10-40.debian.tar.bz2

So it's only in RH and CentOS while maybe effecting couple others.

Eliezer

On 12/01/14 01:10, Simon Beale wrote:
> Debian/Ubuntu appear to use the NMap implementation of netcat, which doesn’t appear to offer proxy support, so technically they aren’t affected:)
>
> Simon
>