host_verify_strict and wildcard SNI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

host_verify_strict and wildcard SNI

Steve Hill

I'm using a transparent proxy and SSL-peek and have hit a problem with
an iOS app which seems to be doing broken things with the SNI.

The app is making an HTTPS connection to a server and presenting an SNI
with a wildcard in it - i.e. "*.example.com".  I'm not sure if this
behaviour is actually illegal, but it certainly doesn't seem to make a
lot of sense to me.

Squid then internally generates a "CONNECT *.example.com:443" request
based on the peeked SNI, which is picked up by hostHeaderIpVerify().
Since *.example.com isn't a valid DNS name, Squid rejects the connection
on the basis that *.example.com doesn't match the IP address that the
client is connecting to.

Unfortunately, I can't see any way of working around the problem -
"host_verify_strict" is disabled, but according to the docs,
"For now suspicious intercepted CONNECT requests are always responded to
with an HTTP 409 (Conflict) error page."

As I understand it, turning host_verify_strict on causes problems with
CDNs which use DNS tricks for load balancing, so I'm not sure I
understand the rationale behind preventing it from being turned off for
CONNECT requests?

--
  - Steve Hill
    Technical Director
    Opendium Limited     http://www.opendium.com

Direct contacts:
    Instant messager: xmpp:[hidden email]
    Email:            [hidden email]
    Phone:            sip:[hidden email]

Sales / enquiries contacts:
    Email:            [hidden email]
    Phone:            +44-1792-824568 / sip:[hidden email]

Support contacts:
    Email:            [hidden email]
    Phone:            +44-1792-825748 / sip:[hidden email]
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
Sounds familiar.

Do you experience occasional problems with CloudFlare sites?


06.07.2016 20:36, Steve Hill пишет:
>
> I'm using a transparent proxy and SSL-peek and have hit a problem with
an iOS app which seems to be doing broken things with the SNI.
>
> The app is making an HTTPS connection to a server and presenting an
SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if this
behaviour is actually illegal, but it certainly doesn't seem to make a
lot of sense to me.
>
> Squid then internally generates a "CONNECT *.example.com:443" request
based on the peeked SNI, which is picked up by hostHeaderIpVerify().
Since *.example.com isn't a valid DNS name, Squid rejects the connection
on the basis that *.example.com doesn't match the IP address that the
client is connecting to.
>
> Unfortunately, I can't see any way of working around the problem -
"host_verify_strict" is disabled, but according to the docs,
> "For now suspicious intercepted CONNECT requests are always responded
to with an HTTP 409 (Conflict) error page."
>
> As I understand it, turning host_verify_strict on causes problems with
CDNs which use DNS tricks for load balancing, so I'm not sure I
understand the rationale behind preventing it from being turned off for
CONNECT requests?
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfV9aAAoJENNXIZxhPexGSaIH/0Q9/FiYOhBeoWIkppSU9joc
uE80bqZ9QP+e0MRcDWjsiZd6RmbcNj5+KnrFsjRLerFF42A5IZ6x9KzkswEz1sO5
CBz3gpUg9uJuTbS9WBEGmw+n1dL8nXSwpFhXM7wjb40m7cAGdFiF5DGdquj/b8bv
WgZMYREFXZaK49NunaEUIvx7DQHEqQaMLLYhQTIrTjIV1RWaiWFl5wLijfJKdpSK
MF/PK847dwmaoquzQPwVFLEuiEXyYpJMYEzQRiJhksklcW2qZRLw8LMDrj3Jrhiq
iKsB3lhyoQR1/SzXHCNxpVrZonQ4HN0LC1JeAbZteaFBYu2IH4Jd9CiTLHU4fqs=
=R47a
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Eliezer Croitoru
Hey Yuri,

These two subjects are not related directly to each other but they might have something in common.
Squid expects clients connections to meet the basic RFC6066 section 3:
https://tools.ietf.org/html/rfc6066#section-3

Which states that a host name should be there and the legal characters of a hostname from both rfc1035 and rc6066 are very speicifc.
If a specific software are trying to request a wrong sni name it's an issue in the client side request or software error handling and enforcement.
A http server would probably respond with a 4XX response code or the default certificate.
There are other options of course but the first thing to check is if the client is a real browser or some special creature that tries it's luck with a special form of ssl.
To my understanding host_verify_strict tries to enforce basic security levels while in a transparent proxy the rules will always change.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Yuri Voinov
Sent: Wednesday, July 6, 2016 10:43 PM
To: [hidden email]
Subject: Re: [squid-users] host_verify_strict and wildcard SNI


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
Sounds familiar.

Do you experience occasional problems with CloudFlare sites?


06.07.2016 20:36, Steve Hill пишет:
>
> I'm using a transparent proxy and SSL-peek and have hit a problem with
an iOS app which seems to be doing broken things with the SNI.
>
> The app is making an HTTPS connection to a server and presenting an
SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if this
behaviour is actually illegal, but it certainly doesn't seem to make a
lot of sense to me.
>
> Squid then internally generates a "CONNECT *.example.com:443" request
based on the peeked SNI, which is picked up by hostHeaderIpVerify().
Since *.example.com isn't a valid DNS name, Squid rejects the connection
on the basis that *.example.com doesn't match the IP address that the
client is connecting to.
>
> Unfortunately, I can't see any way of working around the problem -
"host_verify_strict" is disabled, but according to the docs,
> "For now suspicious intercepted CONNECT requests are always responded
to with an HTTP 409 (Conflict) error page."
>
> As I understand it, turning host_verify_strict on causes problems with
CDNs which use DNS tricks for load balancing, so I'm not sure I
understand the rationale behind preventing it from being turned off for
CONNECT requests?
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfV9aAAoJENNXIZxhPexGSaIH/0Q9/FiYOhBeoWIkppSU9joc
uE80bqZ9QP+e0MRcDWjsiZd6RmbcNj5+KnrFsjRLerFF42A5IZ6x9KzkswEz1sO5
CBz3gpUg9uJuTbS9WBEGmw+n1dL8nXSwpFhXM7wjb40m7cAGdFiF5DGdquj/b8bv
WgZMYREFXZaK49NunaEUIvx7DQHEqQaMLLYhQTIrTjIV1RWaiWFl5wLijfJKdpSK
MF/PK847dwmaoquzQPwVFLEuiEXyYpJMYEzQRiJhksklcW2qZRLw8LMDrj3Jrhiq
iKsB3lhyoQR1/SzXHCNxpVrZonQ4HN0LC1JeAbZteaFBYu2IH4Jd9CiTLHU4fqs=
=R47a
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
I know. Just asked. Since I am familiar with the standards.

07.07.2016 1:54, Eliezer Croitoru пишет:
> Hey Yuri,
>
> These two subjects are not related directly to each other but they might have something in common.
> Squid expects clients connections to meet the basic RFC6066 section 3:
> https://tools.ietf.org/html/rfc6066#section-3
>
> Which states that a host name should be there and the legal characters of a hostname from both rfc1035 and rc6066 are very speicifc.
> If a specific software are trying to request a wrong sni name it's an issue in the client side request or software error handling and enforcement.
> A http server would probably respond with a 4XX response code or the default certificate.
> There are other options of course but the first thing to check is if the client is a real browser or some special creature that tries it's luck with a special form of ssl.
> To my understanding host_verify_strict tries to enforce basic security levels while in a transparent proxy the rules will always change.
>
> Eliezer
>
> ----
> Eliezer Croitoru
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: [hidden email]
>
>
> -----Original Message-----
> From: squid-users [[hidden email]] On Behalf Of Yuri Voinov
> Sent: Wednesday, July 6, 2016 10:43 PM
> To: [hidden email]
> Subject: Re: [squid-users] host_verify_strict and wildcard SNI
>
>
> Sounds familiar.
>
> Do you experience occasional problems with CloudFlare sites?
>
>
> 06.07.2016 20:36, Steve Hill пишет:
>
> > I'm using a transparent proxy and SSL-peek and have hit a problem with
> an iOS app which seems to be doing broken things with the SNI.
>
> > The app is making an HTTPS connection to a server and presenting an
> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if this
> behaviour is actually illegal, but it certainly doesn't seem to make a
> lot of sense to me.
>
> > Squid then internally generates a "CONNECT *.example.com:443" request
> based on the peeked SNI, which is picked up by hostHeaderIpVerify().
> Since *.example.com isn't a valid DNS name, Squid rejects the connection
> on the basis that *.example.com doesn't match the IP address that the
> client is connecting to.
>
> > Unfortunately, I can't see any way of working around the problem -
> "host_verify_strict" is disabled, but according to the docs,
> > "For now suspicious intercepted CONNECT requests are always responded
> to with an HTTP 409 (Conflict) error page."
>
> > As I understand it, turning host_verify_strict on causes problems with
> CDNs which use DNS tricks for load balancing, so I'm not sure I
> understand the rationale behind preventing it from being turned off for
> CONNECT requests?
>
>
>
>


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfWanAAoJENNXIZxhPexGvqgH/2IuLJk7Aa4D7migO+zAFZ5p
AheNbsZcXjkT5eno1WqNkGuVROK/L97HHazYz/QvbZp4ioFJ4PZ40nHknP679KqH
RSiavQlKCmL0AuW6/ztAb7VJRbokUTRGJy39uG9ecw2uEvbS6Iq/LSAH8L9LYZgQ
vf1wd9y7iCVUDJDz++rl36XY6aqZK2u8mUVhlxoBFsOOLVSbupXIQuVEkdXL61Oo
Vrau9hUALBk5zWJ+PBlIIs578zIf36J9OhApBa/bR7/tNdVNYnB7uvSbhrgk3N1N
ChHbm2P2E1mgSMQycVW+I2E5+GvJRvi8K9wMD7TsSwWKJviY7KTS5SFyxDOY224=
=Ox2x
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Eliezer Croitoru
Hey Yuri,

I am not the "standards" guy but I do know that if something can be encoded
it can be "decoded".
There are special cases which needs special "spice" which sometimes is not
present here or there on the shelves.
To my disappointment and happiness there are very good products out there
which are not squid with much better fines invested in them.
I can clearly say that the Squid-Cache project is not the most "advanced"
piece of software in the market and I know that it cannot compare to let say
even 500 coding programmers work.
I have seen couple products that are open source which tries to provide
functionality which is similar to squid only in the protocol level and a
simple proxy with great luck.
Some of them are not as great as they might seems but I think that a young
programmer with enough investment can learn the required subjects to
implement a solution.
However, here admins, users, programmers can ask questions as they please
and I encourage to ask.
I try to answer as much as I can and in many cases my knowledge might not
be enough but I am trying to answer what I can with hope that it will help.
And unlike MD Doctors SysAdmins do not need to swear on something like "do
not harm" and I think it's a good aspect on things.

I am still looking for clues about cloudflare since I have yet to see the
person who hold the keys for them.

Eliezer

----
Eliezer Croitoru <http://ngtech.co.il/lmgtfy/>
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]
 

From: Yuri Voinov [mailto:[hidden email]]
Sent: Wednesday, July 6, 2016 11:15 PM
To: Eliezer Croitoru; [hidden email]
Subject: Re: [squid-users] host_verify_strict and wildcard SNI


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
I know. Just asked. Since I am familiar with the standards.

07.07.2016 1:54, Eliezer Croitoru пишет:
> Hey Yuri,

      >

      > These two subjects are not related directly to each other but
      they might have something in common.

      > Squid expects clients connections to meet the basic RFC6066
      section 3:

      > https://tools.ietf.org/html/rfc6066#section-3
<https://tools.ietf.org/html/rfc6066>

      >

      > Which states that a host name should be there and the legal
      characters of a hostname from both rfc1035 and rc6066 are very
      speicifc.

      > If a specific software are trying to request a wrong sni name
      it's an issue in the client side request or software error
      handling and enforcement.

      > A http server would probably respond with a 4XX response code
      or the default certificate.

      > There are other options of course but the first thing to
      check is if the client is a real browser or some special creature
      that tries it's luck with a special form of ssl.

      > To my understanding host_verify_strict tries to enforce basic
      security levels while in a transparent proxy the rules will always
      change.

      >

      > Eliezer

      >

      > ----

      > Eliezer Croitoru

      > Linux System Administrator

      > Mobile: +972-5-28704261

      > Email: [hidden email] <mailto:[hidden email]>

      >

      >

      > -----Original Message-----

      > From: squid-users
      [mailto:[hidden email]] On Behalf Of
      Yuri Voinov

      > Sent: Wednesday, July 6, 2016 10:43 PM

      > To: [hidden email]
<mailto:[hidden email]>

      > Subject: Re: [squid-users] host_verify_strict and wildcard
      SNI

      >

      >

      > Sounds familiar.

      >

      > Do you experience occasional problems with CloudFlare sites?

      >

      >

      > 06.07.2016 20:36, Steve Hill пишет:

      >

      > > I'm using a transparent proxy and SSL-peek and have hit
      a problem with

      > an iOS app which seems to be doing broken things with the
      SNI.

      >

      > > The app is making an HTTPS connection to a server and
      presenting an

      > SNI with a wildcard in it - i.e. "*.example.com".  I'm not
      sure if this

      > behaviour is actually illegal, but it certainly doesn't seem
      to make a

      > lot of sense to me.

      >

      > > Squid then internally generates a "CONNECT
      *.example.com:443" request

      > based on the peeked SNI, which is picked up by
      hostHeaderIpVerify().

      > Since *.example.com isn't a valid DNS name, Squid rejects the
      connection

      > on the basis that *.example.com doesn't match the IP address
      that the

      > client is connecting to.

      >

      > > Unfortunately, I can't see any way of working around the
      problem -

      > "host_verify_strict" is disabled, but according to the docs,

      > > "For now suspicious intercepted CONNECT requests are
      always responded

      > to with an HTTP 409 (Conflict) error page."

      >

      > > As I understand it, turning host_verify_strict on causes
      problems with

      > CDNs which use DNS tricks for load balancing, so I'm not sure
      I

      > understand the rationale behind preventing it from being
      turned off for

      > CONNECT requests?

      >

      >

      >

      >

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfWanAAoJENNXIZxhPexGvqgH/2IuLJk7Aa4D7migO+zAFZ5p
AheNbsZcXjkT5eno1WqNkGuVROK/L97HHazYz/QvbZp4ioFJ4PZ40nHknP679KqH
RSiavQlKCmL0AuW6/ztAb7VJRbokUTRGJy39uG9ecw2uEvbS6Iq/LSAH8L9LYZgQ
vf1wd9y7iCVUDJDz++rl36XY6aqZK2u8mUVhlxoBFsOOLVSbupXIQuVEkdXL61Oo
Vrau9hUALBk5zWJ+PBlIIs578zIf36J9OhApBa/bR7/tNdVNYnB7uvSbhrgk3N1N
ChHbm2P2E1mgSMQycVW+I2E5+GvJRvi8K9wMD7TsSwWKJviY7KTS5SFyxDOY224=
=Ox2x
-----END PGP SIGNATURE-----

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

winmail.dat (90K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
I am very seriously concerned about the issue CDN, because every day I discover more and more problematic sites, namely in connection with the CDN and HTTPS. For more than four Squid servers are experiencing these problems in my network. And I still do not see any reason why any solutions to these problems.

Moreover, the splice does not solve these problems.

Just skip the whole networks in the proxy bypass.

What is totally unacceptable. Traffic is money. And a lot of money.

07.07.2016 2:38, Eliezer Croitoru пишет:
> Hey Yuri,
>
> I am not the "standards" guy but I do know that if something can be encoded
> it can be "decoded".
> There are special cases which needs special "spice" which sometimes is not
> present here or there on the shelves.
> To my disappointment and happiness there are very good products out there
> which are not squid with much better fines invested in them.
> I can clearly say that the Squid-Cache project is not the most "advanced"
> piece of software in the market and I know that it cannot compare to let say
> even 500 coding programmers work.
> I have seen couple products that are open source which tries to provide
> functionality which is similar to squid only in the protocol level and a
> simple proxy with great luck.
> Some of them are not as great as they might seems but I think that a young
> programmer with enough investment can learn the required subjects to
> implement a solution.
> However, here admins, users, programmers can ask questions as they please
> and I encourage to ask.
> I try to answer as much as I can and in many cases my knowledge might not
> be enough but I am trying to answer what I can with hope that it will help.
> And unlike MD Doctors SysAdmins do not need to swear on something like "do
> not harm" and I think it's a good aspect on things.
>
> I am still looking for clues about cloudflare since I have yet to see the
> person who hold the keys for them.
>
> Eliezer
>
> ----
> Eliezer Croitoru <http://ngtech.co.il/lmgtfy/>
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: [hidden email]

>
> From: Yuri Voinov [[hidden email]]
> Sent: Wednesday, July 6, 2016 11:15 PM
> To: Eliezer Croitoru; [hidden email]
> Subject: Re: [squid-users] host_verify_strict and wildcard SNI
>
>
> I know. Just asked. Since I am familiar with the standards.
>
> 07.07.2016 1:54, Eliezer Croitoru пишет:
> > Hey Yuri,
>
>
>
>       > These two subjects are not related directly to each other but
>       they might have something in common.
>
>       > Squid expects clients connections to meet the basic RFC6066
>       section 3:
>
>       > https://tools.ietf.org/html/rfc6066#section-3
> <https://tools.ietf.org/html/rfc6066>
>
>
>
>       > Which states that a host name should be there and the legal
>       characters of a hostname from both rfc1035 and rc6066 are very
>       speicifc.
>
>       > If a specific software are trying to request a wrong sni name
>       it's an issue in the client side request or software error
>       handling and enforcement.
>
>       > A http server would probably respond with a 4XX response code
>       or the default certificate.
>
>       > There are other options of course but the first thing to
>       check is if the client is a real browser or some special creature
>       that tries it's luck with a special form of ssl.
>
>       > To my understanding host_verify_strict tries to enforce basic
>       security levels while in a transparent proxy the rules will always
>       change.
>
>
>
>       > Eliezer
>
>
>
>       > ----
>
>       > Eliezer Croitoru
>
>       > Linux System Administrator
>
>       > Mobile: +972-5-28704261
>
>       > Email: [hidden email] [hidden email]
>
>
>
>
>
>       > -----Original Message-----
>
>       > From: squid-users
>       [[hidden email]] On Behalf Of
>       Yuri Voinov
>
>       > Sent: Wednesday, July 6, 2016 10:43 PM
>
>       > To: [hidden email]
> [hidden email]
>
>       > Subject: Re: [squid-users] host_verify_strict and wildcard
>       SNI
>
>
>
>
>
>       > Sounds familiar.
>
>
>
>       > Do you experience occasional problems with CloudFlare sites?
>
>
>
>
>
>       > 06.07.2016 20:36, Steve Hill пишет:
>
>
>
>       > > I'm using a transparent proxy and SSL-peek and have hit
>       a problem with
>
>       > an iOS app which seems to be doing broken things with the
>       SNI.
>
>
>
>       > > The app is making an HTTPS connection to a server and
>       presenting an
>
>       > SNI with a wildcard in it - i.e. "*.example.com".  I'm not
>       sure if this
>
>       > behaviour is actually illegal, but it certainly doesn't seem
>       to make a
>
>       > lot of sense to me.
>
>
>
>       > > Squid then internally generates a "CONNECT
>       *.example.com:443" request
>
>       > based on the peeked SNI, which is picked up by
>       hostHeaderIpVerify().
>
>       > Since *.example.com isn't a valid DNS name, Squid rejects the
>       connection
>
>       > on the basis that *.example.com doesn't match the IP address
>       that the
>
>       > client is connecting to.
>
>
>
>       > > Unfortunately, I can't see any way of working around the
>       problem -
>
>       > "host_verify_strict" is disabled, but according to the docs,
>
>       > > "For now suspicious intercepted CONNECT requests are
>       always responded
>
>       > to with an HTTP 409 (Conflict) error page."
>
>
>
>       > > As I understand it, turning host_verify_strict on causes
>       problems with
>
>       > CDNs which use DNS tricks for load balancing, so I'm not sure
>       I
>
>       > understand the rationale behind preventing it from being
>       turned off for
>
>       > CONNECT requests?
>
>
>
>
>
>
>
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfW65AAoJENNXIZxhPexGWaYIAM0SDMtDNaeqMhQAzPn2vIBL
enqBVF1yyg532T3zGg/QwznS6dz2qKiNuMTmVfRgX0Z7QWOe/IiLlDPHboe11MXe
Y2r5JOsPht3uq/iWBPewdFlEkzLxvWlLuG65Rd9TOTmuTvM5OKTnHIHUIhXzEQXW
NUITE/FlVKoUQb5mK4wOMoDCX1gXQ1FKm77F8HxsGdwlLqx4YbMqH4J1AVJu/FwZ
IRNbnXvqXQIEn+iePPwghPxsIDl7iDzQ2H70RDeATdClaPco9bEbvxv/6pdS2hI0
Al9bCx7vNbp0pEgUmzX+O9KOWQAu0s2qhxbJ1z9eZnXFciysPBsZJf1LJ4JPbrg=
=bLEa
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Eliezer Croitoru

If the splice doesn’t solve the issue what would you expect squid to do?

Spilce equals routing…

The other solution which ufdbguard implements is probing the destination hosts.

If you want a solution I can try to see if it is possible but I cannot guarantee that you or anyone will like it.

 

Eliezer

 

----

Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]

 

From: Yuri Voinov [mailto:[hidden email]]
Sent: Wednesday, July 6, 2016 11:49 PM
To: Eliezer Croitoru; [hidden email]
Subject: Re: [squid-users] host_verify_strict and wildcard SNI

 


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 
I am very seriously concerned about the issue CDN, because every day I discover more and more problematic sites, namely in connection with the CDN and HTTPS. For more than four Squid servers are experiencing these problems in my network. And I still do not see any reason why any solutions to these problems.

Moreover, the splice does not solve these problems.

Just skip the whole networks in the proxy bypass.

What is totally unacceptable. Traffic is money. And a lot of money.

07.07.2016 2:38, Eliezer Croitoru пишет:
> Hey Yuri,

      >

      > I am not the "standards" guy but I do know that if something

      can be encoded

      > it can be "decoded".

      > There are special cases which needs special "spice" which

      sometimes is not

      > present here or there on the shelves.

      > To my disappointment and happiness there are very good

      products out there

      > which are not squid with much better fines invested in them.

      > I can clearly say that the Squid-Cache project is not the

      most "advanced"

      > piece of software in the market and I know that it cannot

      compare to let say

      > even 500 coding programmers work.

      > I have seen couple products that are open source which tries

      to provide

      > functionality which is similar to squid only in the protocol

      level and a

      > simple proxy with great luck.

      > Some of them are not as great as they might seems but I think

      that a young

      > programmer with enough investment can learn the required

      subjects to

      > implement a solution.

      > However, here admins, users, programmers can ask questions as

      they please

      > and I encourage to ask.

      > I try to answer as much as I can and in many cases my

      knowledge might not

      > be enough but I am trying to answer what I can with hope that

      it will help.

      > And unlike MD Doctors SysAdmins do not need to swear on

      something like "do

      > not harm" and I think it's a good aspect on things.

      >

      > I am still looking for clues about cloudflare since I have

      yet to see the

      > person who hold the keys for them.

      >

      > Eliezer

      >

      > ----

      > Eliezer Croitoru <http://ngtech.co.il/lmgtfy/>

      > Linux System Administrator

      > Mobile: +972-5-28704261

      > Email: [hidden email]

      > 

      >

      > From: Yuri Voinov [[hidden email]]

      > Sent: Wednesday, July 6, 2016 11:15 PM

      > To: Eliezer Croitoru; [hidden email]

      > Subject: Re: [squid-users] host_verify_strict and wildcard

      SNI

      >

      >

      > I know. Just asked. Since I am familiar with the standards.

      >

      > 07.07.2016 1:54, Eliezer Croitoru пишет:

      > > Hey Yuri,

      >

      >

      >

      >       > These two subjects are not related directly to

      each other but

      >       they might have something in common.

      >

      >       > Squid expects clients connections to meet the

      basic RFC6066

      >       section 3:

      >

      >       > https://tools.ietf.org/html/rfc6066#section-3

      > <https://tools.ietf.org/html/rfc6066>

      >

      >

      >

      >       > Which states that a host name should be there and

      the legal

      >       characters of a hostname from both rfc1035 and rc6066

      are very

      >       speicifc.

      >

      >       > If a specific software are trying to request a

      wrong sni name

      >       it's an issue in the client side request or software

      error

      >       handling and enforcement.

      >

      >       > A http server would probably respond with a 4XX

      response code

      >       or the default certificate.

      >

      >       > There are other options of course but the first

      thing to

      >       check is if the client is a real browser or some

      special creature

      >       that tries it's luck with a special form of ssl.

      >

      >       > To my understanding host_verify_strict tries to

      enforce basic

      >       security levels while in a transparent proxy the rules

      will always

      >       change.

      >

      >

      >

      >       > Eliezer

      >

      >

      >

      >       > ----

      >

      >       > Eliezer Croitoru

      >

      >       > Linux System Administrator

      >

      >       > Mobile: +972-5-28704261

      >

      >       > Email: [hidden email]

      [hidden email]

      >

      >

      >

      >

      >

      >       > -----Original Message-----

      >

      >       > From: squid-users

      >       [[hidden email]] On

      Behalf Of

      >       Yuri Voinov

      >

      >       > Sent: Wednesday, July 6, 2016 10:43 PM

      >

      >       > To: [hidden email]

      > [hidden email]

      >

      >       > Subject: Re: [squid-users] host_verify_strict and

      wildcard

      >       SNI

      >

      >

      >

      >

      >

      >       > Sounds familiar.

      >

      >

      >

      >       > Do you experience occasional problems with

      CloudFlare sites?

      >

      >

      >

      >

      >

      >       > 06.07.2016 20:36, Steve Hill пишет:

      >

      >

      >

      >       > > I'm using a transparent proxy and SSL-peek

      and have hit

      >       a problem with

      >

      >       > an iOS app which seems to be doing broken things

      with the

      >       SNI.

      >

      >

      >

      >       > > The app is making an HTTPS connection to a

      server and

      >       presenting an

      >

      >       > SNI with a wildcard in it - i.e. "*.example.com". 

      I'm not

      >       sure if this

      >

      >       > behaviour is actually illegal, but it certainly

      doesn't seem

      >       to make a

      >

      >       > lot of sense to me.

      >

      >

      >

      >       > > Squid then internally generates a "CONNECT

      >       *.example.com:443" request

      >

      >       > based on the peeked SNI, which is picked up by

      >       hostHeaderIpVerify().

      >

      >       > Since *.example.com isn't a valid DNS name, Squid

      rejects the

      >       connection

      >

      >       > on the basis that *.example.com doesn't match the

      IP address

      >       that the

      >

      >       > client is connecting to.

      >

      >

      >

      >       > > Unfortunately, I can't see any way of working

      around the

      >       problem -

      >

      >       > "host_verify_strict" is disabled, but according to

      the docs,

      >

      >       > > "For now suspicious intercepted CONNECT

      requests are

      >       always responded

      >

      >       > to with an HTTP 409 (Conflict) error page."

      >

      >

      >

      >       > > As I understand it, turning

      host_verify_strict on causes

      >       problems with

      >

      >       > CDNs which use DNS tricks for load balancing, so

      I'm not sure

      >       I

      >

      >       > understand the rationale behind preventing it from

     being

      >       turned off for

      >

      >       > CONNECT requests?

      >

      >

      >

      >

      >

      >

      >

      >

      >
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfW65AAoJENNXIZxhPexGWaYIAM0SDMtDNaeqMhQAzPn2vIBL
enqBVF1yyg532T3zGg/QwznS6dz2qKiNuMTmVfRgX0Z7QWOe/IiLlDPHboe11MXe
Y2r5JOsPht3uq/iWBPewdFlEkzLxvWlLuG65Rd9TOTmuTvM5OKTnHIHUIhXzEQXW
NUITE/FlVKoUQb5mK4wOMoDCX1gXQ1FKm77F8HxsGdwlLqx4YbMqH4J1AVJu/FwZ
IRNbnXvqXQIEn+iePPwghPxsIDl7iDzQ2H70RDeATdClaPco9bEbvxv/6pdS2hI0
Al9bCx7vNbp0pEgUmzX+O9KOWQAu0s2qhxbJ1z9eZnXFciysPBsZJf1LJ4JPbrg=
=bLEa
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Marcus Kool
In reply to this post by Steve Hill


On 07/06/2016 11:36 AM, Steve Hill wrote:

>
> I'm using a transparent proxy and SSL-peek and have hit a problem with an iOS app which seems to be doing broken things with the SNI.
>
> The app is making an HTTPS connection to a server and presenting an SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if this behaviour is actually illegal, but it certainly doesn't seem
> to make a lot of sense to me.
>
> Squid then internally generates a "CONNECT *.example.com:443" request based on the peeked SNI, which is picked up by hostHeaderIpVerify(). Since *.example.com isn't a valid DNS name, Squid rejects the
> connection on the basis that *.example.com doesn't match the IP address that the client is connecting to.
>
> Unfortunately, I can't see any way of working around the problem - "host_verify_strict" is disabled, but according to the docs,
> "For now suspicious intercepted CONNECT requests are always responded to with an HTTP 409 (Conflict) error page."
>
> As I understand it, turning host_verify_strict on causes problems with CDNs which use DNS tricks for load balancing, so I'm not sure I understand the rationale behind preventing it from being turned
> off for CONNECT requests?

An SNI with a wildcard indeed does not make sense.

Since Squid tries to mimic the behavior of the server and of the client,
it deserves a patch where instead of doing a DNS lookup and then doing a
connect (based on the result of the DNS lookup?),
Squid simply connects to the IP address that the client tries to connect to
and does the TLS handshake with the SNI (that does not make sense).
This way it mimics the client a bit better.

Marcus
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Alex Rousskov
On 07/06/2016 05:01 PM, Marcus Kool wrote:
> On 07/06/2016 11:36 AM, Steve Hill wrote:
>> I'm using a transparent proxy and SSL-peek and have hit a problem with
>> an iOS app which seems to be doing broken things with the SNI.
>>
>> The app is making an HTTPS connection to a server and presenting an
>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>> this behaviour is actually illegal, but it certainly doesn't seem
>> to make a lot of sense to me.


> An SNI with a wildcard indeed does not make sense.

There are three rather different questions to consider here:

1. Is wildcard SNI "legal/valid"?
2. Can wildcard SNI "make sense" in some cases?
3. What should Squid do when receiving a wildcard SNI?


Q1. Is wildcard SNI "legal/valid"?

I do not know the answer to that question. The "*.example.com" name is
certainly legal in many DNS contexts. RFC 6066 requires HostName SNI to
be a "fully qualified domain name", but I failed to find a strict-enough
RFC definition of an FQDN that would either accept or reject wildcards
as FQDNs. I would not be surprised if FQDN syntax is not defined to the
level that would allow one to reject wildcards as FQDNs based on syntax
alone.


Q2. Can wildcard SNI "make sense" in some cases?

Yes, of course. The client essentially says "I am trying to connect to
_any_ example.com subdomain at this IP:port address. If you have any
service like that, please connect me". That would work fine in
deployment contexts where several servers with different names provide
essentially the same service and the central "routing point" would pick
the "best" service to use. I am not saying it is a good idea to use
wildcard SNIs, but I can see them "making sense" in some cases.


Q3. What should Squid do when receiving a wildcard SNI?

The first two questions are not really important and each may not even
have a single "correct" answer. I am sure protocol purists can argue
about them forever. The last question is important, which brings us to:

> Since Squid tries to mimic the behavior of the server and of the client,
> it deserves a patch where instead of doing a DNS lookup and then doing a
> connect (based on the result of the DNS lookup?),
> Squid simply connects to the IP address that the client tries to connect to
> and does the TLS handshake with the SNI (that does not make sense).
> This way it mimics the client a bit better.

I believe that is what Squid does already but please correct me if I am
wrong.

When forming a fake CONNECT request, Squid uses SNI information because
that is what ACLs and adaptation services usually want to see. However,
I hope that intercepting Squid always connects to the intended
destination of the intercepted connection instead of trusting its own
fake CONNECT request.

Whether Squid should generate a fake CONNECT with a wildcard host name
is an interesting question:

1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
and adaptation services (at least).

2. A fake CONNECT targeting an IP address instead of a wildcard name may
not give some ACL-driven rules and adaptation services enough
information to make the right decision.

3. A premature rejection of a connection with wildcard SNI does not
allow the admin to make the bump/splice/terminate decision.

#2 is probably the lesser of the three evils, but I wonder whether Squid
should also include the invalid host name as an X-SNI or similar HTTP
header in that CONNECT request, to give advanced ACLs and adaptation
services a better chance to work around known benign issues where the
admin knows the wildcard is not malicious (and to kill wildcard
transactions the admin knows to be malicious!).


A similar question can be asked about SNI names containing unusual
characters. At some point, it would be too dangerous to include SNI
information in the fake CONNECT request because it will interfere with
HTTP rules, but it is not clear where that point is exactly.


Cheers,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Marcus Kool


On 07/06/2016 10:07 PM, Alex Rousskov wrote:
> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>> I'm using a transparent proxy and SSL-peek and have hit a problem with
>>> an iOS app which seems to be doing broken things with the SNI.
>>>
>>> The app is making an HTTPS connection to a server and presenting an
>>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>>> this behaviour is actually illegal, but it certainly doesn't seem
>>> to make a lot of sense to me.

[snip]

>
> Q3. What should Squid do when receiving a wildcard SNI?
>
> The first two questions are not really important and each may not even
> have a single "correct" answer. I am sure protocol purists can argue
> about them forever. The last question is important, which brings us to:
>
>> Since Squid tries to mimic the behavior of the server and of the client,
>> it deserves a patch where instead of doing a DNS lookup and then doing a
>> connect (based on the result of the DNS lookup?),
>> Squid simply connects to the IP address that the client tries to connect to
>> and does the TLS handshake with the SNI (that does not make sense).
>> This way it mimics the client a bit better.
>
> I believe that is what Squid does already but please correct me if I am
> wrong.

Steve said that Squid rejects the connection because of a failed DNS lookup.
So what is Squid doing?  Is it doing  the following ?
- connect to the original IP
- use the presented SNI to the server
- do a DNS lookup and reject

> When forming a fake CONNECT request, Squid uses SNI information because
> that is what ACLs and adaptation services usually want to see. However,
> I hope that intercepting Squid always connects to the intended
> destination of the intercepted connection instead of trusting its own
> fake CONNECT request.

It is interesting to know if an ICAP server or URL rewriter is called
and with which parameters when the ios app breaks.

> Whether Squid should generate a fake CONNECT with a wildcard host name
> is an interesting question:
>
> 1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
> and adaptation services (at least).
 >

> 2. A fake CONNECT targeting an IP address instead of a wildcard name may
> not give some ACL-driven rules and adaptation services enough
> information to make the right decision.
>
> 3. A premature rejection of a connection with wildcard SNI does not
> allow the admin to make the bump/splice/terminate decision.
>
> #2 is probably the lesser of the three evils, but I wonder whether Squid
> should also include the invalid host name as an X-SNI or similar HTTP
> header in that CONNECT request, to give advanced ACLs and adaptation
> services a better chance to work around known benign issues where the
> admin knows the wildcard is not malicious (and to kill wildcard
> transactions the admin knows to be malicious!).

I use url_rewrite_extras with "... sni=\"%ssl::>sni\" ..."
so the URL redirector receives both the original IP address and
the peeked SNI value to make its decisions.
I agree that an ICAP service needs X-SNI or perhaps X-Squid-SNI to
also get both the IP address and the SNI value.

> A similar question can be asked about SNI names containing unusual
> characters. At some point, it would be too dangerous to include SNI
> information in the fake CONNECT request because it will interfere with
> HTTP rules, but it is not clear where that point is exactly.

To support the weirdest apps Squid might have to simply copy all
unusual characters to present the same parameter values to the server.
But we do not want to break things... so which characters are
so unusual that Squid does not want to accept them?

Marcus

> Cheers,
>
> Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Eliezer Croitoru
In reply to this post by Alex Rousskov
Couple thoughts Alex,

Currently the basic splice rules are being used with regex which means that they can work with wildcard.
And I can understand the argument of a client wanting some wildcard domain but I do not know about an application that actually
tries to uses such logic.

There are cases which the RFC do leave the open minds to get wild and I am not saying if these are right or wrong but,
they do states it's a hostname and not a certificate common name or some v3 component.

Practically some client can try to contact some arbitrary website and there are couple aspects to it.
If a user tries to connect to a site using a company proxy, then what companies will want to allow?
Would large companies allow such a connection to be spliced, or maybe they will want to inspect this connection deeper?
What about ISP's, these mostly care about cache and not about ACL's, while there are many who uses squid for content filtering.

From where anyone understands, a wildcard should never be required to be tested against any DNS server in the current state of the internet since these do have a strict policy to honor only valid hostname characters.
Maybe the future will bring the wildcard into the DNS world, should we consider such an option even if the RFC tends to minimize and containerize the options?

Thanks,
Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Alex Rousskov
Sent: Thursday, July 7, 2016 4:07 AM
To: [hidden email]
Subject: Re: [squid-users] host_verify_strict and wildcard SNI

On 07/06/2016 05:01 PM, Marcus Kool wrote:
> On 07/06/2016 11:36 AM, Steve Hill wrote:
>> I'm using a transparent proxy and SSL-peek and have hit a problem with
>> an iOS app which seems to be doing broken things with the SNI.
>>
>> The app is making an HTTPS connection to a server and presenting an
>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>> this behaviour is actually illegal, but it certainly doesn't seem
>> to make a lot of sense to me.


> An SNI with a wildcard indeed does not make sense.

There are three rather different questions to consider here:

1. Is wildcard SNI "legal/valid"?
2. Can wildcard SNI "make sense" in some cases?
3. What should Squid do when receiving a wildcard SNI?


Q1. Is wildcard SNI "legal/valid"?

I do not know the answer to that question. The "*.example.com" name is
certainly legal in many DNS contexts. RFC 6066 requires HostName SNI to
be a "fully qualified domain name", but I failed to find a strict-enough
RFC definition of an FQDN that would either accept or reject wildcards
as FQDNs. I would not be surprised if FQDN syntax is not defined to the
level that would allow one to reject wildcards as FQDNs based on syntax
alone.


Q2. Can wildcard SNI "make sense" in some cases?

Yes, of course. The client essentially says "I am trying to connect to
_any_ example.com subdomain at this IP:port address. If you have any
service like that, please connect me". That would work fine in
deployment contexts where several servers with different names provide
essentially the same service and the central "routing point" would pick
the "best" service to use. I am not saying it is a good idea to use
wildcard SNIs, but I can see them "making sense" in some cases.


Q3. What should Squid do when receiving a wildcard SNI?

The first two questions are not really important and each may not even
have a single "correct" answer. I am sure protocol purists can argue
about them forever. The last question is important, which brings us to:

> Since Squid tries to mimic the behavior of the server and of the client,
> it deserves a patch where instead of doing a DNS lookup and then doing a
> connect (based on the result of the DNS lookup?),
> Squid simply connects to the IP address that the client tries to connect to
> and does the TLS handshake with the SNI (that does not make sense).
> This way it mimics the client a bit better.

I believe that is what Squid does already but please correct me if I am
wrong.

When forming a fake CONNECT request, Squid uses SNI information because
that is what ACLs and adaptation services usually want to see. However,
I hope that intercepting Squid always connects to the intended
destination of the intercepted connection instead of trusting its own
fake CONNECT request.

Whether Squid should generate a fake CONNECT with a wildcard host name
is an interesting question:

1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
and adaptation services (at least).

2. A fake CONNECT targeting an IP address instead of a wildcard name may
not give some ACL-driven rules and adaptation services enough
information to make the right decision.

3. A premature rejection of a connection with wildcard SNI does not
allow the admin to make the bump/splice/terminate decision.

#2 is probably the lesser of the three evils, but I wonder whether Squid
should also include the invalid host name as an X-SNI or similar HTTP
header in that CONNECT request, to give advanced ACLs and adaptation
services a better chance to work around known benign issues where the
admin knows the wildcard is not malicious (and to kill wildcard
transactions the admin knows to be malicious!).


A similar question can be asked about SNI names containing unusual
characters. At some point, it would be too dangerous to include SNI
information in the fake CONNECT request because it will interfere with
HTTP rules, but it is not clear where that point is exactly.


Cheers,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Amos Jeffries
Administrator
In reply to this post by Marcus Kool
On 7/07/2016 1:55 p.m., Marcus Kool wrote:

>
>
> On 07/06/2016 10:07 PM, Alex Rousskov wrote:
>> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>>> I'm using a transparent proxy and SSL-peek and have hit a problem with
>>>> an iOS app which seems to be doing broken things with the SNI.
>>>>
>>>> The app is making an HTTPS connection to a server and presenting an
>>>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>>>> this behaviour is actually illegal, but it certainly doesn't seem
>>>> to make a lot of sense to me.
>
> [snip]
>
>>
>> Q3. What should Squid do when receiving a wildcard SNI?
>>
>> The first two questions are not really important and each may not even
>> have a single "correct" answer. I am sure protocol purists can argue
>> about them forever. The last question is important, which brings us to:
>>
>>> Since Squid tries to mimic the behavior of the server and of the client,
>>> it deserves a patch where instead of doing a DNS lookup and then doing a
>>> connect (based on the result of the DNS lookup?),
>>> Squid simply connects to the IP address that the client tries to
>>> connect to
>>> and does the TLS handshake with the SNI (that does not make sense).
>>> This way it mimics the client a bit better.
>>
>> I believe that is what Squid does already but please correct me if I am
>> wrong.
>
> Steve said that Squid rejects the connection because of a failed DNS
> lookup.
> So what is Squid doing?  Is it doing  the following ?
> - connect to the original IP
> - use the presented SNI to the server
> - do a DNS lookup and reject

No it is doing Host: header verification on the faked CONNECT request
which uses the SNI in the Host: header value. This is not strictly
required for CONNECT messages, but does potentially prevent Squid using
other IPs than the original one the client was contacting.
If the SNI is a valid domain name (ie resolves in DNS). Then Squid
should continue on past the check.


>
>> When forming a fake CONNECT request, Squid uses SNI information because
>> that is what ACLs and adaptation services usually want to see. However,
>> I hope that intercepting Squid always connects to the intended
>> destination of the intercepted connection instead of trusting its own
>> fake CONNECT request.
>
> It is interesting to know if an ICAP server or URL rewriter is called
> and with which parameters when the ios app breaks.
>
>> Whether Squid should generate a fake CONNECT with a wildcard host name
>> is an interesting question:
>>
>> 1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
>> and adaptation services (at least).
>>
>> 2. A fake CONNECT targeting an IP address instead of a wildcard name may
>> not give some ACL-driven rules and adaptation services enough
>> information to make the right decision.
>>
>> 3. A premature rejection of a connection with wildcard SNI does not
>> allow the admin to make the bump/splice/terminate decision.
>>
>> #2 is probably the lesser of the three evils, but I wonder whether Squid
>> should also include the invalid host name as an X-SNI or similar HTTP
>> header in that CONNECT request, to give advanced ACLs and adaptation
>> services a better chance to work around known benign issues where the
>> admin knows the wildcard is not malicious (and to kill wildcard
>> transactions the admin knows to be malicious!).
>
> I use url_rewrite_extras with "... sni=\"%ssl::>sni\" ..."
> so the URL redirector receives both the original IP address and
> the peeked SNI value to make its decisions.
> I agree that an ICAP service needs X-SNI or perhaps X-Squid-SNI to
> also get both the IP address and the SNI value.

That is a problem for the service. Squid can already send anything in:
<http://www.squid-cache.org/Doc/config/adaptation_meta/>.

Maybe you have mistaken it for the inability of most ICAP services to
send things back to Squid in the same way.

>
>> A similar question can be asked about SNI names containing unusual
>> characters. At some point, it would be too dangerous to include SNI
>> information in the fake CONNECT request because it will interfere with
>> HTTP rules, but it is not clear where that point is exactly.
>
> To support the weirdest apps Squid might have to simply copy all
> unusual characters to present the same parameter values to the server.

It is being mapped into the HTTP equivalent value. Which are Host:
header and authority-URI. Only valid FQDN names can make it through the
mapping.

> But we do not want to break things... so which characters are
> so unusual that Squid does not want to accept them?

In Steve's case the asterisk at the start of the domain name. The name
labels must each start with an alphabet character a-z or A-Z.  That is a
fixed requirement. The other characters that follow are where things get
fuzzy depending on what RFCs you follow.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Marcus Kool


On 07/07/2016 07:15 AM, Amos Jeffries wrote:

> On 7/07/2016 1:55 p.m., Marcus Kool wrote:
>>
>>
>> On 07/06/2016 10:07 PM, Alex Rousskov wrote:
>>> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>>>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>>>> I'm using a transparent proxy and SSL-peek and have hit a problem with
>>>>> an iOS app which seems to be doing broken things with the SNI.
>>>>>
>>>>> The app is making an HTTPS connection to a server and presenting an
>>>>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>>>>> this behaviour is actually illegal, but it certainly doesn't seem
>>>>> to make a lot of sense to me.
>>
>> [snip]
>>
>>>
>>> Q3. What should Squid do when receiving a wildcard SNI?
>>>
>>> The first two questions are not really important and each may not even
>>> have a single "correct" answer. I am sure protocol purists can argue
>>> about them forever. The last question is important, which brings us to:
>>>
>>>> Since Squid tries to mimic the behavior of the server and of the client,
>>>> it deserves a patch where instead of doing a DNS lookup and then doing a
>>>> connect (based on the result of the DNS lookup?),
>>>> Squid simply connects to the IP address that the client tries to
>>>> connect to
>>>> and does the TLS handshake with the SNI (that does not make sense).
>>>> This way it mimics the client a bit better.
>>>
>>> I believe that is what Squid does already but please correct me if I am
>>> wrong.
>>
>> Steve said that Squid rejects the connection because of a failed DNS
>> lookup.
>> So what is Squid doing?  Is it doing  the following ?
>> - connect to the original IP
>> - use the presented SNI to the server
>> - do a DNS lookup and reject
>
> No it is doing Host: header verification on the faked CONNECT request
> which uses the SNI in the Host: header value. This is not strictly
> required for CONNECT messages, but does potentially prevent Squid using
> other IPs than the original one the client was contacting.

Squid _has_ the original IP so why would Squid potentially connect to an
other IP ?

> If the SNI is a valid domain name (ie resolves in DNS). Then Squid
> should continue on past the check.

Does Squid do a DNS lookup or only check the value for "valid" characters?

>>> When forming a fake CONNECT request, Squid uses SNI information because
>>> that is what ACLs and adaptation services usually want to see. However,
>>> I hope that intercepting Squid always connects to the intended
>>> destination of the intercepted connection instead of trusting its own
>>> fake CONNECT request.
>>
>> It is interesting to know if an ICAP server or URL rewriter is called
>> and with which parameters when the ios app breaks.
>>
>>> Whether Squid should generate a fake CONNECT with a wildcard host name
>>> is an interesting question:
>>>
>>> 1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
>>> and adaptation services (at least).
>>>
>>> 2. A fake CONNECT targeting an IP address instead of a wildcard name may
>>> not give some ACL-driven rules and adaptation services enough
>>> information to make the right decision.
>>>
>>> 3. A premature rejection of a connection with wildcard SNI does not
>>> allow the admin to make the bump/splice/terminate decision.
>>>
>>> #2 is probably the lesser of the three evils, but I wonder whether Squid
>>> should also include the invalid host name as an X-SNI or similar HTTP
>>> header in that CONNECT request, to give advanced ACLs and adaptation
>>> services a better chance to work around known benign issues where the
>>> admin knows the wildcard is not malicious (and to kill wildcard
>>> transactions the admin knows to be malicious!).
>>
>> I use url_rewrite_extras with "... sni=\"%ssl::>sni\" ..."
>> so the URL redirector receives both the original IP address and
>> the peeked SNI value to make its decisions.
>> I agree that an ICAP service needs X-SNI or perhaps X-Squid-SNI to
>> also get both the IP address and the SNI value.
>
> That is a problem for the service. Squid can already send anything in:
> <http://www.squid-cache.org/Doc/config/adaptation_meta/>.

Which problem are you referring to?

> Maybe you have mistaken it for the inability of most ICAP services to
> send things back to Squid in the same way.

The ICAP server needs both the IP and the SNI which Squid can be configured
to do.  What do you suggest that an ICAP server needs to send back to Squid?

>>> A similar question can be asked about SNI names containing unusual
>>> characters. At some point, it would be too dangerous to include SNI
>>> information in the fake CONNECT request because it will interfere with
>>> HTTP rules, but it is not clear where that point is exactly.
>>
>> To support the weirdest apps Squid might have to simply copy all
>> unusual characters to present the same parameter values to the server.
>
> It is being mapped into the HTTP equivalent value. Which are Host:
> header and authority-URI. Only valid FQDN names can make it through the
> mapping.

Here things get complicated.
It is correct that Squid enforces apps to follow standards or
should Squid try to proxy connections for apps when it can?

Marcus

>> But we do not want to break things... so which characters are
>> so unusual that Squid does not want to accept them?
>
> In Steve's case the asterisk at the start of the domain name. The name
> labels must each start with an alphabet character a-z or A-Z.  That is a
> fixed requirement. The other characters that follow are where things get
> fuzzy depending on what RFCs you follow.
>
> Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Amos Jeffries
Administrator
On 7/07/2016 11:30 p.m., Marcus Kool wrote:

>
>
> On 07/07/2016 07:15 AM, Amos Jeffries wrote:
>> On 7/07/2016 1:55 p.m., Marcus Kool wrote:
>>>
>>>
>>> On 07/06/2016 10:07 PM, Alex Rousskov wrote:
>>>> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>>>>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>>>>> I'm using a transparent proxy and SSL-peek and have hit a problem
>>>>>> with
>>>>>> an iOS app which seems to be doing broken things with the SNI.
>>>>>>
>>>>>> The app is making an HTTPS connection to a server and presenting an
>>>>>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>>>>>> this behaviour is actually illegal, but it certainly doesn't seem
>>>>>> to make a lot of sense to me.
>>>
>>> [snip]
>>>
>>>>
>>>> Q3. What should Squid do when receiving a wildcard SNI?
>>>>
>>>> The first two questions are not really important and each may not even
>>>> have a single "correct" answer. I am sure protocol purists can argue
>>>> about them forever. The last question is important, which brings us to:
>>>>
>>>>> Since Squid tries to mimic the behavior of the server and of the
>>>>> client,
>>>>> it deserves a patch where instead of doing a DNS lookup and then
>>>>> doing a
>>>>> connect (based on the result of the DNS lookup?),
>>>>> Squid simply connects to the IP address that the client tries to
>>>>> connect to
>>>>> and does the TLS handshake with the SNI (that does not make sense).
>>>>> This way it mimics the client a bit better.
>>>>
>>>> I believe that is what Squid does already but please correct me if I am
>>>> wrong.
>>>
>>> Steve said that Squid rejects the connection because of a failed DNS
>>> lookup.
>>> So what is Squid doing?  Is it doing  the following ?
>>> - connect to the original IP
>>> - use the presented SNI to the server
>>> - do a DNS lookup and reject
>>
>> No it is doing Host: header verification on the faked CONNECT request
>> which uses the SNI in the Host: header value. This is not strictly
>> required for CONNECT messages, but does potentially prevent Squid using
>> other IPs than the original one the client was contacting.
>
> Squid _has_ the original IP so why would Squid potentially connect to an
> other IP ?

Because the inbound and outbound connections are not related. The
outbound is normally done to any of the IPs that the request message
domain/Host header resolve to. It takes special circumstances (such as
failing the Host verification) to tie them together.

>
>> If the SNI is a valid domain name (ie resolves in DNS). Then Squid
>> should continue on past the check.
>
> Does Squid do a DNS lookup or only check the value for "valid" characters?

DNS lookup.

>
>>>> When forming a fake CONNECT request, Squid uses SNI information because
>>>> that is what ACLs and adaptation services usually want to see. However,
>>>> I hope that intercepting Squid always connects to the intended
>>>> destination of the intercepted connection instead of trusting its own
>>>> fake CONNECT request.
>>>
>>> It is interesting to know if an ICAP server or URL rewriter is called
>>> and with which parameters when the ios app breaks.
>>>
>>>> Whether Squid should generate a fake CONNECT with a wildcard host name
>>>> is an interesting question:
>>>>
>>>> 1. A fake CONNECT targeting an wildcard name may break ACL-driven rules
>>>> and adaptation services (at least).
>>>>
>>>> 2. A fake CONNECT targeting an IP address instead of a wildcard name
>>>> may
>>>> not give some ACL-driven rules and adaptation services enough
>>>> information to make the right decision.
>>>>
>>>> 3. A premature rejection of a connection with wildcard SNI does not
>>>> allow the admin to make the bump/splice/terminate decision.
>>>>
>>>> #2 is probably the lesser of the three evils, but I wonder whether
>>>> Squid
>>>> should also include the invalid host name as an X-SNI or similar HTTP
>>>> header in that CONNECT request, to give advanced ACLs and adaptation
>>>> services a better chance to work around known benign issues where the
>>>> admin knows the wildcard is not malicious (and to kill wildcard
>>>> transactions the admin knows to be malicious!).
>>>
>>> I use url_rewrite_extras with "... sni=\"%ssl::>sni\" ..."
>>> so the URL redirector receives both the original IP address and
>>> the peeked SNI value to make its decisions.
>>> I agree that an ICAP service needs X-SNI or perhaps X-Squid-SNI to
>>> also get both the IP address and the SNI value.
>>
>> That is a problem for the service. Squid can already send anything in:
>> <http://www.squid-cache.org/Doc/config/adaptation_meta/>.
>
> Which problem are you referring to?

I mean it as: IF it is a problem, then its a problem for the service
side of things since Squid can send the data fine.

>
>> Maybe you have mistaken it for the inability of most ICAP services to
>> send things back to Squid in the same way.
>
> The ICAP server needs both the IP and the SNI which Squid can be configured
> to do.  What do you suggest that an ICAP server needs to send back to
> Squid?

Sorry I misread your earlier post as implying Squid needs to be extended
to send those details.

>
>>>> A similar question can be asked about SNI names containing unusual
>>>> characters. At some point, it would be too dangerous to include SNI
>>>> information in the fake CONNECT request because it will interfere with
>>>> HTTP rules, but it is not clear where that point is exactly.
>>>
>>> To support the weirdest apps Squid might have to simply copy all
>>> unusual characters to present the same parameter values to the server.
>>
>> It is being mapped into the HTTP equivalent value. Which are Host:
>> header and authority-URI. Only valid FQDN names can make it through the
>> mapping.
>
> Here things get complicated.
> It is correct that Squid enforces apps to follow standards or
> should Squid try to proxy connections for apps when it can?

Squid isn't enforcing standards here. As Steve original messge says it:
"generates a "CONNECT *.example.com:443" request based on the peeked SNI"
 - which is arguably invalid HTTP syntax, but oh well.

It then is unable to do a DNS lookup for *.example.com to find out what
its IPs are and does the error handling action for a failure to verify
on a CONNECT message.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Marcus Kool


On 07/07/2016 09:23 AM, Amos Jeffries wrote:

> On 7/07/2016 11:30 p.m., Marcus Kool wrote:
>>
>>
>> On 07/07/2016 07:15 AM, Amos Jeffries wrote:
>>> On 7/07/2016 1:55 p.m., Marcus Kool wrote:
>>>>
>>>>
>>>> On 07/06/2016 10:07 PM, Alex Rousskov wrote:
>>>>> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>>>>>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>>>>>> I'm using a transparent proxy and SSL-peek and have hit a problem
>>>>>>> with
>>>>>>> an iOS app which seems to be doing broken things with the SNI.
>>>>>>>
>>>>>>> The app is making an HTTPS connection to a server and presenting an
>>>>>>> SNI with a wildcard in it - i.e. "*.example.com".  I'm not sure if
>>>>>>> this behaviour is actually illegal, but it certainly doesn't seem
>>>>>>> to make a lot of sense to me.
>>>>
>>>> [snip]
>>>>
>>>>>
>>>>> Q3. What should Squid do when receiving a wildcard SNI?
>>>>>
>>>>> The first two questions are not really important and each may not even
>>>>> have a single "correct" answer. I am sure protocol purists can argue
>>>>> about them forever. The last question is important, which brings us to:
>>>>>
>>>>>> Since Squid tries to mimic the behavior of the server and of the
>>>>>> client,
>>>>>> it deserves a patch where instead of doing a DNS lookup and then
>>>>>> doing a
>>>>>> connect (based on the result of the DNS lookup?),
>>>>>> Squid simply connects to the IP address that the client tries to
>>>>>> connect to
>>>>>> and does the TLS handshake with the SNI (that does not make sense).
>>>>>> This way it mimics the client a bit better.
>>>>>
>>>>> I believe that is what Squid does already but please correct me if I am
>>>>> wrong.
>>>>
>>>> Steve said that Squid rejects the connection because of a failed DNS
>>>> lookup.
>>>> So what is Squid doing?  Is it doing  the following ?
>>>> - connect to the original IP
>>>> - use the presented SNI to the server
>>>> - do a DNS lookup and reject
>>>
>>> No it is doing Host: header verification on the faked CONNECT request
>>> which uses the SNI in the Host: header value. This is not strictly
>>> required for CONNECT messages, but does potentially prevent Squid using
>>> other IPs than the original one the client was contacting.
>>
>> Squid _has_ the original IP so why would Squid potentially connect to an
>> other IP ?
>
> Because the inbound and outbound connections are not related. The
> outbound is normally done to any of the IPs that the request message
> domain/Host header resolve to. It takes special circumstances (such as
> failing the Host verification) to tie them together.

Oops.  An application wants to connect to A.B.C.D has an SNI for foo.example.com
which resolves to A.B.C.D and A.B.C.E and Squid may connect the stream
to A.B.C.E... It is easy to imagine that some applications break with this behavior.

>>> If the SNI is a valid domain name (ie resolves in DNS). Then Squid
>>> should continue on past the check.
>>
>> Does Squid do a DNS lookup or only check the value for "valid" characters?
>
> DNS lookup.

[snip]

>>>>> A similar question can be asked about SNI names containing unusual
>>>>> characters. At some point, it would be too dangerous to include SNI
>>>>> information in the fake CONNECT request because it will interfere with
>>>>> HTTP rules, but it is not clear where that point is exactly.
>>>>
>>>> To support the weirdest apps Squid might have to simply copy all
>>>> unusual characters to present the same parameter values to the server.
>>>
>>> It is being mapped into the HTTP equivalent value. Which are Host:
>>> header and authority-URI. Only valid FQDN names can make it through the
>>> mapping.
>>
>> Here things get complicated.
>> It is correct that Squid enforces apps to follow standards or
>> should Squid try to proxy connections for apps when it can?
>
> Squid isn't enforcing standards here. As Steve original messge says it:
> "generates a "CONNECT *.example.com:443" request based on the peeked SNI"
>   - which is arguably invalid HTTP syntax, but oh well.
>
> It then is unable to do a DNS lookup for *.example.com to find out what
> its IPs are and does the error handling action for a failure to verify
> on a CONNECT message.

yes, the fake CONNECT is dealt with like a regular CONNECT including
DNS lookup.  I fear for other apps (besides the one ios app that Steve
refers to) to break because Squid may connect to a different IP than
the client/app is requesting.
If Squid uses the original IP to connect without doing a DNS lookup,
Steve's app will work and potential issues with other apps are
prevented.

Marcus

> Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Yuri Voinov


07.07.2016 19:08, Marcus Kool пишет:

>
>
> On 07/07/2016 09:23 AM, Amos Jeffries wrote:
>> On 7/07/2016 11:30 p.m., Marcus Kool wrote:
>>>
>>>
>>> On 07/07/2016 07:15 AM, Amos Jeffries wrote:
>>>> On 7/07/2016 1:55 p.m., Marcus Kool wrote:
>>>>>
>>>>>
>>>>> On 07/06/2016 10:07 PM, Alex Rousskov wrote:
>>>>>> On 07/06/2016 05:01 PM, Marcus Kool wrote:
>>>>>>> On 07/06/2016 11:36 AM, Steve Hill wrote:
>>>>>>>> I'm using a transparent proxy and SSL-peek and have hit a problem
>>>>>>>> with
>>>>>>>> an iOS app which seems to be doing broken things with the SNI.
>>>>>>>>
>>>>>>>> The app is making an HTTPS connection to a server and
>>>>>>>> presenting an
>>>>>>>> SNI with a wildcard in it - i.e. "*.example.com". I'm not sure if
>>>>>>>> this behaviour is actually illegal, but it certainly doesn't seem
>>>>>>>> to make a lot of sense to me.
>>>>>
>>>>> [snip]
>>>>>
>>>>>>
>>>>>> Q3. What should Squid do when receiving a wildcard SNI?
>>>>>>
>>>>>> The first two questions are not really important and each may not
>>>>>> even
>>>>>> have a single "correct" answer. I am sure protocol purists can argue
>>>>>> about them forever. The last question is important, which brings
>>>>>> us to:
>>>>>>
>>>>>>> Since Squid tries to mimic the behavior of the server and of the
>>>>>>> client,
>>>>>>> it deserves a patch where instead of doing a DNS lookup and then
>>>>>>> doing a
>>>>>>> connect (based on the result of the DNS lookup?),
>>>>>>> Squid simply connects to the IP address that the client tries to
>>>>>>> connect to
>>>>>>> and does the TLS handshake with the SNI (that does not make sense).
>>>>>>> This way it mimics the client a bit better.
>>>>>>
>>>>>> I believe that is what Squid does already but please correct me
>>>>>> if I am
>>>>>> wrong.
>>>>>
>>>>> Steve said that Squid rejects the connection because of a failed DNS
>>>>> lookup.
>>>>> So what is Squid doing?  Is it doing  the following ?
>>>>> - connect to the original IP
>>>>> - use the presented SNI to the server
>>>>> - do a DNS lookup and reject
>>>>
>>>> No it is doing Host: header verification on the faked CONNECT request
>>>> which uses the SNI in the Host: header value. This is not strictly
>>>> required for CONNECT messages, but does potentially prevent Squid
>>>> using
>>>> other IPs than the original one the client was contacting.
>>>
>>> Squid _has_ the original IP so why would Squid potentially connect
>>> to an
>>> other IP ?
>>
>> Because the inbound and outbound connections are not related. The
>> outbound is normally done to any of the IPs that the request message
>> domain/Host header resolve to. It takes special circumstances (such as
>> failing the Host verification) to tie them together.
>
> Oops.  An application wants to connect to A.B.C.D has an SNI for
> foo.example.com
> which resolves to A.B.C.D and A.B.C.E and Squid may connect the stream
> to A.B.C.E... It is easy to imagine that some applications break with
> this behavior.
>
>>>> If the SNI is a valid domain name (ie resolves in DNS). Then Squid
>>>> should continue on past the check.
>>>
>>> Does Squid do a DNS lookup or only check the value for "valid"
>>> characters?
>>
>> DNS lookup.
>
> [snip]
>
>>>>>> A similar question can be asked about SNI names containing unusual
>>>>>> characters. At some point, it would be too dangerous to include SNI
>>>>>> information in the fake CONNECT request because it will interfere
>>>>>> with
>>>>>> HTTP rules, but it is not clear where that point is exactly.
>>>>>
>>>>> To support the weirdest apps Squid might have to simply copy all
>>>>> unusual characters to present the same parameter values to the
>>>>> server.
>>>>
>>>> It is being mapped into the HTTP equivalent value. Which are Host:
>>>> header and authority-URI. Only valid FQDN names can make it through
>>>> the
>>>> mapping.
>>>
>>> Here things get complicated.
>>> It is correct that Squid enforces apps to follow standards or
>>> should Squid try to proxy connections for apps when it can?
>>
>> Squid isn't enforcing standards here. As Steve original messge says it:
>> "generates a "CONNECT *.example.com:443" request based on the peeked
>> SNI"
>>   - which is arguably invalid HTTP syntax, but oh well.
>>
>> It then is unable to do a DNS lookup for *.example.com to find out what
>> its IPs are and does the error handling action for a failure to verify
>> on a CONNECT message.
>
> yes, the fake CONNECT is dealt with like a regular CONNECT including
> DNS lookup.  I fear for other apps (besides the one ios app that Steve
> refers to) to break because Squid may connect to a different IP than
> the client/app is requesting.
> If Squid uses the original IP to connect without doing a DNS lookup,
> Steve's app will work and potential issues with other apps are
> prevented.
Interestingly, Marcus. Does this mean that the CDN may be at different
points in time different IP connection and it makes it impossible for
client connections through Squid?
>
> Marcus
>
>> Amos
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Marcus Kool


On 07/07/2016 10:49 AM, Yuri wrote:

>>>>>>> A similar question can be asked about SNI names containing unusual
>>>>>>> characters. At some point, it would be too dangerous to include SNI
>>>>>>> information in the fake CONNECT request because it will interfere with
>>>>>>> HTTP rules, but it is not clear where that point is exactly.
>>>>>>
>>>>>> To support the weirdest apps Squid might have to simply copy all
>>>>>> unusual characters to present the same parameter values to the server.
>>>>>
>>>>> It is being mapped into the HTTP equivalent value. Which are Host:
>>>>> header and authority-URI. Only valid FQDN names can make it through the
>>>>> mapping.
>>>>
>>>> Here things get complicated.
>>>> It is correct that Squid enforces apps to follow standards or
>>>> should Squid try to proxy connections for apps when it can?
>>>
>>> Squid isn't enforcing standards here. As Steve original messge says it:
>>> "generates a "CONNECT *.example.com:443" request based on the peeked SNI"
>>>   - which is arguably invalid HTTP syntax, but oh well.
>>>
>>> It then is unable to do a DNS lookup for *.example.com to find out what
>>> its IPs are and does the error handling action for a failure to verify
>>> on a CONNECT message.
>>
>> yes, the fake CONNECT is dealt with like a regular CONNECT including
>> DNS lookup.  I fear for other apps (besides the one ios app that Steve
>> refers to) to break because Squid may connect to a different IP than
>> the client/app is requesting.
>> If Squid uses the original IP to connect without doing a DNS lookup,
>> Steve's app will work and potential issues with other apps are
>> prevented.

> Interestingly, Marcus. Does this mean that the CDN may be at different points in time different IP connection and it makes it impossible for client connections through Squid?

It all depends on the app/client: if it uses a servername/SNI that
resolves to multiple IP addresses but needs to connect to the one
that it specifically wants to CONNECT to, the app can fail since
Squid might choose an other IP address to connect to.

Or, apps might become slow since it might be faster when it reconnects
to the same server that it connected to before.
I think it is best to prevent issues and that Squid should connect
to the IP that the client is trying to connect to.

Marcus
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Yuri Voinov

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
 


07.07.2016 19:59, Marcus Kool пишет:
>
>
> On 07/07/2016 10:49 AM, Yuri wrote:
>
>>>>>>>> A similar question can be asked about SNI names containing unusual
>>>>>>>> characters. At some point, it would be too dangerous to include SNI
>>>>>>>> information in the fake CONNECT request because it will interfere with
>>>>>>>> HTTP rules, but it is not clear where that point is exactly.
>>>>>>>
>>>>>>> To support the weirdest apps Squid might have to simply copy all
>>>>>>> unusual characters to present the same parameter values to the server.
>>>>>>
>>>>>> It is being mapped into the HTTP equivalent value. Which are Host:
>>>>>> header and authority-URI. Only valid FQDN names can make it through the
>>>>>> mapping.
>>>>>
>>>>> Here things get complicated.
>>>>> It is correct that Squid enforces apps to follow standards or
>>>>> should Squid try to proxy connections for apps when it can?
>>>>
>>>> Squid isn't enforcing standards here. As Steve original messge says it:
>>>> "generates a "CONNECT *.example.com:443" request based on the peeked SNI"
>>>>   - which is arguably invalid HTTP syntax, but oh well.
>>>>
>>>> It then is unable to do a DNS lookup for *.example.com to find out what
>>>> its IPs are and does the error handling action for a failure to verify
>>>> on a CONNECT message.
>>>
>>> yes, the fake CONNECT is dealt with like a regular CONNECT including
>>> DNS lookup.  I fear for other apps (besides the one ios app that Steve
>>> refers to) to break because Squid may connect to a different IP than
>>> the client/app is requesting.
>>> If Squid uses the original IP to connect without doing a DNS lookup,
>>> Steve's app will work and potential issues with other apps are
>>> prevented.
>
>> Interestingly, Marcus. Does this mean that the CDN may be at different points in time different IP connection and it makes it impossible for client connections through Squid?
>
> It all depends on the app/client: if it uses a servername/SNI that
> resolves to multiple IP addresses but needs to connect to the one
> that it specifically wants to CONNECT to, the app can fail since
> Squid might choose an other IP address to connect to.
>
> Or, apps might become slow since it might be faster when it reconnects
> to the same server that it connected to before.
> I think it is best to prevent issues and that Squid should connect
> to the IP that the client is trying to connect to.

I suggests, devs will say this is not secure. Client can be compromised etc.etc.etc. :)
>
> Marcus
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQEcBAEBCAAGBQJXfmvzAAoJENNXIZxhPexGQwMIALkYjQH8ke4R44oINkzQfqGR
j5VtmMRfSlcYn82Xe7D4UzkjcGytYDiJJg+0VTsVgPxphgAcKXDP/Tx3lxTpP09e
8w3pmTU5TmgYUNvuZqheSn+Zhsp4lLUN0rj2VwIZZPueMWA6Ypre7YC7vRscEluj
h9p3ZA6LTmj7NpSehWcxPKDxQdJ5HEIMRjzOyXWMJRvjwYU9s55xKYfHy5ZjSGV4
bF87d8Tg746sh+jcje6BpJBKOVNp8ImyxfjI6eFSVAjBsUpeZPa3yb2uq1LunZi1
t50q1C0P93FcqC8SipPcIM/azDEu08VrByG01x12zjgRqMVuIeMkMcvJOT3WVKY=
=0ect
-----END PGP SIGNATURE-----


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

0x613DEC46.asc (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Alex Rousskov
In reply to this post by Eliezer Croitoru
On 07/07/2016 01:37 AM, Eliezer Croitoru wrote:

> Maybe the future will bring the wildcard into the DNS world

FYI: Wildcards have been in DNS world since before RFC 1035 dated 1987:

>    - The results of standard queries where the QNAME contains "*"
>      labels if the data might be used to construct wildcards.

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: host_verify_strict and wildcard SNI

Steve Hill
In reply to this post by Eliezer Croitoru
On 06/07/16 20:54, Eliezer Croitoru wrote:

> There are other options of course but the first thing to check is if the client is a real browser or some special creature that tries it's luck with a special form of ssl.

In this case it isn't a real web browser - it's an iOS app, and the
vendor has stated that they have no intention of fixing it :(

--
  - Steve Hill
    Technical Director
    Opendium Limited     http://www.opendium.com

Direct contacts:
    Instant messager: xmpp:[hidden email]
    Email:            [hidden email]
    Phone:            sip:[hidden email]

Sales / enquiries contacts:
    Email:            [hidden email]
    Phone:            +44-1792-824568 / sip:[hidden email]

Support contacts:
    Email:            [hidden email]
    Phone:            +44-1792-825748 / sip:[hidden email]
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
12