RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

James Harper
> >
> > I'm having some problems with reverse proxy and NTLM authentication.
> > Specifically, the connection to the client is not persisted which I
> > believe invalidates the NTLM authentication protocol. I've added a
> > source port number to the logs which shows that it is indeed creating
> > a new connection for each request. There seems to have been a bit of
> > mailing list activity about similar problems but nothing exactly the
> > same and none of the suggested solutions work.
>
> I've done a bit more testing on this, and it seems that the server returns
> "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> Unauthorized" before passing it onto the client. Does that help?
>

It seems that this is the cause of the problem... The patch following this email fixes it... is there any reason why the version should be forced to 1.0?? Is it to work around some other bug?

James

--- squid3-3.1.20.orig/src/client_side_reply.cc
+++ squid3-3.1.20/src/client_side_reply.cc
@@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()

     reply = HTTPMSGLOCK(rep);

+#if 0
     if (reply->sline.protocol == PROTO_HTTP) {
         /* enforce 1.0 reply version (but only on real HTTP traffic) */
         reply->sline.version = HttpVersion(1,0);
     }
+#endif

     /* do header conversions */
     buildReplyHeader();
Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

Amos Jeffries
Administrator
On 12/06/2012 6:08 p.m., James Harper wrote:

>>> I'm having some problems with reverse proxy and NTLM authentication.
>>> Specifically, the connection to the client is not persisted which I
>>> believe invalidates the NTLM authentication protocol. I've added a
>>> source port number to the logs which shows that it is indeed creating
>>> a new connection for each request. There seems to have been a bit of
>>> mailing list activity about similar problems but nothing exactly the
>>> same and none of the suggested solutions work.
>> I've done a bit more testing on this, and it seems that the server returns
>> "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
>> Unauthorized" before passing it onto the client. Does that help?
>>
> It seems that this is the cause of the problem... The patch following this email fixes it... is there any reason why the version should be forced to 1.0?? Is it to work around some other bug?

Because Squid 3.1 is not HTTP/1.1 compliant on the client-facing
channels. Offering it will trick the clients into believeing they can
use features which will break their connectivity.

The problem is somewhere in the code which determines
"Connection:keep-alive" and "Connection:close".  Squid should be adding
"Connection:keep-alive" unless something causes "Connection:close" to be
necessary.

>
> James
>
> --- squid3-3.1.20.orig/src/client_side_reply.cc
> +++ squid3-3.1.20/src/client_side_reply.cc
> @@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()
>
>       reply = HTTPMSGLOCK(rep);
>
> +#if 0
>       if (reply->sline.protocol == PROTO_HTTP) {
>           /* enforce 1.0 reply version (but only on real HTTP traffic) */
>           reply->sline.version = HttpVersion(1,0);
>       }
> +#endif
>
>       /* do header conversions */
>       buildReplyHeader();

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

James Harper
> >> I've done a bit more testing on this, and it seems that the server
> >> returns
> >> "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> >> Unauthorized" before passing it onto the client. Does that help?
> >>
> > It seems that this is the cause of the problem... The patch following this
> email fixes it... is there any reason why the version should be forced to 1.0??
> Is it to work around some other bug?
>
> Because Squid 3.1 is not HTTP/1.1 compliant on the client-facing channels.
> Offering it will trick the clients into believeing they can use features which will
> break their connectivity.
>
> The problem is somewhere in the code which determines "Connection:keep-
> alive" and "Connection:close".  Squid should be adding "Connection:keep-
> alive" unless something causes "Connection:close" to be necessary.
>

Actually it turns out that sending HTTP\1.0 back to the client (Windows Terminal Server Gateway client) causes it to drop the connections itself. I thought it was Squid dropping the connections originally but that turned out not to be the case. It was only when I worked around the HTTPS encryption that I could actually monitor the contents of the packets and see what was going on.

So as far as I can see there is no other fix apart from my patch to not modifying the HTTP version in the response sent back to the client. I'm quite happily connected through my Terminal Server Gateway now without any problems at all, but from what you have said it would seem that I'm just lucky that that RPC protocol for TSG doesn't use any unsupported (by squid) HTTP/1.1 features, and that my patch is likely to introduce other problems in other protocols...
 
Bring on squid 3.2.0 and full HTTP/1.1 support, I guess :)

Thanks

James

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

cl00m
In reply to this post by James Harper
Hi James,

Your patch can interests me, but I'm a little bit confused on how apply it ?
I've edited the client_side_reply.cc of the src before compile squid, and I
don't actually understand what to modify.

----------------------------
void
clientReplyContext::cloneReply()
{
    assert(reply == NULL);

    HttpReply *rep = http->storeEntry()->getReply()->clone();

    reply = HTTPMSGLOCK(rep);

    if (reply->sline.protocol == PROTO_HTTP) {
        /* enforce 1.0 reply version (but only on real HTTP traffic) */
    }

    /* do header conversions */
    buildReplyHeader();
}
-------------------------------

Sorry for my newbyness ;) !
Have a good day, regards,

Clem

-----Message d'origine-----
De : James Harper [mailto:[hidden email]]
Envoyé : mardi 12 juin 2012 08:08
À : James Harper; [hidden email]
Objet : [squid-users] RE: NTLM and persistent connections reverse proxy
3.1.20 - SOLVED + PATCH

> >
> > I'm having some problems with reverse proxy and NTLM authentication.
> > Specifically, the connection to the client is not persisted which I
> > believe invalidates the NTLM authentication protocol. I've added a
> > source port number to the logs which shows that it is indeed
> > creating a new connection for each request. There seems to have been
> > a bit of mailing list activity about similar problems but nothing
> > exactly the same and none of the suggested solutions work.
>
> I've done a bit more testing on this, and it seems that the server
> returns
> "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> Unauthorized" before passing it onto the client. Does that help?
>

It seems that this is the cause of the problem... The patch following this
email fixes it... is there any reason why the version should be forced to
1.0?? Is it to work around some other bug?

James

--- squid3-3.1.20.orig/src/client_side_reply.cc
+++ squid3-3.1.20/src/client_side_reply.cc
@@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()

     reply = HTTPMSGLOCK(rep);

+#if 0
     if (reply->sline.protocol == PROTO_HTTP) {
         /* enforce 1.0 reply version (but only on real HTTP traffic) */
         reply->sline.version = HttpVersion(1,0);
     }
+#endif

     /* do header conversions */
     buildReplyHeader();

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

James Harper
>
> Hi James,
>
> Your patch can interests me, but I'm a little bit confused on how apply it ?
> I've edited the client_side_reply.cc of the src before compile squid, and I
> don't actually understand what to modify.
>

Comment out or delete the whole "if (reply->..." statement

James

> ----------------------------
> void
> clientReplyContext::cloneReply()
> {
>     assert(reply == NULL);
>
>     HttpReply *rep = http->storeEntry()->getReply()->clone();
>
>     reply = HTTPMSGLOCK(rep);
>
>     if (reply->sline.protocol == PROTO_HTTP) {
>         /* enforce 1.0 reply version (but only on real HTTP traffic) */
>     }
>
>     /* do header conversions */
>     buildReplyHeader();
> }
> -------------------------------
>
> Sorry for my newbyness ;) !
> Have a good day, regards,
>
> Clem
>
> -----Message d'origine-----
> De : James Harper [mailto:[hidden email]]
> Envoyé : mardi 12 juin 2012 08:08
> À : James Harper; [hidden email] Objet : [squid-users] RE:
> NTLM and persistent connections reverse proxy
> 3.1.20 - SOLVED + PATCH
>
> > >
> > > I'm having some problems with reverse proxy and NTLM authentication.
> > > Specifically, the connection to the client is not persisted which I
> > > believe invalidates the NTLM authentication protocol. I've added a
> > > source port number to the logs which shows that it is indeed
> > > creating a new connection for each request. There seems to have been
> > > a bit of mailing list activity about similar problems but nothing
> > > exactly the same and none of the suggested solutions work.
> >
> > I've done a bit more testing on this, and it seems that the server
> > returns
> > "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> > Unauthorized" before passing it onto the client. Does that help?
> >
>
> It seems that this is the cause of the problem... The patch following this email
> fixes it... is there any reason why the version should be forced to 1.0?? Is it to
> work around some other bug?
>
> James
>
> --- squid3-3.1.20.orig/src/client_side_reply.cc
> +++ squid3-3.1.20/src/client_side_reply.cc
> @@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()
>
>      reply = HTTPMSGLOCK(rep);
>
> +#if 0
>      if (reply->sline.protocol == PROTO_HTTP) {
>          /* enforce 1.0 reply version (but only on real HTTP traffic) */
>          reply->sline.version = HttpVersion(1,0);
>      }
> +#endif
>
>      /* do header conversions */
>      buildReplyHeader();

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

cl00m
Ok great, thanks !

-----Message d'origine-----
De : James Harper [mailto:[hidden email]]
Envoyé : mardi 12 juin 2012 10:34
À : Clem; [hidden email]
Objet : RE: [squid-users] RE: NTLM and persistent connections reverse proxy
3.1.20 - SOLVED + PATCH

>
> Hi James,
>
> Your patch can interests me, but I'm a little bit confused on how apply it
?
> I've edited the client_side_reply.cc of the src before compile squid,
> and I don't actually understand what to modify.
>

Comment out or delete the whole "if (reply->..." statement

James

> ----------------------------
> void
> clientReplyContext::cloneReply()
> {
>     assert(reply == NULL);
>
>     HttpReply *rep = http->storeEntry()->getReply()->clone();
>
>     reply = HTTPMSGLOCK(rep);
>
>     if (reply->sline.protocol == PROTO_HTTP) {
>         /* enforce 1.0 reply version (but only on real HTTP traffic) */
>     }
>
>     /* do header conversions */
>     buildReplyHeader();
> }
> -------------------------------
>
> Sorry for my newbyness ;) !
> Have a good day, regards,
>
> Clem
>
> -----Message d'origine-----
> De : James Harper [mailto:[hidden email]]
> Envoyé : mardi 12 juin 2012 08:08
> À : James Harper; [hidden email] Objet : [squid-users] RE:
> NTLM and persistent connections reverse proxy
> 3.1.20 - SOLVED + PATCH
>
> > >
> > > I'm having some problems with reverse proxy and NTLM authentication.
> > > Specifically, the connection to the client is not persisted which
> > > I believe invalidates the NTLM authentication protocol. I've added
> > > a source port number to the logs which shows that it is indeed
> > > creating a new connection for each request. There seems to have
> > > been a bit of mailing list activity about similar problems but
> > > nothing exactly the same and none of the suggested solutions work.
> >
> > I've done a bit more testing on this, and it seems that the server
> > returns
> > "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> > Unauthorized" before passing it onto the client. Does that help?
> >
>
> It seems that this is the cause of the problem... The patch following
> this email fixes it... is there any reason why the version should be
> forced to 1.0?? Is it to work around some other bug?
>
> James
>
> --- squid3-3.1.20.orig/src/client_side_reply.cc
> +++ squid3-3.1.20/src/client_side_reply.cc
> @@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()
>
>      reply = HTTPMSGLOCK(rep);
>
> +#if 0
>      if (reply->sline.protocol == PROTO_HTTP) {
>          /* enforce 1.0 reply version (but only on real HTTP traffic) */
>          reply->sline.version = HttpVersion(1,0);
>      }
> +#endif
>
>      /* do header conversions */
>      buildReplyHeader();

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

cl00m
Thank you very much for your "patch" James, I can, at last, use my squid in
front of my exchange to forward all stuff to IIS rpc proxy in ntlm ! On XP
and W7 clients

-----Message d'origine-----
De : Clem [mailto:[hidden email]]
Envoyé : mardi 12 juin 2012 10:40
À : 'James Harper'; [hidden email]
Objet : RE: [squid-users] RE: NTLM and persistent connections reverse proxy
3.1.20 - SOLVED + PATCH

Ok great, thanks !

-----Message d'origine-----
De : James Harper [mailto:[hidden email]]
Envoyé : mardi 12 juin 2012 10:34
À : Clem; [hidden email]
Objet : RE: [squid-users] RE: NTLM and persistent connections reverse proxy
3.1.20 - SOLVED + PATCH

>
> Hi James,
>
> Your patch can interests me, but I'm a little bit confused on how
> apply it
?
> I've edited the client_side_reply.cc of the src before compile squid,
> and I don't actually understand what to modify.
>

Comment out or delete the whole "if (reply->..." statement

James

> ----------------------------
> void
> clientReplyContext::cloneReply()
> {
>     assert(reply == NULL);
>
>     HttpReply *rep = http->storeEntry()->getReply()->clone();
>
>     reply = HTTPMSGLOCK(rep);
>
>     if (reply->sline.protocol == PROTO_HTTP) {
>         /* enforce 1.0 reply version (but only on real HTTP traffic) */
>     }
>
>     /* do header conversions */
>     buildReplyHeader();
> }
> -------------------------------
>
> Sorry for my newbyness ;) !
> Have a good day, regards,
>
> Clem
>
> -----Message d'origine-----
> De : James Harper [mailto:[hidden email]]
> Envoyé : mardi 12 juin 2012 08:08
> À : James Harper; [hidden email] Objet : [squid-users] RE:
> NTLM and persistent connections reverse proxy
> 3.1.20 - SOLVED + PATCH
>
> > >
> > > I'm having some problems with reverse proxy and NTLM authentication.
> > > Specifically, the connection to the client is not persisted which
> > > I believe invalidates the NTLM authentication protocol. I've added
> > > a source port number to the logs which shows that it is indeed
> > > creating a new connection for each request. There seems to have
> > > been a bit of mailing list activity about similar problems but
> > > nothing exactly the same and none of the suggested solutions work.
> >
> > I've done a bit more testing on this, and it seems that the server
> > returns
> > "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0 401
> > Unauthorized" before passing it onto the client. Does that help?
> >
>
> It seems that this is the cause of the problem... The patch following
> this email fixes it... is there any reason why the version should be
> forced to 1.0?? Is it to work around some other bug?
>
> James
>
> --- squid3-3.1.20.orig/src/client_side_reply.cc
> +++ squid3-3.1.20/src/client_side_reply.cc
> @@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()
>
>      reply = HTTPMSGLOCK(rep);
>
> +#if 0
>      if (reply->sline.protocol == PROTO_HTTP) {
>          /* enforce 1.0 reply version (but only on real HTTP traffic) */
>          reply->sline.version = HttpVersion(1,0);
>      }
> +#endif
>
>      /* do header conversions */
>      buildReplyHeader();

Reply | Threaded
Open this post in threaded view
|

RE: NTLM and persistent connections reverse proxy 3.1.20 - SOLVED + PATCH

Amos Jeffries
Administrator
Be wary of HTTP/1.1 features using 1xx status codes which start
appearing when HTTP/1.1 is sent to clients. 1xx control messages are a
mandatory feature of HTTP/1.1 which Squid-3.1 does not support. Thus the
forced HTTP/1.0 sent by official packages.

Amos


On 13.06.2012 01:34, Clem wrote:

> Thank you very much for your "patch" James, I can, at last, use my
> squid in
> front of my exchange to forward all stuff to IIS rpc proxy in ntlm !
> On XP
> and W7 clients
>
> -----Message d'origine-----
> De : Clem
>
> Ok great, thanks !
>
> -----Message d'origine-----
> De : James Harper [mailto:[hidden email]]
> Envoyé : mardi 12 juin 2012 10:34
> À : Clem; [hidden email]
> Objet : RE: [squid-users] RE: NTLM and persistent connections reverse
> proxy
> 3.1.20 - SOLVED + PATCH
>
>>
>> Hi James,
>>
>> Your patch can interests me, but I'm a little bit confused on how
>> apply it
> ?
>> I've edited the client_side_reply.cc of the src before compile
>> squid,
>> and I don't actually understand what to modify.
>>
>
> Comment out or delete the whole "if (reply->..." statement
>
> James
>
>> ----------------------------
>> void
>> clientReplyContext::cloneReply()
>> {
>>     assert(reply == NULL);
>>
>>     HttpReply *rep = http->storeEntry()->getReply()->clone();
>>
>>     reply = HTTPMSGLOCK(rep);
>>
>>     if (reply->sline.protocol == PROTO_HTTP) {
>>         /* enforce 1.0 reply version (but only on real HTTP traffic)
>> */
>>     }
>>
>>     /* do header conversions */
>>     buildReplyHeader();
>> }
>> -------------------------------
>>
>> Sorry for my newbyness ;) !
>> Have a good day, regards,
>>
>> Clem
>>
>> -----Message d'origine-----
>> De : James Harper [mailto:[hidden email]]
>> Envoyé : mardi 12 juin 2012 08:08
>> À : James Harper; [hidden email] Objet : [squid-users]
>> RE:
>> NTLM and persistent connections reverse proxy
>> 3.1.20 - SOLVED + PATCH
>>
>> > >
>> > > I'm having some problems with reverse proxy and NTLM
>> authentication.
>> > > Specifically, the connection to the client is not persisted
>> which
>> > > I believe invalidates the NTLM authentication protocol. I've
>> added
>> > > a source port number to the logs which shows that it is indeed
>> > > creating a new connection for each request. There seems to have
>> > > been a bit of mailing list activity about similar problems but
>> > > nothing exactly the same and none of the suggested solutions
>> work.
>> >
>> > I've done a bit more testing on this, and it seems that the server
>> > returns
>> > "HTTP/1.1 401 Unauthorized" but squid turns this into "HTTP/1.0
>> 401
>> > Unauthorized" before passing it onto the client. Does that help?
>> >
>>
>> It seems that this is the cause of the problem... The patch
>> following
>> this email fixes it... is there any reason why the version should be
>> forced to 1.0?? Is it to work around some other bug?
>>
>> James
>>
>> --- squid3-3.1.20.orig/src/client_side_reply.cc
>> +++ squid3-3.1.20/src/client_side_reply.cc
>> @@ -1469,10 +1469,12 @@ clientReplyContext::cloneReply()
>>
>>      reply = HTTPMSGLOCK(rep);
>>
>> +#if 0
>>      if (reply->sline.protocol == PROTO_HTTP) {
>>          /* enforce 1.0 reply version (but only on real HTTP
>> traffic) */
>>          reply->sline.version = HttpVersion(1,0);
>>      }
>> +#endif
>>
>>      /* do header conversions */
>>      buildReplyHeader();