Re: Reverse proxying Exchange OWA wembail with SSL offloading

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Scott
On Wed, Oct 28, 2020 at 12:00:01PM +0000, [hidden email] wrote:

> Date: Thu, 29 Oct 2020 00:08:34 +1300
> From: Amos Jeffries <[hidden email]>
> To: [hidden email]
> Subject: Re: [squid-users] Reverse proxying Exchange OWA wembail with SSL
>  offloading - not working on IE/Chrome
>
> On 28/10/20 5:25 pm, Scott wrote:
> >
> > Here are the logs (first not working, followed by working).
> >
> > Note this is the login attempt, not the loading of the initial page.  You'll
> > see in the NOT WORKING section that the browser does NOT return a cookie to
> > the server, which is where the problem may be.  Again, I'm not sure why - I'm
> > thinking perhaps the browser/javascript is rejecting the cookie as it's
> > missing the "secure" attribute (because the back-end is talking plain HTTP).
> >
>
> The complete absence of a cookie may be expected to break something.
>
> The absence of a "secure" flag should only make the cookie vulnerable to
> leaking. It should not affect anything depending on that cookies value.
>
>
> Amos
>

After some more research and experimentation I've confirmed that my
suspicions are correct.

Recent browsers are no longer accepting cookies with the SameSite flag set
without the Secure flag set.

It's not an issue with Squid (although one Squid could solve - but I'm unsure
of the implications).

Here is a useful link:
https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications

I tested Chrome 85 on Windows - the default settings DO NOT allow for these
cookies.  However after setting
        Cookies without SameSite must be secure
to Disabled, these cookies are permitted and OWA works.

There are obvious implications for sites doing SSL offloading here.  Are
sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
flag?

Interesting.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Amos Jeffries
Administrator
On 29/10/20 12:06 pm, Scott wrote:

> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
>> Date: Thu, 29 Oct 2020 00:08:34 +1300
>> From: Amos Jeffries
>>
>> On 28/10/20 5:25 pm, Scott wrote:
>>>
>>> Here are the logs (first not working, followed by working).
>>>
>>> Note this is the login attempt, not the loading of the initial page.  You'll
>>> see in the NOT WORKING section that the browser does NOT return a cookie to
>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
>>>
>>
>> The complete absence of a cookie may be expected to break something.
>>
>> The absence of a "secure" flag should only make the cookie vulnerable to
>> leaking. It should not affect anything depending on that cookies value.
>>
>>
>> Amos
>>
>
> After some more research and experimentation I've confirmed that my
> suspicions are correct.
>
> Recent browsers are no longer accepting cookies with the SameSite flag set
> without the Secure flag set.
>
> It's not an issue with Squid (although one Squid could solve - but I'm unsure
> of the implications).

Implications are that the server may have intentionally used the
combination it did, no mistakes.

The server is given "Front-End-Https: On" so that it is aware the client
is using HTTPS and can set (or not) the secure flag appropriately to
what it needs. Squid is not aware of whether the cookie is safe to use
on HTTP or restrict to just HTTPS.


>
> Here is a useful link:
> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
>
> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
> cookies.  However after setting
> Cookies without SameSite must be secure
> to Disabled, these cookies are permitted and OWA works.
>
> There are obvious implications for sites doing SSL offloading here.  Are
> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
> flag?


Neither. When a site frontend is entirely https:// with no http://
resources mixed in the Secure flag can be used by the server regardless
of what the internal connections are.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Eliezer Croitoru-3
In reply to this post by Scott
Hey Scott,

In many frontends there is a basic way to signal about the existence of a Frontend and the relevant
details about the client and other factors of the connection.

Specifically with haproxy these settings are used:
    http-request set-header X-Forwarded-Host %[req.hdr(Host)]
    http-request set-header X-Forwarded-Proto https

https://www.haproxy.com/blog/ssl-offloading-impact-on-web-applications/
https://www.digitalocean.com/community/tutorials/how-to-implement-ssl-termination-with-haproxy-on-ubuntu-14-04

In nginx:
https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/
https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-load-balancing-with-ssl-termination


In squid you can use:
http://www.squid-cache.org/Doc/config/request_header_replace/
http://www.squid-cache.org/Doc/config/request_header_add/

An example should be provided in wiki some day be someone(not me).

----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: [hidden email]

-----Original Message-----
From: squid-users <[hidden email]> On Behalf Of Scott
Sent: Thursday, October 29, 2020 1:07 AM
To: [hidden email]
Subject: Re: [squid-users] Reverse proxying Exchange OWA wembail with SSL offloading

On Wed, Oct 28, 2020 at 12:00:01PM +0000, [hidden email] wrote:

> Date: Thu, 29 Oct 2020 00:08:34 +1300
> From: Amos Jeffries <[hidden email]>
> To: [hidden email]
> Subject: Re: [squid-users] Reverse proxying Exchange OWA wembail with SSL
>  offloading - not working on IE/Chrome
>
> On 28/10/20 5:25 pm, Scott wrote:
> >
> > Here are the logs (first not working, followed by working).
> >
> > Note this is the login attempt, not the loading of the initial page.  You'll
> > see in the NOT WORKING section that the browser does NOT return a cookie to
> > the server, which is where the problem may be.  Again, I'm not sure why - I'm
> > thinking perhaps the browser/javascript is rejecting the cookie as it's
> > missing the "secure" attribute (because the back-end is talking plain HTTP).
> >
>
> The complete absence of a cookie may be expected to break something.
>
> The absence of a "secure" flag should only make the cookie vulnerable to
> leaking. It should not affect anything depending on that cookies value.
>
>
> Amos
>

After some more research and experimentation I've confirmed that my
suspicions are correct.

Recent browsers are no longer accepting cookies with the SameSite flag set
without the Secure flag set.

It's not an issue with Squid (although one Squid could solve - but I'm unsure
of the implications).

Here is a useful link:
https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications

I tested Chrome 85 on Windows - the default settings DO NOT allow for these
cookies.  However after setting
        Cookies without SameSite must be secure
to Disabled, these cookies are permitted and OWA works.

There are obvious implications for sites doing SSL offloading here.  Are
sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
flag?

Interesting.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Scott
In reply to this post by Amos Jeffries
On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:

> On 29/10/20 12:06 pm, Scott wrote:
> > On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
> >> Date: Thu, 29 Oct 2020 00:08:34 +1300
> >> From: Amos Jeffries
> >>
> >> On 28/10/20 5:25 pm, Scott wrote:
> >>>
> >>> Here are the logs (first not working, followed by working).
> >>>
> >>> Note this is the login attempt, not the loading of the initial page.  You'll
> >>> see in the NOT WORKING section that the browser does NOT return a cookie to
> >>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
> >>> thinking perhaps the browser/javascript is rejecting the cookie as it's
> >>> missing the "secure" attribute (because the back-end is talking plain HTTP).
> >>>
> >>
> >> The complete absence of a cookie may be expected to break something.
> >>
> >> The absence of a "secure" flag should only make the cookie vulnerable to
> >> leaking. It should not affect anything depending on that cookies value.
> >>
> >>
> >> Amos
> >>
> >
> > After some more research and experimentation I've confirmed that my
> > suspicions are correct.
> >
> > Recent browsers are no longer accepting cookies with the SameSite flag set
> > without the Secure flag set.
> >
> > It's not an issue with Squid (although one Squid could solve - but I'm unsure
> > of the implications).
>
> Implications are that the server may have intentionally used the
> combination it did, no mistakes.
>
> The server is given "Front-End-Https: On" so that it is aware the client
> is using HTTPS and can set (or not) the secure flag appropriately to
> what it needs. Squid is not aware of whether the cookie is safe to use
> on HTTP or restrict to just HTTPS.
>
>
> >
> > Here is a useful link:
> > https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
> >
> > I tested Chrome 85 on Windows - the default settings DO NOT allow for these
> > cookies.  However after setting
> > Cookies without SameSite must be secure
> > to Disabled, these cookies are permitted and OWA works.
> >
> > There are obvious implications for sites doing SSL offloading here.  Are
> > sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
> > flag?
>
>
> Neither. When a site frontend is entirely https:// with no http://
> resources mixed in the Secure flag can be used by the server regardless
> of what the internal connections are.
>
>
> Amos

My point is that, assuming browsers are now enforcing SameSite cookies must
be secure, then doing SSL offload (whereby the origin server does NOT flag
the cookie as secure) will break.

In this particular case, I use squid to do reverse proxy with SSL offload to
a MS Exchange server.
Because the requests are HTTP IIS does not set the secure flag on cookies,
and browsers now reject them.  This breaks things.

I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
so the secure flag is now being set by IIS.  Problem solved.

In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
think) will not be an option.  I've seen other proxy manufacturers provide
cookie manipulation; I assume for this kind of issue.

Do you think Squid should have such a feature?

Thanks
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Amos Jeffries
Administrator
On 30/10/20 3:27 pm, Scott wrote:

> On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:
>> On 29/10/20 12:06 pm, Scott wrote:
>>> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
>>>> Date: Thu, 29 Oct 2020 00:08:34 +1300
>>>> From: Amos Jeffries
>>>>
>>>> On 28/10/20 5:25 pm, Scott wrote:
>>>>>
>>>>> Here are the logs (first not working, followed by working).
>>>>>
>>>>> Note this is the login attempt, not the loading of the initial page.  You'll
>>>>> see in the NOT WORKING section that the browser does NOT return a cookie to
>>>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
>>>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
>>>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
>>>>>
>>>>
>>>> The complete absence of a cookie may be expected to break something.
>>>>
>>>> The absence of a "secure" flag should only make the cookie vulnerable to
>>>> leaking. It should not affect anything depending on that cookies value.
>>>>
>>>>
>>>> Amos
>>>>
>>>
>>> After some more research and experimentation I've confirmed that my
>>> suspicions are correct.
>>>
>>> Recent browsers are no longer accepting cookies with the SameSite flag set
>>> without the Secure flag set.
>>>
>>> It's not an issue with Squid (although one Squid could solve - but I'm unsure
>>> of the implications).
>>
>> Implications are that the server may have intentionally used the
>> combination it did, no mistakes.
>>
>> The server is given "Front-End-Https: On" so that it is aware the client
>> is using HTTPS and can set (or not) the secure flag appropriately to
>> what it needs. Squid is not aware of whether the cookie is safe to use
>> on HTTP or restrict to just HTTPS.
>>
>>
>>>
>>> Here is a useful link:
>>> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
>>>
>>> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
>>> cookies.  However after setting
>>> Cookies without SameSite must be secure
>>> to Disabled, these cookies are permitted and OWA works.
>>>
>>> There are obvious implications for sites doing SSL offloading here.  Are
>>> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
>>> flag?
>>
>>
>> Neither. When a site frontend is entirely https:// with no http://
>> resources mixed in the Secure flag can be used by the server regardless
>> of what the internal connections are.
>>
>>
>> Amos
>
> My point is that, assuming browsers are now enforcing SameSite cookies must
> be secure, then doing SSL offload (whereby the origin server does NOT flag
> the cookie as secure) will break.

But offloading does not mean the server omits the secure flag.

Servers which choose to send it when offloading work fine. Servers that
choose to omit it have problems with the Google paranoid interpretation
of security.


>
> In this particular case, I use squid to do reverse proxy with SSL offload to
> a MS Exchange server.

You also have a TLS connection to the exchange:443 peer.

Notice that in the logs you showed transactions sent to that peer get
the secure flag set, despite the SSL offload being done by Squid at the
same time.



> Because the requests are HTTP IIS does not set the secure flag on cookies,
> and browsers now reject them.  This breaks things.
>
> I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
> so the secure flag is now being set by IIS.  Problem solved.
>
> In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
> think) will not be an option.  I've seen other proxy manufacturers provide
> cookie manipulation; I assume for this kind of issue.
>
> Do you think Squid should have such a feature?

We have ICAP, eCAP, and *_header_replace features already. So any
proposed new feature would have to be better than what they already provide.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Scott
On Sat, Oct 31, 2020 at 12:49:16AM +1300, Amos Jeffries wrote:

> On 30/10/20 3:27 pm, Scott wrote:
> > On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:
> >> On 29/10/20 12:06 pm, Scott wrote:
> >>> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
> >>>> Date: Thu, 29 Oct 2020 00:08:34 +1300
> >>>> From: Amos Jeffries
> >>>>
> >>>> On 28/10/20 5:25 pm, Scott wrote:
> >>>>>
> >>>>> Here are the logs (first not working, followed by working).
> >>>>>
> >>>>> Note this is the login attempt, not the loading of the initial page.  You'll
> >>>>> see in the NOT WORKING section that the browser does NOT return a cookie to
> >>>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
> >>>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
> >>>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
> >>>>>
> >>>>
> >>>> The complete absence of a cookie may be expected to break something.
> >>>>
> >>>> The absence of a "secure" flag should only make the cookie vulnerable to
> >>>> leaking. It should not affect anything depending on that cookies value.
> >>>>
> >>>>
> >>>> Amos
> >>>>
> >>>
> >>> After some more research and experimentation I've confirmed that my
> >>> suspicions are correct.
> >>>
> >>> Recent browsers are no longer accepting cookies with the SameSite flag set
> >>> without the Secure flag set.
> >>>
> >>> It's not an issue with Squid (although one Squid could solve - but I'm unsure
> >>> of the implications).
> >>
> >> Implications are that the server may have intentionally used the
> >> combination it did, no mistakes.
> >>
> >> The server is given "Front-End-Https: On" so that it is aware the client
> >> is using HTTPS and can set (or not) the secure flag appropriately to
> >> what it needs. Squid is not aware of whether the cookie is safe to use
> >> on HTTP or restrict to just HTTPS.
> >>
> >>
> >>>
> >>> Here is a useful link:
> >>> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
> >>>
> >>> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
> >>> cookies.  However after setting
> >>> Cookies without SameSite must be secure
> >>> to Disabled, these cookies are permitted and OWA works.
> >>>
> >>> There are obvious implications for sites doing SSL offloading here.  Are
> >>> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
> >>> flag?
> >>
> >>
> >> Neither. When a site frontend is entirely https:// with no http://
> >> resources mixed in the Secure flag can be used by the server regardless
> >> of what the internal connections are.
> >>
> >>
> >> Amos
> >
> > My point is that, assuming browsers are now enforcing SameSite cookies must
> > be secure, then doing SSL offload (whereby the origin server does NOT flag
> > the cookie as secure) will break.
>
> But offloading does not mean the server omits the secure flag.
>
> Servers which choose to send it when offloading work fine. Servers that
> choose to omit it have problems with the Google paranoid interpretation
> of security.

Aren't origin servers oblivious to SSL offloading?  I thought they just
happily accepted HTTP connections with no knowledge of any secure channel
between client and reverse proxy.

If that's correct then, although not proscribed by the RFC, it's unlikely
that a Set-Cookie header would ever contain a Secure flag because the server
would be saying "I insist this Cookie be transmitted securely, but here it is
over an insecure channel".

>
>
> >
> > In this particular case, I use squid to do reverse proxy with SSL offload to
> > a MS Exchange server.
>
> You also have a TLS connection to the exchange:443 peer.

I have both for testing, but only one is active at any one time (as
referenced in a cache_peer_access).  Yes I now use the 443 peer because
Chrome/IE/Edge + SSL Offloading no longer works.

>
> Notice that in the logs you showed transactions sent to that peer get
> the secure flag set, despite the SSL offload being done by Squid at the
> same time.
>
>

Not true AFAICT.  Any cookies you see in those logs with Secure set are shown
under the "WORKING" heading whose entries are:
2020/10/28 12:01:23.549 kid1| 11,2| http.cc(2263) sendRequest: HTTP Server local=squid-internal:62597 remote=exchange:443 FD 30 flags=1

No SSL offloading there.  Otherwise it would say "remote=exchange:80"

>
> > Because the requests are HTTP IIS does not set the secure flag on cookies,
> > and browsers now reject them.  This breaks things.
> >
> > I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
> > so the secure flag is now being set by IIS.  Problem solved.
> >
> > In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
> > think) will not be an option.  I've seen other proxy manufacturers provide
> > cookie manipulation; I assume for this kind of issue.
> >
> > Do you think Squid should have such a feature?
>
> We have ICAP, eCAP, and *_header_replace features already. So any
> proposed new feature would have to be better than what they already provide.
>

Unless I'm reading the online command reference incorrectly, *_header_replace
does not allow for the insertion of text so can't help here.  Please tell me
I'm wrong - I'd love for Squid to have this capability like other proxies.

>
> Amos
>

I can't tell if we're agreeing or not :).  Suffice to say SSL offloading of
OWA with modern browsers is becoming a thing of the past, at least with Squid
(without ICAP).

This MS article basically sums up what we've discussed here.
https://docs.microsoft.com/en-us/answers/questions/73384/chrome-80-samesite-feature-causes-owa-login-loop.html

The advice therein talks about:
- Toggling the setting in Chrome
- Do a cookie rewrite on the SSL offloader device if possible
- change the backend communication to HTTPS (turn off SSL offloading)

Anyway this conversation may help someone else.
Thanks
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Alex Rousskov
In reply to this post by Scott
On 10/29/20 10:27 PM, Scott wrote:

> My point is that, assuming browsers are now enforcing SameSite cookies must
> be secure, then doing SSL offload (whereby the origin server does NOT flag
> the cookie as secure) will break.

...

> I've seen other proxy manufacturers provide
> cookie manipulation; I assume for this kind of issue.

> Do you think Squid should have such a feature?

Yes, I do.

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Amos Jeffries
Administrator
In reply to this post by Scott
On 31/10/20 3:20 am, Scott wrote:

> On Sat, Oct 31, 2020 at 12:49:16AM +1300, Amos Jeffries wrote:
>> On 30/10/20 3:27 pm, Scott wrote:
>>> On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:
>>>> On 29/10/20 12:06 pm, Scott wrote:
>>>>> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
>>>>>> Date: Thu, 29 Oct 2020 00:08:34 +1300
>>>>>> From: Amos Jeffries
>>>>>>
>>>>>> On 28/10/20 5:25 pm, Scott wrote:
>>>>>>>
>>>>>>> Here are the logs (first not working, followed by working).
>>>>>>>
>>>>>>> Note this is the login attempt, not the loading of the initial page.  You'll
>>>>>>> see in the NOT WORKING section that the browser does NOT return a cookie to
>>>>>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
>>>>>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
>>>>>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
>>>>>>>
>>>>>>
>>>>>> The complete absence of a cookie may be expected to break something.
>>>>>>
>>>>>> The absence of a "secure" flag should only make the cookie vulnerable to
>>>>>> leaking. It should not affect anything depending on that cookies value.
>>>>>>
>>>>>>
>>>>>> Amos
>>>>>>
>>>>>
>>>>> After some more research and experimentation I've confirmed that my
>>>>> suspicions are correct.
>>>>>
>>>>> Recent browsers are no longer accepting cookies with the SameSite flag set
>>>>> without the Secure flag set.
>>>>>
>>>>> It's not an issue with Squid (although one Squid could solve - but I'm unsure
>>>>> of the implications).
>>>>
>>>> Implications are that the server may have intentionally used the
>>>> combination it did, no mistakes.
>>>>
>>>> The server is given "Front-End-Https: On" so that it is aware the client
>>>> is using HTTPS and can set (or not) the secure flag appropriately to
>>>> what it needs. Squid is not aware of whether the cookie is safe to use
>>>> on HTTP or restrict to just HTTPS.
>>>>
>>>>
>>>>>
>>>>> Here is a useful link:
>>>>> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
>>>>>
>>>>> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
>>>>> cookies.  However after setting
>>>>> Cookies without SameSite must be secure
>>>>> to Disabled, these cookies are permitted and OWA works.
>>>>>
>>>>> There are obvious implications for sites doing SSL offloading here.  Are
>>>>> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
>>>>> flag?
>>>>
>>>>
>>>> Neither. When a site frontend is entirely https:// with no http://
>>>> resources mixed in the Secure flag can be used by the server regardless
>>>> of what the internal connections are.
>>>>
>>>>
>>>> Amos
>>>
>>> My point is that, assuming browsers are now enforcing SameSite cookies must
>>> be secure, then doing SSL offload (whereby the origin server does NOT flag
>>> the cookie as secure) will break.
>>
>> But offloading does not mean the server omits the secure flag.
>>
>> Servers which choose to send it when offloading work fine. Servers that
>> choose to omit it have problems with the Google paranoid interpretation
>> of security.
>
> Aren't origin servers oblivious to SSL offloading?  I thought they just
> happily accepted HTTP connections with no knowledge of any secure channel
> between client and reverse proxy.


It varies.  Mozilla have a brief writeup about the headers used:
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto>


>
> If that's correct then, although not proscribed by the RFC, it's unlikely
> that a Set-Cookie header would ever contain a Secure flag because the server
> would be saying "I insist this Cookie be transmitted securely, but here it is
> over an insecure channel".
>

That assumes the only type of security is HTTPS. The channel between
proxy and server may be secured via other means than TLS, even as it
transmits "plain text" HTTP.

There should be server settings to administrate that. I do not use OWA
or IIS though, so now sure if they are available for your case.


>>>
>>> In this particular case, I use squid to do reverse proxy with SSL offload to
>>> a MS Exchange server.
>>
>> You also have a TLS connection to the exchange:443 peer.
>
> I have both for testing, but only one is active at any one time (as
> referenced in a cache_peer_access).  Yes I now use the 443 peer because
> Chrome/IE/Edge + SSL Offloading no longer works.
>

Please understand that with Squid acting as a reverse-proxy "offloading"
is still happening. It is the cert you put in the https_port in
Squid.conf that the clients are dealing with, and Squid which is doing
any client cert validation.


For working security on the cache_peer you should identify the CA cert
which signed the peer server's TLS certificate. Load that into Squid as
cache_peer tls-cafile=  and you can drop the DONT_VERIFY_PEER flag.


>>
>> Notice that in the logs you showed transactions sent to that peer get
>> the secure flag set, despite the SSL offload being done by Squid at the
>> same time.
>>
>>
>
> Not true AFAICT.  Any cookies you see in those logs with Secure set are shown
> under the "WORKING" heading whose entries are:
> 2020/10/28 12:01:23.549 kid1| 11,2| http.cc(2263) sendRequest: HTTP Server local=squid-internal:62597 remote=exchange:443 FD 30 flags=1
>
> No SSL offloading there.  Otherwise it would say "remote=exchange:80"
>

Squid is receiving the HTTP(S) message. That means Squid is terminating
the TLS from client, decrypting the traffic. Which is offloading.

The connection to the server is independent. Squid will actually be
multiplexing traffic for multiple clients on that connection. No client
has participation or knowledge of that TLS existence.


A non-offloading setup would be a CONNECT tunnel through Squid to the
server so TLS can be negotiated between client and server.




>>
>>> Because the requests are HTTP IIS does not set the secure flag on cookies,
>>> and browsers now reject them.  This breaks things.
>>>
>>> I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
>>> so the secure flag is now being set by IIS.  Problem solved.
>>>
>>> In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
>>> think) will not be an option.  I've seen other proxy manufacturers provide
>>> cookie manipulation; I assume for this kind of issue.
>>>
>>> Do you think Squid should have such a feature?
>>
>> We have ICAP, eCAP, and *_header_replace features already. So any
>> proposed new feature would have to be better than what they already provide.
>>
>
> Unless I'm reading the online command reference incorrectly, *_header_replace
> does not allow for the insertion of text so can't help here.  Please tell me
> I'm wrong - I'd love for Squid to have this capability like other proxies.


Aye. There are limits and issues with all those features. I was speaking
there about what a new feature proposal would need to work with for
consideration.


The options right now are (in order of preference):

* create an eCAP module that just edits the headers as-needed.

* create an ICAP service to do header edits as-needed.

* create an external_acl_type helper that takes the reply headers as
input and delivers Squid a set of annotations that can be injected as
Set-Cookie headers by http_reply_header_add. With *header_access to
remove broken values first.

* new feature in Squid.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxying Exchange OWA wembail with SSL offloading

Amos Jeffries
Administrator
In reply to this post by Scott
On 31/10/20 3:20 am, Scott wrote:

> On Sat, Oct 31, 2020 at 12:49:16AM +1300, Amos Jeffries wrote:
>> On 30/10/20 3:27 pm, Scott wrote:
>>> On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:
>>>> On 29/10/20 12:06 pm, Scott wrote:
>>>>> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
>>>>>> Date: Thu, 29 Oct 2020 00:08:34 +1300
>>>>>> From: Amos Jeffries
>>>>>>
>>>>>> On 28/10/20 5:25 pm, Scott wrote:
>>>>>>>
>>>>>>> Here are the logs (first not working, followed by working).
>>>>>>>
>>>>>>> Note this is the login attempt, not the loading of the initial page.  You'll
>>>>>>> see in the NOT WORKING section that the browser does NOT return a cookie to
>>>>>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
>>>>>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
>>>>>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
>>>>>>>
>>>>>>
>>>>>> The complete absence of a cookie may be expected to break something.
>>>>>>
>>>>>> The absence of a "secure" flag should only make the cookie vulnerable to
>>>>>> leaking. It should not affect anything depending on that cookies value.
>>>>>>
>>>>>>
>>>>>> Amos
>>>>>>
>>>>>
>>>>> After some more research and experimentation I've confirmed that my
>>>>> suspicions are correct.
>>>>>
>>>>> Recent browsers are no longer accepting cookies with the SameSite flag set
>>>>> without the Secure flag set.
>>>>>
>>>>> It's not an issue with Squid (although one Squid could solve - but I'm unsure
>>>>> of the implications).
>>>>
>>>> Implications are that the server may have intentionally used the
>>>> combination it did, no mistakes.
>>>>
>>>> The server is given "Front-End-Https: On" so that it is aware the client
>>>> is using HTTPS and can set (or not) the secure flag appropriately to
>>>> what it needs. Squid is not aware of whether the cookie is safe to use
>>>> on HTTP or restrict to just HTTPS.
>>>>
>>>>
>>>>>
>>>>> Here is a useful link:
>>>>> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
>>>>>
>>>>> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
>>>>> cookies.  However after setting
>>>>> Cookies without SameSite must be secure
>>>>> to Disabled, these cookies are permitted and OWA works.
>>>>>
>>>>> There are obvious implications for sites doing SSL offloading here.  Are
>>>>> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
>>>>> flag?
>>>>
>>>>
>>>> Neither. When a site frontend is entirely https:// with no http://
>>>> resources mixed in the Secure flag can be used by the server regardless
>>>> of what the internal connections are.
>>>>
>>>>
>>>> Amos
>>>
>>> My point is that, assuming browsers are now enforcing SameSite cookies must
>>> be secure, then doing SSL offload (whereby the origin server does NOT flag
>>> the cookie as secure) will break.
>>
>> But offloading does not mean the server omits the secure flag.
>>
>> Servers which choose to send it when offloading work fine. Servers that
>> choose to omit it have problems with the Google paranoid interpretation
>> of security.
>
> Aren't origin servers oblivious to SSL offloading?  I thought they just
> happily accepted HTTP connections with no knowledge of any secure channel
> between client and reverse proxy.


It varies.  Mozilla have a brief writeup about the headers used:
<https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto>


>
> If that's correct then, although not proscribed by the RFC, it's unlikely
> that a Set-Cookie header would ever contain a Secure flag because the server
> would be saying "I insist this Cookie be transmitted securely, but here it is
> over an insecure channel".
>

That assumes the only type of security is HTTPS. The channel between
proxy and server may be secured via other means than TLS, even as it
transmits "plain text" HTTP.

There should be server settings to administrate that. I do not use OWA
or IIS though, so now sure if they are available for your case.


>>>
>>> In this particular case, I use squid to do reverse proxy with SSL offload to
>>> a MS Exchange server.
>>
>> You also have a TLS connection to the exchange:443 peer.
>
> I have both for testing, but only one is active at any one time (as
> referenced in a cache_peer_access).  Yes I now use the 443 peer because
> Chrome/IE/Edge + SSL Offloading no longer works.
>

Please understand that with Squid acting as a reverse-proxy "offloading"
is still happening. It is the cert you put in the https_port in
Squid.conf that the clients are dealing with, and Squid which is doing
any client cert validation.


For working security on the cache_peer you should identify the CA cert
which signed the peer server's TLS certificate. Load that into Squid as
cache_peer tls-cafile=  and you can drop the DONT_VERIFY_PEER flag.


>>
>> Notice that in the logs you showed transactions sent to that peer get
>> the secure flag set, despite the SSL offload being done by Squid at the
>> same time.
>>
>>
>
> Not true AFAICT.  Any cookies you see in those logs with Secure set are shown
> under the "WORKING" heading whose entries are:
> 2020/10/28 12:01:23.549 kid1| 11,2| http.cc(2263) sendRequest: HTTP Server local=squid-internal:62597 remote=exchange:443 FD 30 flags=1
>
> No SSL offloading there.  Otherwise it would say "remote=exchange:80"
>

Squid is receiving the HTTP(S) message. That means Squid is terminating
the TLS from client, decrypting the traffic. Which is what I understand
to be offloading.

The connection to the server is independent. Squid will actually be
multiplexing traffic for multiple clients on that connection. No client
has participation or knowledge of that TLS existence.


Non-offloading would be a CONNECT tunnel through Squid to the server so
TLS can be negotiated directly between client and server.


>>
>>> Because the requests are HTTP IIS does not set the secure flag on cookies,
>>> and browsers now reject them.  This breaks things.
>>>
>>> I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
>>> so the secure flag is now being set by IIS.  Problem solved.
>>>
>>> In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
>>> think) will not be an option.  I've seen other proxy manufacturers provide
>>> cookie manipulation; I assume for this kind of issue.
>>>
>>> Do you think Squid should have such a feature?
>>
>> We have ICAP, eCAP, and *_header_replace features already. So any
>> proposed new feature would have to be better than what they already provide.
>>
>
> Unless I'm reading the online command reference incorrectly, *_header_replace
> does not allow for the insertion of text so can't help here.  Please tell me
> I'm wrong - I'd love for Squid to have this capability like other proxies.


Aye. There are limits and issues with all those features. I was speaking
there about what a new feature proposal would need to work with for
consideration.


The options right now are (in order of preference):

* create an eCAP module that just edits the headers as-needed.

* create an ICAP service to do header edits as-needed.

* create an external_acl_type helper that takes the reply headers as
input and delivers Squid a set of annotations that can be injected as
Set-Cookie headers by http_reply_header_add. With *header_access to
remove broken values first.

* new feature in Squid.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users