How to enable caching for https websites on Squid

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

How to enable caching for https websites on Squid

Sekar Duraisamy
Hi Team,

Please let me know how to enable caching for https websites and can we
configure squid proxy to maintain anonymous as we can configure for
http?
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Amos Jeffries
Administrator
On 19/12/17 23:33, Sekar Duraisamy wrote:
> Hi Team,
>
> Please let me know how to enable caching for https websites and can we
> configure squid proxy to maintain anonymous as we can configure for
> http?

To cache encryption protected content you must first remove the
encryption. That destroys the "anonymous" part completely.

Also, when a service uses TLS properly it is not possible to decrypt in
a proxy.

If you accept the above limitations, look into the SSL-Bump feature of
Squid for details on how to configure decrypting of HTTPS content.


Once Squid is handling decrypted content the caching should "just
happen", same as with HTTP traffic going through the proxy.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Matus UHLAR - fantomas
>On 19/12/17 23:33, Sekar Duraisamy wrote:
>>Please let me know how to enable caching for https websites and can we
>>configure squid proxy to maintain anonymous as we can configure for
>>http?

On 21.12.17 01:10, Amos Jeffries wrote:

>To cache encryption protected content you must first remove the
>encryption. That destroys the "anonymous" part completely.
>
>Also, when a service uses TLS properly it is not possible to decrypt
>in a proxy.
>
>If you accept the above limitations, look into the SSL-Bump feature
>of Squid for details on how to configure decrypting of HTTPS content.
>
>
>Once Squid is handling decrypted content the caching should "just
>happen", same as with HTTP traffic going through the proxy.

and I think you should read the last paragraph as:

  "caching often will not happen, since most of web developers don't know hot
   so use and benefit of it thus they try to disable caching globally"
--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I'm not interested in your website anymore.
If you need cookies, bake them yourself.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Amos Jeffries
Administrator
On 21/12/17 01:23, Matus UHLAR - fantomas wrote:

>> On 19/12/17 23:33, Sekar Duraisamy wrote:
>>> Please let me know how to enable caching for https websites and can we
>>> configure squid proxy to maintain anonymous as we can configure for
>>> http?
>
> On 21.12.17 01:10, Amos Jeffries wrote:
>> To cache encryption protected content you must first remove the
>> encryption. That destroys the "anonymous" part completely.
>>
>> Also, when a service uses TLS properly it is not possible to decrypt
>> in a proxy.
>>
>> If you accept the above limitations, look into the SSL-Bump feature of
>> Squid for details on how to configure decrypting of HTTPS content.
>>
>>
>> Once Squid is handling decrypted content the caching should "just
>> happen", same as with HTTP traffic going through the proxy.
>
> and I think you should read the last paragraph as:
>
>   "caching often will not happen, since most of web developers don't
> know hot
>    so use and benefit of it thus they try to disable caching globally"

That is nothing special for HTTPS, it happens worse in regular HTTP.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Matus UHLAR - fantomas
>On 21/12/17 01:23, Matus UHLAR - fantomas wrote:
>>and I think you should read the last paragraph as:
>>
>>  "caching often will not happen, since most of web developers
>>don't know hot
>>   so use and benefit of it thus they try to disable caching globally"

On 21.12.17 02:20, Amos Jeffries wrote:
>That is nothing special for HTTPS, it happens worse in regular HTTP.

do you want to say that breaking into https can cause http caching more
efficient?
do you have any evidence of that?

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
M$ Win's are shit, do not use it !
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Amos Jeffries
Administrator
On 21/12/17 02:41, Matus UHLAR - fantomas wrote:

>> On 21/12/17 01:23, Matus UHLAR - fantomas wrote:
>>> and I think you should read the last paragraph as:
>>>
>>>  "caching often will not happen, since most of web developers don't
>>> know hot
>>>   so use and benefit of it thus they try to disable caching globally"
>
> On 21.12.17 02:20, Amos Jeffries wrote:
>> That is nothing special for HTTPS, it happens worse in regular HTTP.
>
> do you want to say that breaking into https can cause http caching more
> efficient?
> do you have any evidence of that?
>

No, I am saying that the problem you pointed at is a _larger_ problem in
http:// because those dev are having to actively prevent caching. Many
are also under the false impression that https:// goes end-to-end and
caching does not happen there other than Browser cache. So those who
develop sites with HTTPS in mind do not go to quite such extremes to
block proxies caching.

HTTPS has _other_ problems that impact on caching efficiency.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Sekar Duraisamy
Hi Amos,

Thanks for your reply .

"To cache encryption protected content you must first remove the
encryption. That destroys the "anonymous" part completely."

Could you please provide little more details about this line about it
destroys the anonymous while we decrypt the encryption and enable
caching for https?

 https caching for anonymous proxy is not recommended?

On Wed, Dec 20, 2017 at 8:42 PM, Amos Jeffries <[hidden email]> wrote:

> On 21/12/17 02:41, Matus UHLAR - fantomas wrote:
>>>
>>> On 21/12/17 01:23, Matus UHLAR - fantomas wrote:
>>>>
>>>> and I think you should read the last paragraph as:
>>>>
>>>>  "caching often will not happen, since most of web developers don't know
>>>> hot
>>>>   so use and benefit of it thus they try to disable caching globally"
>>
>>
>> On 21.12.17 02:20, Amos Jeffries wrote:
>>>
>>> That is nothing special for HTTPS, it happens worse in regular HTTP.
>>
>>
>> do you want to say that breaking into https can cause http caching more
>> efficient?
>> do you have any evidence of that?
>>
>
> No, I am saying that the problem you pointed at is a _larger_ problem in
> http:// because those dev are having to actively prevent caching. Many are
> also under the false impression that https:// goes end-to-end and caching
> does not happen there other than Browser cache. So those who develop sites
> with HTTPS in mind do not go to quite such extremes to block proxies
> caching.
>
> HTTPS has _other_ problems that impact on caching efficiency.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Matus UHLAR - fantomas
On 29.12.17 12:38, Sekar Duraisamy wrote:
>"To cache encryption protected content you must first remove the
>encryption. That destroys the "anonymous" part completely."
>
>Could you please provide little more details about this line about it
>destroys the anonymous while we decrypt the encryption and enable
>caching for https?

the whole point of SSL and HTTPS is that nobody between client (browser) and
the final server knows what's inside. This logically prevents caching, since
you can not know what is the content you are transferring, so you can't know
if you can provide the contant from cache.

you need to break into https - behave as the final server, provide your
own certificate instead (because you can't fake the real server's) and look
into content.

Note that many clients will complain about your certificate - you need to
import your proxy's certificate to clients' browsers to avoid that,

and still, some clients will detect that they are not communicating to
final server and refuse to work (this has been reported a few times here).

> https caching for anonymous proxy is not recommended?

your customer may look anonymous to the world (hidden behind your proxy)
even without breaking HTTPS.

But by decrypting https they will lose privacy.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows found: (R)emove, (E)rase, (D)elete
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Sekar Duraisamy
Thanks for your reply.

So the same proxy certificate will be expose for all the requests even
though we are sending more requests through load-balancing of more IP
addresses from the server which will be an anonymity risk?


On Fri, Dec 29, 2017 at 3:17 PM, Matus UHLAR - fantomas
<[hidden email]> wrote:

> On 29.12.17 12:38, Sekar Duraisamy wrote:
>>
>> "To cache encryption protected content you must first remove the
>> encryption. That destroys the "anonymous" part completely."
>>
>> Could you please provide little more details about this line about it
>> destroys the anonymous while we decrypt the encryption and enable
>> caching for https?
>
>
> the whole point of SSL and HTTPS is that nobody between client (browser) and
> the final server knows what's inside. This logically prevents caching, since
> you can not know what is the content you are transferring, so you can't know
> if you can provide the contant from cache.
>
> you need to break into https - behave as the final server, provide your
> own certificate instead (because you can't fake the real server's) and look
> into content.
>
> Note that many clients will complain about your certificate - you need to
> import your proxy's certificate to clients' browsers to avoid that,
>
> and still, some clients will detect that they are not communicating to
> final server and refuse to work (this has been reported a few times here).
>
>> https caching for anonymous proxy is not recommended?
>
>
> your customer may look anonymous to the world (hidden behind your proxy)
> even without breaking HTTPS.
> But by decrypting https they will lose privacy.
>
> --
> Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Windows found: (R)emove, (E)rase, (D)elete
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Sekar Duraisamy
In reply to this post by Amos Jeffries
Hi Amos,

Thanks for your information

"To cache encryption protected content you must first remove the
encryption. That destroys the "anonymous" part completely."

Could you please provide little more details about affecting anonymous
service. Do you meant it will affect customers anonymous or from proxy
server?

We used to disable via and forwarded_for header to make squid proxy as
anonymous in HTTP.

When we use certificate in the Proxy server to decrypt the content of
HTTPS, multiple customers will hit to the same HTTPS website in a day
through our proxy, that website always see single certificate even
though multiple customers from multiple IPs. Is there a chance from
website can block
because of they will see more requests from more IP's but single
certificate for the all the requests to the same doamin ?

On Wed, Dec 20, 2017 at 8:42 PM, Amos Jeffries <[hidden email]> wrote:

> On 21/12/17 02:41, Matus UHLAR - fantomas wrote:
>>>
>>> On 21/12/17 01:23, Matus UHLAR - fantomas wrote:
>>>>
>>>> and I think you should read the last paragraph as:
>>>>
>>>>  "caching often will not happen, since most of web developers don't know
>>>> hot
>>>>   so use and benefit of it thus they try to disable caching globally"
>>
>>
>> On 21.12.17 02:20, Amos Jeffries wrote:
>>>
>>> That is nothing special for HTTPS, it happens worse in regular HTTP.
>>
>>
>> do you want to say that breaking into https can cause http caching more
>> efficient?
>> do you have any evidence of that?
>>
>
> No, I am saying that the problem you pointed at is a _larger_ problem in
> http:// because those dev are having to actively prevent caching. Many are
> also under the false impression that https:// goes end-to-end and caching
> does not happen there other than Browser cache. So those who develop sites
> with HTTPS in mind do not go to quite such extremes to block proxies
> caching.
>
> HTTPS has _other_ problems that impact on caching efficiency.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Matus UHLAR - fantomas
On 09.01.18 17:15, Sekar Duraisamy wrote:
>"To cache encryption protected content you must first remove the
>encryption. That destroys the "anonymous" part completely."
>
>Could you please provide little more details about affecting anonymous
>service. Do you meant it will affect customers anonymous or from proxy
>server?

I believe you have been answered already multiple times, but once more:

the customer will have no privacy against proxy server - the proxy server
will see everything they access, all the content etc.

This is impossible with SSL - SSL has been created just to provide privacy
to users, so nobody sees the content, only the final server.

With HTTPS decrypting the destination server will only see your proxy
accessing, no IP, browser info (if you decide to hide it) but the proxy will
see everything.  Proxy admins will be able to see their passwords, their
mail, banking account information, etc.
 
If your users are OK with that, fine.  The question is if they really want
this kind of anonymity.

>When we use certificate in the Proxy server to decrypt the content of
>HTTPS, multiple customers will hit to the same HTTPS website in a day
>through our proxy, that website always see single certificate even
>though multiple customers from multiple IPs. Is there a chance from
>website can block
>because of they will see more requests from more IP's but single
>certificate for the all the requests to the same doamin ?

The end servers will not see your proxy certificate.
The HTTP server certificate is used to authentize server, not the client.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The only substitute for good manners is fast reflexes.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: How to enable caching for https websites on Squid

Sekar Duraisamy
Thanks for your reply. Yes. I agree about the customers privacy and we
should not decrypt. My users are not even using this proxy for their
personal purpose and not passing any personal information and simple
browse the pages to explore the wiki pages, technical information or
any images like that.

Just thought of enabling cache for https for few websites to just save
more internet bandwidth utilization and cost saving of internet usage
as most of the websites now moved from http to https.

On Tue, Jan 9, 2018 at 5:42 PM, Matus UHLAR - fantomas
<[hidden email]> wrote:

> On 09.01.18 17:15, Sekar Duraisamy wrote:
>>
>> "To cache encryption protected content you must first remove the
>> encryption. That destroys the "anonymous" part completely."
>>
>> Could you please provide little more details about affecting anonymous
>> service. Do you meant it will affect customers anonymous or from proxy
>> server?
>
>
> I believe you have been answered already multiple times, but once more:
>
> the customer will have no privacy against proxy server - the proxy server
> will see everything they access, all the content etc.
>
> This is impossible with SSL - SSL has been created just to provide privacy
> to users, so nobody sees the content, only the final server.
>
> With HTTPS decrypting the destination server will only see your proxy
> accessing, no IP, browser info (if you decide to hide it) but the proxy will
> see everything.  Proxy admins will be able to see their passwords, their
> mail, banking account information, etc.
>  If your users are OK with that, fine.  The question is if they really want
> this kind of anonymity.
>
>> When we use certificate in the Proxy server to decrypt the content of
>> HTTPS, multiple customers will hit to the same HTTPS website in a day
>> through our proxy, that website always see single certificate even
>> though multiple customers from multiple IPs. Is there a chance from
>> website can block
>> because of they will see more requests from more IP's but single
>> certificate for the all the requests to the same doamin ?
>
>
> The end servers will not see your proxy certificate. The HTTP server
> certificate is used to authentize server, not the client.
>
> --
> Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> The only substitute for good manners is fast reflexes.
> _______________________________________________
>
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users