SSL intercept in explicit mode

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

SSL intercept in explicit mode

Danilo V
Is it possible/feasible to configure squid in explicit mode with ssl intercept?
Due to architecture of my network it is not possible to implement transparent proxy.
What would be the behavior of applications that dont support proxy - i.e. dont forward requests to proxy?
Any guides?

Danilo

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Matus UHLAR - fantomas
On 13.03.18 13:44, Danilo V wrote:
>Is it possible/feasible to configure squid in explicit mode with ssl
>intercept?

explicit is not intercept, intercept is not explicit.

explicit is where browser is configured (manually or automatically via WPAD)
to use the proxy.

intercept is where network device forcifully redirects http/https connections
to the proxy.

maybe you mean SSL bump in explicit mode?

>Due to architecture of my network it is not possible to implement
>transparent proxy.

excuse me?
by "transparent" people mean what we usually call "intercept".

>What would be the behavior of applications that dont support proxy - i.e.
>dont forward requests to proxy?

they mest be intercepted.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Danilo V
I mean SSL bump in explicit mode.
So intercept is a essencial requirement for running SSL bump?

Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas <[hidden email]> escreveu:
On 13.03.18 13:44, Danilo V wrote:
>Is it possible/feasible to configure squid in explicit mode with ssl
>intercept?

explicit is not intercept, intercept is not explicit.

explicit is where browser is configured (manually or automatically via WPAD)
to use the proxy.

intercept is where network device forcifully redirects http/https connections
to the proxy.

maybe you mean SSL bump in explicit mode?

>Due to architecture of my network it is not possible to implement
>transparent proxy.

excuse me?
by "transparent" people mean what we usually call "intercept".

>What would be the behavior of applications that dont support proxy - i.e.
>dont forward requests to proxy?

they mest be intercepted.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Marcus Kool
"SSL bump" is the name of a complex Squid feature.
With ssl_bump ACLs one can decide which domains can be 'spliced' (go through the proxy untouched) or can be 'bumped' (decrypted).

Interception is not a requirement for SSL bump.

Marcus

On 13/03/18 11:44, Danilo V wrote:

> I mean SSL bump in explicit mode.
> So intercept is a essencial requirement for running SSL bump?
>
> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas <[hidden email] <mailto:[hidden email]>> escreveu:
>
>     On 13.03.18 13:44, Danilo V wrote:
>      >Is it possible/feasible to configure squid in explicit mode with ssl
>      >intercept?
>
>     explicit is not intercept, intercept is not explicit.
>
>     explicit is where browser is configured (manually or automatically via WPAD)
>     to use the proxy.
>
>     intercept is where network device forcifully redirects http/https connections
>     to the proxy.
>
>     maybe you mean SSL bump in explicit mode?
>
>      >Due to architecture of my network it is not possible to implement
>      >transparent proxy.
>
>     excuse me?
>     by "transparent" people mean what we usually call "intercept".
>
>      >What would be the behavior of applications that dont support proxy - i.e.
>      >dont forward requests to proxy?
>
>     they mest be intercepted.
>
>     --
>     Matus UHLAR - fantomas, [hidden email] <mailto:[hidden email]> ; http://www.fantomas.sk/
>     Warning: I wish NOT to receive e-mail advertising to this address.
>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>     _______________________________________________
>     squid-users mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov
Moreover,

SSL Bump combines with interception/explicit proxy in one setup.

And works perfectly.


13.03.2018 21:14, Marcus Kool пишет:

> "SSL bump" is the name of a complex Squid feature.
> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
> through the proxy untouched) or can be 'bumped' (decrypted).
>
> Interception is not a requirement for SSL bump.
>
> Marcus
>
> On 13/03/18 11:44, Danilo V wrote:
>> I mean SSL bump in explicit mode.
>> So intercept is a essencial requirement for running SSL bump?
>>
>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>> <[hidden email] <mailto:[hidden email]>> escreveu:
>>
>>     On 13.03.18 13:44, Danilo V wrote:
>>      >Is it possible/feasible to configure squid in explicit mode
>> with ssl
>>      >intercept?
>>
>>     explicit is not intercept, intercept is not explicit.
>>
>>     explicit is where browser is configured (manually or
>> automatically via WPAD)
>>     to use the proxy.
>>
>>     intercept is where network device forcifully redirects http/https
>> connections
>>     to the proxy.
>>
>>     maybe you mean SSL bump in explicit mode?
>>
>>      >Due to architecture of my network it is not possible to implement
>>      >transparent proxy.
>>
>>     excuse me?
>>     by "transparent" people mean what we usually call "intercept".
>>
>>      >What would be the behavior of applications that dont support
>> proxy - i.e.
>>      >dont forward requests to proxy?
>>
>>     they mest be intercepted.
>>
>>     --
>>     Matus UHLAR - fantomas, [hidden email]
>> <mailto:[hidden email]> ; http://www.fantomas.sk/
>>     Warning: I wish NOT to receive e-mail advertising to this address.
>>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>     _______________________________________________
>>     squid-users mailing list
>>     [hidden email]
>> <mailto:[hidden email]>
>>     http://lists.squid-cache.org/listinfo/squid-users
>>
>>
>>
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
>>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Aaron Turner
What version are you using Yuri?  Can you share your config?
Everytime I use ssl bump, I have massive memory leaks.  It's been
effectively unusable for me.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:10 AM, Yuri <[hidden email]> wrote:

> Moreover,
>
> SSL Bump combines with interception/explicit proxy in one setup.
>
> And works perfectly.
>
>
> 13.03.2018 21:14, Marcus Kool пишет:
>> "SSL bump" is the name of a complex Squid feature.
>> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
>> through the proxy untouched) or can be 'bumped' (decrypted).
>>
>> Interception is not a requirement for SSL bump.
>>
>> Marcus
>>
>> On 13/03/18 11:44, Danilo V wrote:
>>> I mean SSL bump in explicit mode.
>>> So intercept is a essencial requirement for running SSL bump?
>>>
>>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>>> <[hidden email] <mailto:[hidden email]>> escreveu:
>>>
>>>     On 13.03.18 13:44, Danilo V wrote:
>>>      >Is it possible/feasible to configure squid in explicit mode
>>> with ssl
>>>      >intercept?
>>>
>>>     explicit is not intercept, intercept is not explicit.
>>>
>>>     explicit is where browser is configured (manually or
>>> automatically via WPAD)
>>>     to use the proxy.
>>>
>>>     intercept is where network device forcifully redirects http/https
>>> connections
>>>     to the proxy.
>>>
>>>     maybe you mean SSL bump in explicit mode?
>>>
>>>      >Due to architecture of my network it is not possible to implement
>>>      >transparent proxy.
>>>
>>>     excuse me?
>>>     by "transparent" people mean what we usually call "intercept".
>>>
>>>      >What would be the behavior of applications that dont support
>>> proxy - i.e.
>>>      >dont forward requests to proxy?
>>>
>>>     they mest be intercepted.
>>>
>>>     --
>>>     Matus UHLAR - fantomas, [hidden email]
>>> <mailto:[hidden email]> ; http://www.fantomas.sk/
>>>     Warning: I wish NOT to receive e-mail advertising to this address.
>>>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>>     _______________________________________________
>>>     squid-users mailing list
>>>     [hidden email]
>>> <mailto:[hidden email]>
>>>     http://lists.squid-cache.org/listinfo/squid-users
>>>
>>>
>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> [hidden email]
>>> http://lists.squid-cache.org/listinfo/squid-users
>>>
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov
I've used it on all versions starting from 3.4.

Now I'm using Squid 5.0.0.

I'm afraid, my config is completely useless, because of it contains tons
of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
with different memory allocator for custom infrastructure.

You can't just take my config, implement it and hope it will give same
results for you.

At least, it uses non-system CA bundle, platform-specific configuration
parameters combinations, etc.

I can say, than SSL Bump is not directly related to memory leaks. Squid
itself almost not contains memory leaks now. Usually misconfiguration
leads to memory overhead.

As a recommendation, I can give some advices.

1. Use server with enough RAM. 4 Gb usually enough just for default
squid configuration. Usually whole system RAM usage should never be
bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
should always be free during normal running. This prevents OS allocator
pressure to your proxy and, also, increasing performance of proxy). In
case of medium proxy server 16 Gb of RAM seems big enough, but never try
to fill it up completely.

2. Don't set giant cache_mem. Remember how you platform allocates whole
RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
memory-related settings.

3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.

4. Never remember - SSL Bump increases your cache memory pressure due to
increasing caching. So, you still require to have enough memory in your
system.


13.03.2018 22:25, Aaron Turner пишет:

> What version are you using Yuri?  Can you share your config?
> Everytime I use ssl bump, I have massive memory leaks.  It's been
> effectively unusable for me.
> --
> Aaron Turner
> https://synfin.net/         Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality.  "Something cannot emerge from nothing,"
> he said.  This is profound thinking if you understand how unstable
> "the truth" can be.  -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <[hidden email]> wrote:
>> Moreover,
>>
>> SSL Bump combines with interception/explicit proxy in one setup.
>>
>> And works perfectly.
>>
>>
>> 13.03.2018 21:14, Marcus Kool пишет:
>>> "SSL bump" is the name of a complex Squid feature.
>>> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
>>> through the proxy untouched) or can be 'bumped' (decrypted).
>>>
>>> Interception is not a requirement for SSL bump.
>>>
>>> Marcus
>>>
>>> On 13/03/18 11:44, Danilo V wrote:
>>>> I mean SSL bump in explicit mode.
>>>> So intercept is a essencial requirement for running SSL bump?
>>>>
>>>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>>>> <[hidden email] <mailto:[hidden email]>> escreveu:
>>>>
>>>>     On 13.03.18 13:44, Danilo V wrote:
>>>>      >Is it possible/feasible to configure squid in explicit mode
>>>> with ssl
>>>>      >intercept?
>>>>
>>>>     explicit is not intercept, intercept is not explicit.
>>>>
>>>>     explicit is where browser is configured (manually or
>>>> automatically via WPAD)
>>>>     to use the proxy.
>>>>
>>>>     intercept is where network device forcifully redirects http/https
>>>> connections
>>>>     to the proxy.
>>>>
>>>>     maybe you mean SSL bump in explicit mode?
>>>>
>>>>      >Due to architecture of my network it is not possible to implement
>>>>      >transparent proxy.
>>>>
>>>>     excuse me?
>>>>     by "transparent" people mean what we usually call "intercept".
>>>>
>>>>      >What would be the behavior of applications that dont support
>>>> proxy - i.e.
>>>>      >dont forward requests to proxy?
>>>>
>>>>     they mest be intercepted.
>>>>
>>>>     --
>>>>     Matus UHLAR - fantomas, [hidden email]
>>>> <mailto:[hidden email]> ; http://www.fantomas.sk/
>>>>     Warning: I wish NOT to receive e-mail advertising to this address.
>>>>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>>>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>>>     _______________________________________________
>>>>     squid-users mailing list
>>>>     [hidden email]
>>>> <mailto:[hidden email]>
>>>>     http://lists.squid-cache.org/listinfo/squid-users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> squid-users mailing list
>>>> [hidden email]
>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> [hidden email]
>>> http://lists.squid-cache.org/listinfo/squid-users
>> --
>> "C++ seems like a language suitable for firing other people's legs."
>>
>> *****************************
>> * C++20 : Bug to the future *
>> *****************************
>>
>>
>>
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
>>
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Aaron Turner
"Usually misconfiguration leads to memory overhead."

This may be true, but if you look in the list archives a few months
ago I basically chased my tail in circles and nobody could tell me
what I was doing wrong and so many of the docs are so old that they're
worse then useless, they seem to suggest the wrong thing.

It was literally leaking GB's worth of RAM.  I even disabled all
caching and process sizes were growing into the GB's.  Turn off
ssl-bump and the leak went away.

This is what I was using:

http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
sslflags=NO_DEFAULT_CA
http_port localhost:3128
ssl_bump bump all

sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
sslcrtd_children 32 startup=2 idle=2
sslproxy_session_cache_size 100 MB
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

This was on a machine (EC2 VM) with 14GB of RAM.

--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:47 AM, Yuri <[hidden email]> wrote:

> I've used it on all versions starting from 3.4.
>
> Now I'm using Squid 5.0.0.
>
> I'm afraid, my config is completely useless, because of it contains tons
> of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
> with different memory allocator for custom infrastructure.
>
> You can't just take my config, implement it and hope it will give same
> results for you.
>
> At least, it uses non-system CA bundle, platform-specific configuration
> parameters combinations, etc.
>
> I can say, than SSL Bump is not directly related to memory leaks. Squid
> itself almost not contains memory leaks now. Usually misconfiguration
> leads to memory overhead.
>
> As a recommendation, I can give some advices.
>
> 1. Use server with enough RAM. 4 Gb usually enough just for default
> squid configuration. Usually whole system RAM usage should never be
> bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
> should always be free during normal running. This prevents OS allocator
> pressure to your proxy and, also, increasing performance of proxy). In
> case of medium proxy server 16 Gb of RAM seems big enough, but never try
> to fill it up completely.
>
> 2. Don't set giant cache_mem. Remember how you platform allocates whole
> RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
> memory-related settings.
>
> 3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.
>
> 4. Never remember - SSL Bump increases your cache memory pressure due to
> increasing caching. So, you still require to have enough memory in your
> system.
>
>
> 13.03.2018 22:25, Aaron Turner пишет:
>> What version are you using Yuri?  Can you share your config?
>> Everytime I use ssl bump, I have massive memory leaks.  It's been
>> effectively unusable for me.
>> --
>> Aaron Turner
>> https://synfin.net/         Twitter: @synfinatic
>> My father once told me that respect for the truth comes close to being
>> the basis for all morality.  "Something cannot emerge from nothing,"
>> he said.  This is profound thinking if you understand how unstable
>> "the truth" can be.  -- Frank Herbert, Dune
>>
>>
>> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <[hidden email]> wrote:
>>> Moreover,
>>>
>>> SSL Bump combines with interception/explicit proxy in one setup.
>>>
>>> And works perfectly.
>>>
>>>
>>> 13.03.2018 21:14, Marcus Kool пишет:
>>>> "SSL bump" is the name of a complex Squid feature.
>>>> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
>>>> through the proxy untouched) or can be 'bumped' (decrypted).
>>>>
>>>> Interception is not a requirement for SSL bump.
>>>>
>>>> Marcus
>>>>
>>>> On 13/03/18 11:44, Danilo V wrote:
>>>>> I mean SSL bump in explicit mode.
>>>>> So intercept is a essencial requirement for running SSL bump?
>>>>>
>>>>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>>>>> <[hidden email] <mailto:[hidden email]>> escreveu:
>>>>>
>>>>>     On 13.03.18 13:44, Danilo V wrote:
>>>>>      >Is it possible/feasible to configure squid in explicit mode
>>>>> with ssl
>>>>>      >intercept?
>>>>>
>>>>>     explicit is not intercept, intercept is not explicit.
>>>>>
>>>>>     explicit is where browser is configured (manually or
>>>>> automatically via WPAD)
>>>>>     to use the proxy.
>>>>>
>>>>>     intercept is where network device forcifully redirects http/https
>>>>> connections
>>>>>     to the proxy.
>>>>>
>>>>>     maybe you mean SSL bump in explicit mode?
>>>>>
>>>>>      >Due to architecture of my network it is not possible to implement
>>>>>      >transparent proxy.
>>>>>
>>>>>     excuse me?
>>>>>     by "transparent" people mean what we usually call "intercept".
>>>>>
>>>>>      >What would be the behavior of applications that dont support
>>>>> proxy - i.e.
>>>>>      >dont forward requests to proxy?
>>>>>
>>>>>     they mest be intercepted.
>>>>>
>>>>>     --
>>>>>     Matus UHLAR - fantomas, [hidden email]
>>>>> <mailto:[hidden email]> ; http://www.fantomas.sk/
>>>>>     Warning: I wish NOT to receive e-mail advertising to this address.
>>>>>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>>>>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>>>>     _______________________________________________
>>>>>     squid-users mailing list
>>>>>     [hidden email]
>>>>> <mailto:[hidden email]>
>>>>>     http://lists.squid-cache.org/listinfo/squid-users
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> squid-users mailing list
>>>>> [hidden email]
>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>
>>>> _______________________________________________
>>>> squid-users mailing list
>>>> [hidden email]
>>>> http://lists.squid-cache.org/listinfo/squid-users
>>> --
>>> "C++ seems like a language suitable for firing other people's legs."
>>>
>>> *****************************
>>> * C++20 : Bug to the future *
>>> *****************************
>>>
>>>
>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> [hidden email]
>>> http://lists.squid-cache.org/listinfo/squid-users
>>>
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov

AFAIK,

SSL bump subsystem uses OpenSSL memory routines. So, first of all, most probably leaks (if any) can be OpenSSL-related, but not squid itself.

Now let's see your config snippets.

13.03.2018 23:00, Aaron Turner пишет:
"Usually misconfiguration leads to memory overhead."

This may be true, but if you look in the list archives a few months
ago I basically chased my tail in circles and nobody could tell me
what I was doing wrong and so many of the docs are so old that they're
worse then useless, they seem to suggest the wrong thing.

It was literally leaking GB's worth of RAM.  I even disabled all
caching and process sizes were growing into the GB's.  Turn off
ssl-bump and the leak went away.

This is what I was using:

http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
sslflags=NO_DEFAULT_CA
http_port localhost:3128
ssl_bump bump all
bump all is useless without peek/splice.

Let's see on my config snippets:

https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
# Cert database on ramdisk
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 1GB
sslcrtd_children 32 startup=10 idle=5

# SSL bump rules
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex "/usr/local/squid/etc/acl.url.nobump"
ssl_bump peek DiscoverSNIHost
ssl_bump splice NoSSLIntercept
ssl_bump bump all


sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
sslcrtd_children 32 startup=2 idle=2
This is defaults. Pay attention, -M is limits use ssl_db directory to 4 Mb in size. It's too few for production servers. My ramdisk for ssl db is 1+ Gb in size:

/dev/ramdisk/ramdisk1           961M   14M  890M   2% /ramdisk1/ssl_db

sslproxy_session_cache_size 100 MB
This is disbalanced size instead of previous setting. Why so big?

#  TAG: sslproxy_session_cache_size
#        Sets the cache size to use for ssl session
#Default:
# sslproxy_session_cache_size 2 MB

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
NEVER use this options. It is unsafe.

SSL Bump is dangerous enough itself. Don't do it more unsafe additionally by yourself.

This was on a machine (EC2 VM) with 14GB of RAM.
Pay attention on several places:

1. OS memory allocator.
2. OpenSSL version.
3. OS configuration (IPC, shared memory, swap - all memory related).
4. Squid's memory/pools configuration.

Don't forget about: Often memory fragmentation seems like leaks. But no leaks occurs indeed.

Also, don't forget - squid's memory consumption is not only cache_mem, but also caching on-disk metadata (swap.state), pools settings, working memory areas, processes memory. And - also - such things like content adaptation (did you know wide uses ecap gzip adapter is leaky itself?).

But this is just for example.

In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.

I know, this appears SSL Bump is leaky. But this is not correct.

--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:47 AM, Yuri [hidden email] wrote:
I've used it on all versions starting from 3.4.

Now I'm using Squid 5.0.0.

I'm afraid, my config is completely useless, because of it contains tons
of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
with different memory allocator for custom infrastructure.

You can't just take my config, implement it and hope it will give same
results for you.

At least, it uses non-system CA bundle, platform-specific configuration
parameters combinations, etc.

I can say, than SSL Bump is not directly related to memory leaks. Squid
itself almost not contains memory leaks now. Usually misconfiguration
leads to memory overhead.

As a recommendation, I can give some advices.

1. Use server with enough RAM. 4 Gb usually enough just for default
squid configuration. Usually whole system RAM usage should never be
bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
should always be free during normal running. This prevents OS allocator
pressure to your proxy and, also, increasing performance of proxy). In
case of medium proxy server 16 Gb of RAM seems big enough, but never try
to fill it up completely.

2. Don't set giant cache_mem. Remember how you platform allocates whole
RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
memory-related settings.

3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.

4. Never remember - SSL Bump increases your cache memory pressure due to
increasing caching. So, you still require to have enough memory in your
system.


13.03.2018 22:25, Aaron Turner пишет:
What version are you using Yuri?  Can you share your config?
Everytime I use ssl bump, I have massive memory leaks.  It's been
effectively unusable for me.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:10 AM, Yuri [hidden email] wrote:
Moreover,

SSL Bump combines with interception/explicit proxy in one setup.

And works perfectly.


13.03.2018 21:14, Marcus Kool пишет:
"SSL bump" is the name of a complex Squid feature.
With ssl_bump ACLs one can decide which domains can be 'spliced' (go
through the proxy untouched) or can be 'bumped' (decrypted).

Interception is not a requirement for SSL bump.

Marcus

On 13/03/18 11:44, Danilo V wrote:
I mean SSL bump in explicit mode.
So intercept is a essencial requirement for running SSL bump?

Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
<[hidden email] [hidden email]> escreveu:

    On 13.03.18 13:44, Danilo V wrote:
     >Is it possible/feasible to configure squid in explicit mode
with ssl
     >intercept?

    explicit is not intercept, intercept is not explicit.

    explicit is where browser is configured (manually or
automatically via WPAD)
    to use the proxy.

    intercept is where network device forcifully redirects http/https
connections
    to the proxy.

    maybe you mean SSL bump in explicit mode?

     >Due to architecture of my network it is not possible to implement
     >transparent proxy.

    excuse me?
    by "transparent" people mean what we usually call "intercept".

     >What would be the behavior of applications that dont support
proxy - i.e.
     >dont forward requests to proxy?

    they mest be intercepted.

    --
    Matus UHLAR - fantomas, [hidden email]
[hidden email] ; http://www.fantomas.sk/
    Warning: I wish NOT to receive e-mail advertising to this address.
    Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
    Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
    _______________________________________________
    squid-users mailing list
    [hidden email]
[hidden email]
    http://lists.squid-cache.org/listinfo/squid-users



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov

FInally,

just take a look:

This is SSL Bump-aware setup. Seems no memory leaks, yes? Normal memory distribution.

Let's see on overall OS memory:

No leaks.

13.03.2018 23:44, Yuri пишет:

AFAIK,

SSL bump subsystem uses OpenSSL memory routines. So, first of all, most probably leaks (if any) can be OpenSSL-related, but not squid itself.

Now let's see your config snippets.

13.03.2018 23:00, Aaron Turner пишет:
"Usually misconfiguration leads to memory overhead."

This may be true, but if you look in the list archives a few months
ago I basically chased my tail in circles and nobody could tell me
what I was doing wrong and so many of the docs are so old that they're
worse then useless, they seem to suggest the wrong thing.

It was literally leaking GB's worth of RAM.  I even disabled all
caching and process sizes were growing into the GB's.  Turn off
ssl-bump and the leak went away.

This is what I was using:

http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
sslflags=NO_DEFAULT_CA
http_port localhost:3128
ssl_bump bump all
bump all is useless without peek/splice.

Let's see on my config snippets:

https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt key=/usr/local/squid/etc/rootCA2.key tls-cafile=/usr/local/squid/etc/rootCA12.crt options=SINGLE_DH_USE:SINGLE_ECDH_USE tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
# Cert database on ramdisk
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 1GB
sslcrtd_children 32 startup=10 idle=5

# SSL bump rules
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex "/usr/local/squid/etc/acl.url.nobump"
ssl_bump peek DiscoverSNIHost
ssl_bump splice NoSSLIntercept
ssl_bump bump all

sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
sslcrtd_children 32 startup=2 idle=2
This is defaults. Pay attention, -M is limits use ssl_db directory to 4 Mb in size. It's too few for production servers. My ramdisk for ssl db is 1+ Gb in size:

/dev/ramdisk/ramdisk1           961M   14M  890M   2% /ramdisk1/ssl_db

sslproxy_session_cache_size 100 MB
This is disbalanced size instead of previous setting. Why so big?

#  TAG: sslproxy_session_cache_size
#        Sets the cache size to use for ssl session
#Default:
# sslproxy_session_cache_size 2 MB

sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
NEVER use this options. It is unsafe.

SSL Bump is dangerous enough itself. Don't do it more unsafe additionally by yourself.
This was on a machine (EC2 VM) with 14GB of RAM.
Pay attention on several places:

1. OS memory allocator.
2. OpenSSL version.
3. OS configuration (IPC, shared memory, swap - all memory related).
4. Squid's memory/pools configuration.

Don't forget about: Often memory fragmentation seems like leaks. But no leaks occurs indeed.

Also, don't forget - squid's memory consumption is not only cache_mem, but also caching on-disk metadata (swap.state), pools settings, working memory areas, processes memory. And - also - such things like content adaptation (did you know wide uses ecap gzip adapter is leaky itself?).

But this is just for example.

In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.

I know, this appears SSL Bump is leaky. But this is not correct.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:47 AM, Yuri [hidden email] wrote:
I've used it on all versions starting from 3.4.

Now I'm using Squid 5.0.0.

I'm afraid, my config is completely useless, because of it contains tons
of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
with different memory allocator for custom infrastructure.

You can't just take my config, implement it and hope it will give same
results for you.

At least, it uses non-system CA bundle, platform-specific configuration
parameters combinations, etc.

I can say, than SSL Bump is not directly related to memory leaks. Squid
itself almost not contains memory leaks now. Usually misconfiguration
leads to memory overhead.

As a recommendation, I can give some advices.

1. Use server with enough RAM. 4 Gb usually enough just for default
squid configuration. Usually whole system RAM usage should never be
bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
should always be free during normal running. This prevents OS allocator
pressure to your proxy and, also, increasing performance of proxy). In
case of medium proxy server 16 Gb of RAM seems big enough, but never try
to fill it up completely.

2. Don't set giant cache_mem. Remember how you platform allocates whole
RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
memory-related settings.

3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.

4. Never remember - SSL Bump increases your cache memory pressure due to
increasing caching. So, you still require to have enough memory in your
system.


13.03.2018 22:25, Aaron Turner пишет:
What version are you using Yuri?  Can you share your config?
Everytime I use ssl bump, I have massive memory leaks.  It's been
effectively unusable for me.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 9:10 AM, Yuri [hidden email] wrote:
Moreover,

SSL Bump combines with interception/explicit proxy in one setup.

And works perfectly.


13.03.2018 21:14, Marcus Kool пишет:
"SSL bump" is the name of a complex Squid feature.
With ssl_bump ACLs one can decide which domains can be 'spliced' (go
through the proxy untouched) or can be 'bumped' (decrypted).

Interception is not a requirement for SSL bump.

Marcus

On 13/03/18 11:44, Danilo V wrote:
I mean SSL bump in explicit mode.
So intercept is a essencial requirement for running SSL bump?

Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
<[hidden email] [hidden email]> escreveu:

    On 13.03.18 13:44, Danilo V wrote:
     >Is it possible/feasible to configure squid in explicit mode
with ssl
     >intercept?

    explicit is not intercept, intercept is not explicit.

    explicit is where browser is configured (manually or
automatically via WPAD)
    to use the proxy.

    intercept is where network device forcifully redirects http/https
connections
    to the proxy.

    maybe you mean SSL bump in explicit mode?

     >Due to architecture of my network it is not possible to implement
     >transparent proxy.

    excuse me?
    by "transparent" people mean what we usually call "intercept".

     >What would be the behavior of applications that dont support
proxy - i.e.
     >dont forward requests to proxy?

    they mest be intercepted.

    --
    Matus UHLAR - fantomas, [hidden email]
[hidden email] ; http://www.fantomas.sk/
    Warning: I wish NOT to receive e-mail advertising to this address.
    Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
    Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
    _______________________________________________
    squid-users mailing list
    [hidden email]
[hidden email]
    http://lists.squid-cache.org/listinfo/squid-users



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Aaron Turner
In reply to this post by Yuri Voinov
Thanks Yuri.  That helps.  As for the "sslproxy_flags
DONT_VERIFY_PEER", yes I understand the risks.  In my specific case,
where my "users" are actually a bunch of automated web clients doing
some web crawling it's the right thing to do.
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Tue, Mar 13, 2018 at 10:44 AM, Yuri <[hidden email]> wrote:

> AFAIK,
>
> SSL bump subsystem uses OpenSSL memory routines. So, first of all, most
> probably leaks (if any) can be OpenSSL-related, but not squid itself.
>
> Now let's see your config snippets.
>
> 13.03.2018 23:00, Aaron Turner пишет:
>
> "Usually misconfiguration leads to memory overhead."
>
> This may be true, but if you look in the list archives a few months
> ago I basically chased my tail in circles and nobody could tell me
> what I was doing wrong and so many of the docs are so old that they're
> worse then useless, they seem to suggest the wrong thing.
>
> It was literally leaking GB's worth of RAM.  I even disabled all
> caching and process sizes were growing into the GB's.  Turn off
> ssl-bump and the leak went away.
>
> This is what I was using:
>
> http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
> sslflags=NO_DEFAULT_CA
> http_port localhost:3128
> ssl_bump bump all
>
> bump all is useless without peek/splice.
>
> Let's see on my config snippets:
>
> https_port 3127 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> http_port 3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> # Cert database on ramdisk
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /ramdisk1/ssl_db -M 1GB
> sslcrtd_children 32 startup=10 idle=5
>
> # SSL bump rules
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/acl.url.nobump"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
>
>
> sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> sslcrtd_children 32 startup=2 idle=2
>
> This is defaults. Pay attention, -M is limits use ssl_db directory to 4 Mb
> in size. It's too few for production servers. My ramdisk for ssl db is 1+ Gb
> in size:
>
> /dev/ramdisk/ramdisk1           961M   14M  890M   2% /ramdisk1/ssl_db
>
> sslproxy_session_cache_size 100 MB
>
> This is disbalanced size instead of previous setting. Why so big?
>
> #  TAG: sslproxy_session_cache_size
> #        Sets the cache size to use for ssl session
> #Default:
> # sslproxy_session_cache_size 2 MB
>
> sslproxy_cert_error allow all
> sslproxy_flags DONT_VERIFY_PEER
>
> NEVER use this options. It is unsafe.
>
> SSL Bump is dangerous enough itself. Don't do it more unsafe additionally by
> yourself.
>
> This was on a machine (EC2 VM) with 14GB of RAM.
>
> Pay attention on several places:
>
> 1. OS memory allocator.
> 2. OpenSSL version.
> 3. OS configuration (IPC, shared memory, swap - all memory related).
> 4. Squid's memory/pools configuration.
>
> Don't forget about: Often memory fragmentation seems like leaks. But no
> leaks occurs indeed.
>
> Also, don't forget - squid's memory consumption is not only cache_mem, but
> also caching on-disk metadata (swap.state), pools settings, working memory
> areas, processes memory. And - also - such things like content adaptation
> (did you know wide uses ecap gzip adapter is leaky itself?).
>
> But this is just for example.
>
> In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.
>
> I know, this appears SSL Bump is leaky. But this is not correct.
>
> --
> Aaron Turner
> https://synfin.net/         Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality.  "Something cannot emerge from nothing,"
> he said.  This is profound thinking if you understand how unstable
> "the truth" can be.  -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:47 AM, Yuri <[hidden email]> wrote:
>
> I've used it on all versions starting from 3.4.
>
> Now I'm using Squid 5.0.0.
>
> I'm afraid, my config is completely useless, because of it contains tons
> of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
> with different memory allocator for custom infrastructure.
>
> You can't just take my config, implement it and hope it will give same
> results for you.
>
> At least, it uses non-system CA bundle, platform-specific configuration
> parameters combinations, etc.
>
> I can say, than SSL Bump is not directly related to memory leaks. Squid
> itself almost not contains memory leaks now. Usually misconfiguration
> leads to memory overhead.
>
> As a recommendation, I can give some advices.
>
> 1. Use server with enough RAM. 4 Gb usually enough just for default
> squid configuration. Usually whole system RAM usage should never be
> bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
> should always be free during normal running. This prevents OS allocator
> pressure to your proxy and, also, increasing performance of proxy). In
> case of medium proxy server 16 Gb of RAM seems big enough, but never try
> to fill it up completely.
>
> 2. Don't set giant cache_mem. Remember how you platform allocates whole
> RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
> memory-related settings.
>
> 3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.
>
> 4. Never remember - SSL Bump increases your cache memory pressure due to
> increasing caching. So, you still require to have enough memory in your
> system.
>
>
> 13.03.2018 22:25, Aaron Turner пишет:
>
> What version are you using Yuri?  Can you share your config?
> Everytime I use ssl bump, I have massive memory leaks.  It's been
> effectively unusable for me.
> --
> Aaron Turner
> https://synfin.net/         Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality.  "Something cannot emerge from nothing,"
> he said.  This is profound thinking if you understand how unstable
> "the truth" can be.  -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <[hidden email]> wrote:
>
> Moreover,
>
> SSL Bump combines with interception/explicit proxy in one setup.
>
> And works perfectly.
>
>
> 13.03.2018 21:14, Marcus Kool пишет:
>
> "SSL bump" is the name of a complex Squid feature.
> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
> through the proxy untouched) or can be 'bumped' (decrypted).
>
> Interception is not a requirement for SSL bump.
>
> Marcus
>
> On 13/03/18 11:44, Danilo V wrote:
>
> I mean SSL bump in explicit mode.
> So intercept is a essencial requirement for running SSL bump?
>
> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
> <[hidden email] <mailto:[hidden email]>> escreveu:
>
>     On 13.03.18 13:44, Danilo V wrote:
>      >Is it possible/feasible to configure squid in explicit mode
> with ssl
>      >intercept?
>
>     explicit is not intercept, intercept is not explicit.
>
>     explicit is where browser is configured (manually or
> automatically via WPAD)
>     to use the proxy.
>
>     intercept is where network device forcifully redirects http/https
> connections
>     to the proxy.
>
>     maybe you mean SSL bump in explicit mode?
>
>      >Due to architecture of my network it is not possible to implement
>      >transparent proxy.
>
>     excuse me?
>     by "transparent" people mean what we usually call "intercept".
>
>      >What would be the behavior of applications that dont support
> proxy - i.e.
>      >dont forward requests to proxy?
>
>     they mest be intercepted.
>
>     --
>     Matus UHLAR - fantomas, [hidden email]
> <mailto:[hidden email]> ; http://www.fantomas.sk/
>     Warning: I wish NOT to receive e-mail advertising to this address.
>     Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>     Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>     _______________________________________________
>     squid-users mailing list
>     [hidden email]
> <mailto:[hidden email]>
>     http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Alex Rousskov
In reply to this post by Yuri Voinov
Yuri,

    The quality of many of your recent mailing list posts was
exceptionally high: to-the-point, with a healthy level of technical
detail, cool triage, actionable advice, and no distractions (up to the
footer:-). Your new approach resulted in a much more enjoyable
experience for me personally and, I bet, for many other list readers.
Thank you and please keep it up!

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov
As practical experience shows, it is counterproductive to swear. :) Especially when you need to solve the problem;)

It's just that sometimes a bad character wins :)

14.03.2018 03:30, Alex Rousskov пишет:
Yuri,

    The quality of many of your recent mailing list posts was
exceptionally high: to-the-point, with a healthy level of technical
detail, cool triage, actionable advice, and no distractions (up to the
footer:-). Your new approach resulted in a much more enjoyable
experience for me personally and, I bet, for many other list readers.
Thank you and please keep it up!

Alex.


-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Eliezer Croitoru
In reply to this post by Yuri Voinov
Thank Yuri!!

I believe that this post is milestone in for the SSL-BUMP feature.
Now the only thing left regarding weird memory leaks is to compare with these technical details:
3.5.27
4.0.24
5.0.0_alpha\head

I cannot test and compare it myself due to the lack of time and CPU but I believe that it will help to clear some doubts about stability of the above versions.

All The Bests and Thanks,
Eliezer

----
http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


From: squid-users [mailto:[hidden email]] On Behalf Of Yuri
Sent: Tuesday, March 13, 2018 19:45
To: Aaron Turner <[hidden email]>
Cc: [hidden email]
Subject: Re: [squid-users] SSL intercept in explicit mode

AFAIK,
SSL bump subsystem uses OpenSSL memory routines. So, first of all, most probably leaks (if any) can be OpenSSL-related, but not squid itself.
Now let's see your config snippets.
13.03.2018 23:00, Aaron Turner пишет:
"Usually misconfiguration leads to memory overhead."

This may be true, but if you look in the list archives a few months
ago I basically chased my tail in circles and nobody could tell me
what I was doing wrong and so many of the docs are so old that they're
worse then useless, they seem to suggest the wrong thing.

It was literally leaking GB's worth of RAM.  I even disabled all
caching and process sizes were growing into the GB's.  Turn off
ssl-bump and the leak went away.

This is what I was using:
<SNIP>
 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Matus UHLAR - fantomas
In reply to this post by Danilo V
On 13.03.18 14:44, Danilo V wrote:
>I mean SSL bump in explicit mode.
>So intercept is a essencial requirement for running SSL bump?

No, you asked for "explicit mode with ssl intercept" which I pointed out is
illogical.


>Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas <
>[hidden email]> escreveu:
>> On 13.03.18 13:44, Danilo V wrote:
>> >Is it possible/feasible to configure squid in explicit mode with ssl
>> >intercept?
>>
>> maybe you mean SSL bump in explicit mode?

It is possible to bump explicit proxy.

>> >Due to architecture of my network it is not possible to implement
>> >transparent proxy.
>>
>> excuse me?
>> by "transparent" people mean what we usually call "intercept".

>> >What would be the behavior of applications that dont support proxy - i.e.
>> >dont forward requests to proxy?
>>
>> they mest be intercepted.

"must" be intercepted. Since you said that it's not possible transparent (I
believe you have meant intercepting) proxy, it's apparently not possible to
handle applications that do not support proxy.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Danilo V
Thanks for the explanation.
Do you have any guide?

Em qua, 14 de mar de 2018 às 10:26, Matus UHLAR - fantomas <[hidden email]> escreveu:
On 13.03.18 14:44, Danilo V wrote:
>I mean SSL bump in explicit mode.
>So intercept is a essencial requirement for running SSL bump?

No, you asked for "explicit mode with ssl intercept" which I pointed out is
illogical.


>Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas <
>[hidden email]> escreveu:
>> On 13.03.18 13:44, Danilo V wrote:
>> >Is it possible/feasible to configure squid in explicit mode with ssl
>> >intercept?
>>
>> maybe you mean SSL bump in explicit mode?

It is possible to bump explicit proxy.

>> >Due to architecture of my network it is not possible to implement
>> >transparent proxy.
>>
>> excuse me?
>> by "transparent" people mean what we usually call "intercept".

>> >What would be the behavior of applications that dont support proxy - i.e.
>> >dont forward requests to proxy?
>>
>> they mest be intercepted.

"must" be intercepted. Since you said that it's not possible transparent (I
believe you have meant intercepting) proxy, it's apparently not possible to
handle applications that do not support proxy.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Yuri Voinov

I guess, your using wrong approach.

You trying to find ready-to-use solution for custom configuration.

At maximum, you can find some bricks for this. And anyway you should build your custom solution yourself.

Bricks is here: https://wiki.squid-cache.org :-)

14.03.2018 20:28, Danilo V пишет:
Thanks for the explanation.
Do you have any guide?

Em qua, 14 de mar de 2018 às 10:26, Matus UHLAR - fantomas <[hidden email]> escreveu:
On 13.03.18 14:44, Danilo V wrote:
>I mean SSL bump in explicit mode.
>So intercept is a essencial requirement for running SSL bump?

No, you asked for "explicit mode with ssl intercept" which I pointed out is
illogical.


>Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas <
>[hidden email]> escreveu:
>> On 13.03.18 13:44, Danilo V wrote:
>> >Is it possible/feasible to configure squid in explicit mode with ssl
>> >intercept?
>>
>> maybe you mean SSL bump in explicit mode?

It is possible to bump explicit proxy.

>> >Due to architecture of my network it is not possible to implement
>> >transparent proxy.
>>
>> excuse me?
>> by "transparent" people mean what we usually call "intercept".

>> >What would be the behavior of applications that dont support proxy - i.e.
>> >dont forward requests to proxy?
>>
>> they mest be intercepted.

"must" be intercepted. Since you said that it's not possible transparent (I
believe you have meant intercepting) proxy, it's apparently not possible to
handle applications that do not support proxy.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

-- 
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Eliezer Croitoru
In reply to this post by Danilo V

Hey Danilo,

 

I have tried to understand the issue and scenario from 0 but now I’m now sure I understood it.

What have achieved until now in your setup?

Any network can be “simplified” in order to understand on what you do have control and what you do not.

From your words:

“applications that dont support proxy - i.e. dont forward requests to proxy?”

 

I understand that you are talking about some kind of client such as a browser or a other software.

Can you be more specific?

 

From the older posts I understand it might involve ssl-bump but I am missing some details on the clients.

 

Please provide more details on your environment so we can somehow make a summery for your use case.

 

Thanks,

Eliezer

----

Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]

 

From: squid-users <[hidden email]> On Behalf Of Danilo V
Sent: Tuesday, March 13, 2018 15:45
To: [hidden email]
Subject: [squid-users] SSL intercept in explicit mode

 

Is it possible/feasible to configure squid in explicit mode with ssl intercept?

Due to architecture of my network it is not possible to implement transparent proxy.

What would be the behavior of applications that dont support proxy - i.e. dont forward requests to proxy?

Any guides?

 

Danilo


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

MK2018
In reply to this post by Aaron Turner
Aaron Turner wrote
> Thanks Yuri.  That helps.  As for the "sslproxy_flags
> DONT_VERIFY_PEER", yes I understand the risks.  In my specific case,
> where my "users" are actually a bunch of automated web clients doing
> some web crawling it's the right thing to do.
> --
> Aaron Turner

I tried using bump all myself with actual human beings (200+) using browsers
ranging from Mozilla Firefox, Seamonkey, Chrome, to Safari and Opera.

I don't know why I had to face it, but with bump all I got many errors with
many websites. It only worked with me like this:

http_port 3128 ssl-bump cert=/ssl_cert/myCA.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=999MB
sslcrtd_children 100
ssl_bump none BadSSL
ssl_bump server-first all

Like you see, I'm using server-first word in place of bump word. This is the
only way I got it to work with natural human browsing. I also could not use
intercept mode, because every major browser considers it a crime to let it
go! They would just spit all sorts of errors at user's face and have you
clean the spitting up :D :D

Of course, BadSSL above is the ACL for all sites using the new fiasco of
hardcoded certificates (certificate-pinning), otherwise, they don't pass at
all!




--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL intercept in explicit mode

Amos Jeffries
Administrator
On 14/04/18 10:05, MK2018 wrote:

> Aaron Turner wrote
>> Thanks Yuri.  That helps.  As for the "sslproxy_flags
>> DONT_VERIFY_PEER", yes I understand the risks.  In my specific case,
>> where my "users" are actually a bunch of automated web clients doing
>> some web crawling it's the right thing to do.
>> --
>> Aaron Turner
>
> I tried using bump all myself with actual human beings (200+) using browsers
> ranging from Mozilla Firefox, Seamonkey, Chrome, to Safari and Opera.
>
> I don't know why I had to face it, but with bump all I got many errors with
> many websites. It only worked with me like this:
>
> http_port 3128 ssl-bump cert=/ssl_cert/myCA.pem
> generate-host-certificates=on dynamic_cert_mem_cache_size=999MB
> sslcrtd_children 100
> ssl_bump none BadSSL
> ssl_bump server-first all
>

FYI this is "server-first all". peek and splice before "bump all" is
similar but also different in ways that allow it to handle more problems
in better ways.


> Like you see, I'm using server-first word in place of bump word. This is the
> only way I got it to work with natural human browsing. I also could not use
> intercept mode, because every major browser considers it a crime to let it
> go! They would just spit all sorts of errors at user's face and have you
> clean the spitting up :D :D

You do need the browser to trust your CA certificate. This is an
absolute requirement of using SSL-Bump features. Always has been.

>
> Of course, BadSSL above is the ACL for all sites using the new fiasco of
> hardcoded certificates (certificate-pinning), otherwise, they don't pass at
> all!
>

Indeed, its quite sad situation really. Sites using actually secure TLS
have to downgrade to using the broken CA system for passing grades on
sites that test only the "TLS everywhere" groups over-hyped way of doing
things.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
12