Squid SSL db on ramdisk

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid SSL db on ramdisk

Yuri Voinov
Amos,

how do you think - if I'll put SSL db (usually places in
/var/lib/ssl_db) on ramdisk, does this give some gain for bump performance?

How reasonable to do that?

Also, I think, doing that,  I can reduce in memory cache size for
security_file_certgen helper.

How do you think?

--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Amos Jeffries
Administrator
On 10/02/18 12:55, Yuri wrote:
> Amos,
>
> how do you think - if I'll put SSL db (usually places in
> /var/lib/ssl_db) on ramdisk, does this give some gain for bump performance?
>

I expect so, but do not use bumping myself so cannot say for certain.

> How reasonable to do that?
>
> Also, I think, doing that,  I can reduce in memory cache size for
> security_file_certgen helper.
>
> How do you think?

I don't think it will have any effect on that. The size of the DB
content does not related to *where* it is stored.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov


10.02.2018 13:30, Amos Jeffries пишет:
> On 10/02/18 12:55, Yuri wrote:
>> Amos,
>>
>> how do you think - if I'll put SSL db (usually places in
>> /var/lib/ssl_db) on ramdisk, does this give some gain for bump performance?
>>
> I expect so, but do not use bumping myself so cannot say for certain.
Ok, will do tests.
>
>> How reasonable to do that?
>>
>> Also, I think, doing that,  I can reduce in memory cache size for
>> security_file_certgen helper.
>>
>> How do you think?
> I don't think it will have any effect on that. The size of the DB
> content does not related to *where* it is stored.
No-no. I mean, security_file_certgen uses memory cache to buffer slow
disk IO for certificates DB. If we're put cert DB onto ramdisk (in fact,
in RAM), so we're can easy reduce helper -M value. Correct?
>
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users

--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Alex Rousskov
On 02/10/2018 06:43 AM, Yuri wrote:

> security_file_certgen uses memory cache to buffer slow
> disk IO for certificates DB.

It does not.


> If we're put cert DB onto ramdisk (in fact,
> in RAM), so we're can easy reduce helper -M value.

security_file_certgen -M is the helper database size, not the
buffer/cache size.

The buffer/cache you might be thinking about is inside Squid, not inside
the helper. See dynamic_cert_mem_cache_size. dynamic_cert_mem_cache_size
is not related to -M.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov


10.02.2018 22:18, Alex Rousskov пишет:
> On 02/10/2018 06:43 AM, Yuri wrote:
>
>> security_file_certgen uses memory cache to buffer slow
>> disk IO for certificates DB.
> It does not.
Ahhhhhhaaaaaaa, I just misunderstanding options....
>
>
>> If we're put cert DB onto ramdisk (in fact,
>> in RAM), so we're can easy reduce helper -M value.
> security_file_certgen -M is the helper database size, not the
> buffer/cache size.
Ah. Got it. I.e., I can set -M in according FS size, using for store SSL
DB, correct?
>
> The buffer/cache you might be thinking about is inside Squid, not inside
> the helper. See dynamic_cert_mem_cache_size. dynamic_cert_mem_cache_size
> is not related to -M.
Tks, Alex. But wait,

dynamic_cert_mem_cache_size

is http(s)_port option?

>
>
> HTH,
>
> Alex.

--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Alex Rousskov
On 02/10/2018 09:23 AM, Yuri wrote:

> I can set -M in according FS size, using for store SSL DB, correct?

Yes, -M limits the sum of sizes of all (serialized) certificates stored
in the helper database. The helper tries to account for the filesystem
block size, but I doubt its calculations are very precise.


> dynamic_cert_mem_cache_size is http(s)_port option?

Yes, it is. If the needed dynamically-generated certificate is found in
the dynamic certificate memory cache, then Squid does not ask the helper
to generate that certificate. This in-Squid RAM cache stores raw (not
serialized) certificates. As you know, Squid does not compute the size
of raw (not serialized) certificates correctly, resulting in bug #4005
issues: https://bugs.squid-cache.org/show_bug.cgi?id=4005

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov


10.02.2018 22:36, Alex Rousskov пишет:
> On 02/10/2018 09:23 AM, Yuri wrote:
>
>> I can set -M in according FS size, using for store SSL DB, correct?
> Yes, -M limits the sum of sizes of all (serialized) certificates stored
> in the helper database. The helper tries to account for the filesystem
> block size, but I doubt its calculations are very precise.
Tks for clarifying :)
Got it. Will correct my configs :-)
>
>
>> dynamic_cert_mem_cache_size is http(s)_port option?
> Yes, it is. If the needed dynamically-generated certificate is found in
> the dynamic certificate memory cache, then Squid does not ask the helper
> to generate that certificate. This in-Squid RAM cache stores raw (not
> serialized) certificates. As you know, Squid does not compute the size
> of raw (not serialized) certificates correctly, resulting in bug #4005
> issues: https://bugs.squid-cache.org/show_bug.cgi?id=4005
Aha, and in this case helper speed is critical and using helper storage
on ramdisk will very useful....
>
> Alex.

--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov
One more question.

What is correct syntax for -M option? I'm just in doubt. Helper eats -M
5MB, but not -M 1024MB, however eats -M 1 GB.

root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 2GB
/usr/local/squid/libexec/security_file_certgen: Error when parsing -M
options value
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 2 GB
^C
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 2GB
/usr/local/squid/libexec/security_file_certgen: Error when parsing -M
options value
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 5MB
^C
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 5GB
/usr/local/squid/libexec/security_file_certgen: Error when parsing -M
options value
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 5 MB
^C
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M 1024MB
/usr/local/squid/libexec/security_file_certgen: Error when parsing -M
options value
root @ lemanruss /patch/tmp #
/usr/local/squid/libexec/security_file_certgen -s /ramdisk1/ssl_db -M
1024 MB
^C

How to correctly specify -M with 2 Gb size?


10.02.2018 22:39, Yuri пишет:

>
> 10.02.2018 22:36, Alex Rousskov пишет:
>> On 02/10/2018 09:23 AM, Yuri wrote:
>>
>>> I can set -M in according FS size, using for store SSL DB, correct?
>> Yes, -M limits the sum of sizes of all (serialized) certificates stored
>> in the helper database. The helper tries to account for the filesystem
>> block size, but I doubt its calculations are very precise.
> Tks for clarifying :)
> Got it. Will correct my configs :-)
>>
>>> dynamic_cert_mem_cache_size is http(s)_port option?
>> Yes, it is. If the needed dynamically-generated certificate is found in
>> the dynamic certificate memory cache, then Squid does not ask the helper
>> to generate that certificate. This in-Squid RAM cache stores raw (not
>> serialized) certificates. As you know, Squid does not compute the size
>> of raw (not serialized) certificates correctly, resulting in bug #4005
>> issues: https://bugs.squid-cache.org/show_bug.cgi?id=4005
> Aha, and in this case helper speed is critical and using helper storage
> on ramdisk will very useful....
>> Alex.
--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Alex Rousskov
On 02/10/2018 10:03 AM, Yuri wrote:

> What is correct syntax for -M option?

The correct syntax is, roughly,

  -M <integer>[bytes|KB|MB|GB]

with "bytes" as the default unit.

However, you found a bug in the parsing code: The helper mishandles
values exceeding 2147483647 bytes (on most platforms) due to a signed
integer overflow in helper's parseBytesOptionValue().

Furthermore, I have not tested it, but I suspect there is at least one
bug in the mainline parseBytesOptionValue() code as well. Both functions
should be rewritten (even if the second one "works"), and the correct
format should be documented (including size limits).


> How to correctly specify -M with 2 Gb size?

You cannot specify that size until the above-mentioned bug is fixed.

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov


11.02.2018 00:59, Alex Rousskov пишет:
> On 02/10/2018 10:03 AM, Yuri wrote:
>
>> What is correct syntax for -M option?
> The correct syntax is, roughly,
>
>   -M <integer>[bytes|KB|MB|GB]
Exactly with space between integer and units?

>
> with "bytes" as the default unit.
>
> However, you found a bug in the parsing code: The helper mishandles
> values exceeding 2147483647 bytes (on most platforms) due to a signed
> integer overflow in helper's parseBytesOptionValue().
>
> Furthermore, I have not tested it, but I suspect there is at least one
> bug in the mainline parseBytesOptionValue() code as well. Both functions
> should be rewritten (even if the second one "works"), and the correct
> format should be documented (including size limits).
>
>
>> How to correctly specify -M with 2 Gb size?
> You cannot specify that size until the above-mentioned bug is fixed.
>
> Alex.
--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Alex Rousskov
On 02/10/2018 12:02 PM, Yuri wrote:
>
>
> 11.02.2018 00:59, Alex Rousskov пишет:
>> On 02/10/2018 10:03 AM, Yuri wrote:
>>
>>> What is correct syntax for -M option?
>> The correct syntax is, roughly,
>>
>>   -M <integer>[bytes|KB|MB|GB]

> Exactly with space between integer and units?

Without anything between integer and units. For example: 2GB

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov
int m; declaration inside static bool parseBytesOptionValue(size_t *
bptr, char const * value) ?

If I set it long, as by as int d, seems ok.


11.02.2018 01:04, Alex Rousskov пишет:

> On 02/10/2018 12:02 PM, Yuri wrote:
>>
>> 11.02.2018 00:59, Alex Rousskov пишет:
>>> On 02/10/2018 10:03 AM, Yuri wrote:
>>>
>>>> What is correct syntax for -M option?
>>> The correct syntax is, roughly,
>>>
>>>   -M <integer>[bytes|KB|MB|GB]
>> Exactly with space between integer and units?
> Without anything between integer and units. For example: 2GB
>
> Alex.
--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov
Yes, confirmed.

When I've replaced int m; and int d; to long m; and long d; - works like
charm.


11.02.2018 01:08, Yuri пишет:

> int m; declaration inside static bool parseBytesOptionValue(size_t *
> bptr, char const * value) ?
>
> If I set it long, as by as int d, seems ok.
>
>
> 11.02.2018 01:04, Alex Rousskov пишет:
>> On 02/10/2018 12:02 PM, Yuri wrote:
>>> 11.02.2018 00:59, Alex Rousskov пишет:
>>>> On 02/10/2018 10:03 AM, Yuri wrote:
>>>>
>>>>> What is correct syntax for -M option?
>>>> The correct syntax is, roughly,
>>>>
>>>>   -M <integer>[bytes|KB|MB|GB]
>>> Exactly with space between integer and units?
>> Without anything between integer and units. For example: 2GB
>>
>> Alex.
--
*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Vacheslav
Works like a charm is a stubborn phrase, never experienced that when being charmed one problem is gone and replaced with numerous others, like sick relatives?

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Yuri
Sent: Saturday, February 10, 2018 10:57 PM
To: Alex Rousskov <[hidden email]>; [hidden email]
Subject: Re: [squid-users] Squid SSL db on ramdisk

Yes, confirmed.

When I've replaced int m; and int d; to long m; and long d; - works like charm.


11.02.2018 01:08, Yuri пишет:

> int m; declaration inside static bool parseBytesOptionValue(size_t *
> bptr, char const * value) ?
>
> If I set it long, as by as int d, seems ok.
>
>
> 11.02.2018 01:04, Alex Rousskov пишет:
>> On 02/10/2018 12:02 PM, Yuri wrote:
>>> 11.02.2018 00:59, Alex Rousskov пишет:
>>>> On 02/10/2018 10:03 AM, Yuri wrote:
>>>>
>>>>> What is correct syntax for -M option?
>>>> The correct syntax is, roughly,
>>>>
>>>>   -M <integer>[bytes|KB|MB|GB]
>>> Exactly with space between integer and units?
>> Without anything between integer and units. For example: 2GB
>>
>> Alex.

--
*****************************
* C++20 : Bug to the future *
*****************************




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid SSL db on ramdisk

Yuri Voinov

If there is nothing to say on the topic - it's better to keep quiet.

I'm not talking with you. And when I need your opinion - I'll call you.


12.02.2018 14:15, Vacheslav пишет:
Works like a charm is a stubborn phrase, never experienced that when being charmed one problem is gone and replaced with numerous others, like sick relatives?

-----Original Message-----
From: squid-users [[hidden email]] On Behalf Of Yuri
Sent: Saturday, February 10, 2018 10:57 PM
To: Alex Rousskov [hidden email]; [hidden email]
Subject: Re: [squid-users] Squid SSL db on ramdisk

Yes, confirmed.

When I've replaced int m; and int d; to long m; and long d; - works like charm.


11.02.2018 01:08, Yuri пишет:
int m; declaration inside static bool parseBytesOptionValue(size_t * 
bptr, char const * value) ?

If I set it long, as by as int d, seems ok.


11.02.2018 01:04, Alex Rousskov пишет:
On 02/10/2018 12:02 PM, Yuri wrote:
11.02.2018 00:59, Alex Rousskov пишет:
On 02/10/2018 10:03 AM, Yuri wrote:

What is correct syntax for -M option?
The correct syntax is, roughly,

  -M <integer>[bytes|KB|MB|GB]
Exactly with space between integer and units?
Without anything between integer and units. For example: 2GB

Alex.
--
*****************************
* C++20 : Bug to the future *
*****************************





-- 
*****************************
* C++20 : Bug to the future *
*****************************

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment