Sqlite3 with Squid

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Sqlite3 with Squid

Eliezer Croitoru-3

Hey,

 

I am wondering what can I use Sqlite3 with squid?

I was thinking about holding some of the config dynamic parts inside sqlite db (in a specific setup)

And then generate the config file from sqlite.

 

What do you think?

 

Thanks,

Eliezer

 

----

Eliezer Croitoru

Tech Support

Mobile: +972-5-28704261

Email: [hidden email]

 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Antony Stone
On Thursday 10 December 2020 at 12:49:48, Eliezer Croitor wrote:

> Hey,
>
> I am wondering what can I use Sqlite3 with squid?
>
> I was thinking about holding some of the config dynamic parts inside sqlite
> db (in a specific setup)

Can you give some examples of such "config dynamic parts"?

> And then generate the config file from sqlite.
>
> What do you think?

I'm not sure I can see what I might want to be dynamic about a Squid
configuration.

However, you also say "generate the config file from sqlite".

Any reasonable scripting language can do that for you, and then just tell
Squid to reload the new config file.  Squid doesn't need to know where it came
from.


Regards,


Antony.

--
"Remember: the S in IoT stands for Security."

 - Jan-Piet Mens

                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Eliezer Croitoru-3
Thanks Antony,

From what I have seen in the past many embedded devices like Android or iPhone or even Firewalls use Sqlite.
It's good for Firmwares..
Let say I am a vendor of a proxy and I want to be able to pull single config file from one device to the other.
Let say for a testing lab, ie I have a specific test case and I need the to test the proxy\fw\firmware.
If I wrote a perl script it would probably work for then next 20++ years.
It's possible to take a proxy firmware and based on the Sqlite re-produce a very similar setup on different hardware.
We can call this similar pretty exact assuming the hardware and the OS are taking care of the relevant lower levels.

Indeed squid doesn't need to know where it came from, however the input can vary from device to device and environment.
Technically speaking we might be able to script with some help couple of these things.
For example Localnet, Manager, Ports, SSL_SAFE Ports, Certificate creation and Input validation scripts.
I believe that these days some things in squid are pretty static.
For example there were some integer overflow that was detected in the past for something.
Once the input validation is done in another separate process we can separate the logic into an external acl helper and ICAP.
The only issue now is the "cache" cleanup cycle.

Amos or Alex might remember or know how to trigger external_acl helper cache cleanup.
I don't know what it might affect since there is some context code per request or connection or session.

Can someone help me only to grasp this concept?

Thanks,
Eliezer

----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: [hidden email]

-----Original Message-----
From: squid-users <[hidden email]> On Behalf Of Antony Stone
Sent: Thursday, December 10, 2020 1:55 PM
To: [hidden email]
Subject: Re: [squid-users] Sqlite3 with Squid

On Thursday 10 December 2020 at 12:49:48, Eliezer Croitor wrote:

> Hey,
>
> I am wondering what can I use Sqlite3 with squid?
>
> I was thinking about holding some of the config dynamic parts inside sqlite
> db (in a specific setup)

Can you give some examples of such "config dynamic parts"?

> And then generate the config file from sqlite.
>
> What do you think?

I'm not sure I can see what I might want to be dynamic about a Squid
configuration.

However, you also say "generate the config file from sqlite".

Any reasonable scripting language can do that for you, and then just tell
Squid to reload the new config file.  Squid doesn't need to know where it came
from.


Regards,


Antony.

--
"Remember: the S in IoT stands for Security."

 - Jan-Piet Mens

                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Amos Jeffries
Administrator
On 11/12/20 12:03 pm, Eliezer Croitor wrote:
> Amos or Alex might remember or know how to trigger external_acl helper cache cleanup.
> I don't know what it might affect since there is some context code per request or connection or session.
>

"squid -k reconfigure" is the best trigger I know of.


> Can someone help me only to grasp this concept?

Like Antony I am not quite understanding what the concept you are trying
to describe is. It sounds a lot like what tools like Puppet do for
network wide management.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Eliezer Croitoru-3
Well indeed it's very similar.
I would need to think about it a bit more to grasp it again in my mind.
However in the embedded world ruby/perl/python are not usually available so..

----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: [hidden email]

-----Original Message-----
From: squid-users <[hidden email]> On Behalf Of Amos Jeffries
Sent: Saturday, December 12, 2020 11:56 AM
To: [hidden email]
Subject: Re: [squid-users] Sqlite3 with Squid

On 11/12/20 12:03 pm, Eliezer Croitor wrote:
> Amos or Alex might remember or know how to trigger external_acl helper cache cleanup.
> I don't know what it might affect since there is some context code per request or connection or session.
>

"squid -k reconfigure" is the best trigger I know of.


> Can someone help me only to grasp this concept?

Like Antony I am not quite understanding what the concept you are trying
to describe is. It sounds a lot like what tools like Puppet do for
network wide management.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Amos Jeffries
Administrator
On 13/12/20 11:01 am, Eliezer Croitor wrote:
> Well indeed it's very similar.
> I would need to think about it a bit more to grasp it again in my mind.
> However in the embedded world ruby/perl/python are not usually available so..
>

True. Though for limited devices you can do the same thing they do to
transfer a pre-built squid.conf (and included files) with scp or just
telnet.

My understanding is that tools using sqlite like you describe are also
using it internally for config and/or data storage anyway. Squid is not,
so it is also in the category of "not usually available" for us.

Long-term we have a wishlist entry to make the cachemgr API accept POST
requests with config lines to add/replace to its running configuration.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Sqlite3 with Squid

Eliezer Croitoru-3
I am not sure I understood but if I understand right, then probably it's better to know add config using POST.
It's better to write some dynamic ACL external helper that  can read config files with ttl of 10-20 seconds.
This will give the overall experience a GOOD ENOUGH.

So the squid is not directly accessible?

Thanks!

----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: [hidden email]

-----Original Message-----
From: squid-users <[hidden email]> On Behalf Of Amos Jeffries
Sent: Sunday, December 13, 2020 12:28 PM
To: [hidden email]
Subject: Re: [squid-users] Sqlite3 with Squid

On 13/12/20 11:01 am, Eliezer Croitor wrote:
> Well indeed it's very similar.
> I would need to think about it a bit more to grasp it again in my mind.
> However in the embedded world ruby/perl/python are not usually available so..
>

True. Though for limited devices you can do the same thing they do to
transfer a pre-built squid.conf (and included files) with scp or just
telnet.

My understanding is that tools using sqlite like you describe are also
using it internally for config and/or data storage anyway. Squid is not,
so it is also in the category of "not usually available" for us.

Long-term we have a wishlist entry to make the cachemgr API accept POST
requests with config lines to add/replace to its running configuration.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users