Forcing squid to fail when the whitelist doesn't exist

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Forcing squid to fail when the whitelist doesn't exist

Matthew Macdonald-Wallace
Hey all,

We're re-configuring a squid proxy solution for a client and as part of it we made the assumption that squid would fail if we asked it to read a whitelist that wasn't present.

We've now discovered that Squid fails to read the file, throws an error in the log ( Error: Cannot open file /etc/squid/whitelist.txt for reading ), and then starts up anyway and listens on port 3128 but without the whitelist present.

I've also discovered the "-C" flag that helps us ignore even more serious issues, however I can't find anything either in the documentation or by searching that shows us how to make squid stop as soon as it encounters an error.

Is this possible? I've searched the FAQ and various other sources, but of course "stop squid from starting when error" or similar just returns a load of results about how to fix various errors that stop squid from starting, rather than deliberately wanting Squid to fail.

Thanks in advance,

Matt


--
Matthew Macdonald-Wallace MIET
Co-Founder
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438


--
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Alex Rousskov
On 7/7/20 10:52 AM, Matthew Macdonald-Wallace wrote:

> We're re-configuring a squid proxy solution for a client and as part of
> it we made the assumption that squid would fail if we asked it to read a
> whitelist that wasn't present.
>
> We've now discovered that Squid fails to read the file, throws an error
> in the log ( Error: Cannot open file /etc/squid/whitelist.txt for
> reading ), and then starts up anyway

Yes, this kind of error ignorance is an old known Squid problem. Some
developers have thought that it is better to start Squid "if at all
possible" than to fail on (in their view "minor") error. New features
are usually more "conservative", but even now that "conservative"
approach does not always win.

IMO, quality pull requests making missing files a fatal configuration
error should be welcomed. They may not be backported to stable versions,
of course. The solution would probably revolve around throwing an
exception in ConfigParser::strtokFile(). Making missing file treatment
configurable, especially on a per-file basis should be welcomed as well,
probably by extending the new parameters syntax mentioned below.

Meanwhile, try using the newer parameters() syntax instead of abusing
double quotes. It should work the way you expect. Here is the
corresponding quote from squid.conf.documented:

> Squid supports reading configuration option parameters from external
> files using the syntax:
>     parameters("/path/filename")
> For example:
>     acl whitelist dstdomain parameters("/etc/squid/whitelist.txt")


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Matthew Macdonald-Wallace



On Tue, 7 Jul 2020 at 18:53, Alex Rousskov <[hidden email]> wrote:
On 7/7/20 10:52 AM, Matthew Macdonald-Wallace wrote:

> We're re-configuring a squid proxy solution for a client and as part of
> it we made the assumption that squid would fail if we asked it to read a
> whitelist that wasn't present.
>
> We've now discovered that Squid fails to read the file, throws an error
> in the log ( Error: Cannot open file /etc/squid/whitelist.txt for
> reading ), and then starts up anyway

Yes, this kind of error ignorance is an old known Squid problem. Some
developers have thought that it is better to start Squid "if at all
possible" than to fail on (in their view "minor") error. New features
are usually more "conservative", but even now that "conservative"
approach does not always win.

IMO, quality pull requests making missing files a fatal configuration
error should be welcomed. They may not be backported to stable versions,
of course. The solution would probably revolve around throwing an
exception in ConfigParser::strtokFile(). Making missing file treatment
configurable, especially on a per-file basis should be welcomed as well,
probably by extending the new parameters syntax mentioned below.


Thanks, it did seem a bit odd as a default behaviour, good to know something like this would be welcomed (by you at least!).
 
Meanwhile, try using the newer parameters() syntax instead of abusing
double quotes. It should work the way you expect. Here is the
corresponding quote from squid.conf.documented:

> Squid supports reading configuration option parameters from external
> files using the syntax:
>     parameters("/path/filename")
> For example:
>     acl whitelist dstdomain parameters("/etc/squid/whitelist.txt")


I'll check the version that we're running and see if I can do this.  I suspect that due to "enterprise requirements" our version won't be the latest, but hopefully it will support this.

Thanks again,

Matt 


--
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Alex Rousskov
On 7/7/20 1:57 PM, Matthew Macdonald-Wallace wrote:
> On Tue, 7 Jul 2020 at 18:53, Alex Rousskov
>     Meanwhile, try using the newer parameters() syntax instead of abusing
>     double quotes. It should work the way you expect. Here is the
>     corresponding quote from squid.conf.documented:

>     > Squid supports reading configuration option parameters from external
>     > files using the syntax:
>     >     parameters("/path/filename")
>     > For example:
>     >     acl whitelist dstdomain parameters("/etc/squid/whitelist.txt")


> I'll check the version that we're running and see if I can do this.  I
> suspect that due to "enterprise requirements" our version won't be the
> latest, but hopefully it will support this.

AFAICT, all supported Squid versions have parameters(). You will need to
enable configuration_includes_quoted_values in squid.conf to get that
feature. IIRC, that directive was supposed to be on by default, but that
cannot happen until we fix regex support (at least).

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Matthew Macdonald-Wallace

> I'll check the version that we're running and see if I can do this.  I
> suspect that due to "enterprise requirements" our version won't be the
> latest, but hopefully it will support this.

AFAICT, all supported Squid versions have parameters(). You will need to
enable configuration_includes_quoted_values in squid.conf to get that
feature. IIRC, that directive was supposed to be on by default, but that
cannot happen until we fix regex support (at least).

Alex.

Turns out we're running 3.5.x - I see from the wiki that this is a deprecated release as of two years ago, but is still the most recent release from rhel.

The parameters trick above doesn't seem to be working as expected, I'm wondering if we need to be on v4 to get this behaviour?

Cheers,

Matt


--
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Alex Rousskov
On 7/9/20 2:25 AM, Matthew Macdonald-Wallace wrote:
>
>     > I'll check the version that we're running and see if I can do this.  I
>     > suspect that due to "enterprise requirements" our version won't be the
>     > latest, but hopefully it will support this.
>
>     AFAICT, all supported Squid versions have parameters(). You will need to
>     enable configuration_includes_quoted_values in squid.conf to get that
>     feature. IIRC, that directive was supposed to be on by default, but that
>     cannot happen until we fix regex support (at least).

> Turns out we're running 3.5.x

I have not tested this, but if my quick reading of the latest v3.5 code
is correct, then the missing parameters() file is treated as a FATAL
configuration error (in ConfigParser::NextToken). How do you use
parameters()? What happens when you use parameters() with an existing
file? A missing file?

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Matthew Macdonald-Wallace


On Thu, 9 Jul 2020 at 16:30, Alex Rousskov <[hidden email]> wrote:
On 7/9/20 2:25 AM, Matthew Macdonald-Wallace wrote:
>
>     > I'll check the version that we're running and see if I can do this.  I
>     > suspect that due to "enterprise requirements" our version won't be the
>     > latest, but hopefully it will support this.
>
>     AFAICT, all supported Squid versions have parameters(). You will need to
>     enable configuration_includes_quoted_values in squid.conf to get that
>     feature. IIRC, that directive was supposed to be on by default, but that
>     cannot happen until we fix regex support (at least).

> Turns out we're running 3.5.x

I have not tested this, but if my quick reading of the latest v3.5 code
is correct, then the missing parameters() file is treated as a FATAL
configuration error (in ConfigParser::NextToken). How do you use
parameters()? What happens when you use parameters() with an existing
file? A missing file?

For some reason, the behaviour is the same (it starts regardless).

For now, I've added a conditional into the SystemD service file that checks for the whitelist and only starts if it is present, and that works.

I've also logged a ticket in our internal tracker to look into upgrading to v4 of squid, so for now it's working even if it's not the prettiest behaviour!

Thanks for all your help :)

Matt


--
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Alex Rousskov
On 7/9/20 11:51 AM, Matthew Macdonald-Wallace wrote:
> > Turns out we're running 3.5.x
>
> I have not tested this, but if my quick reading of the latest v3.5 code
> is correct, then the missing parameters() file is treated as a FATAL
> configuration error (in ConfigParser::NextToken).


FWIW, Squid v3.5.26 works as expected in my test:

> configuration_includes_quoted_values on
> acl goodGuys ssl::server_name parameters("/missing/goodGuys.acl")
> configuration_includes_quoted_values off


> 2020/07/09 14:15:17| WARNING: file :/missing/goodGuys.acl not found
> 2020/07/09 14:15:17| FATAL: Error opening config file: parameters
> FATAL: Bungled /usr/local/squid/./etc/squid-v3p5.conf line 60: acl goodGuys ssl::server_name parameters("/missing/goodGuys.acl")
> Squid Cache (Version 3.5.26-BZR): Terminated abnormally.

Please note that configuration_includes_quoted_values is required.
Without it, Squid interprets the string
`parameters("/missing/goodGuys.acl")` as a single domain name.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Forcing squid to fail when the whitelist doesn't exist

Matthew Macdonald-Wallace
Awesome, thanks! 

On Thu, 9 Jul 2020, 19:25 Alex Rousskov, <[hidden email]> wrote:
On 7/9/20 11:51 AM, Matthew Macdonald-Wallace wrote:
> > Turns out we're running 3.5.x
>
> I have not tested this, but if my quick reading of the latest v3.5 code
> is correct, then the missing parameters() file is treated as a FATAL
> configuration error (in ConfigParser::NextToken).


FWIW, Squid v3.5.26 works as expected in my test:

> configuration_includes_quoted_values on
> acl goodGuys ssl::server_name parameters("/missing/goodGuys.acl")
> configuration_includes_quoted_values off


> 2020/07/09 14:15:17| WARNING: file :/missing/goodGuys.acl not found
> 2020/07/09 14:15:17| FATAL: Error opening config file: parameters
> FATAL: Bungled /usr/local/squid/./etc/squid-v3p5.conf line 60: acl goodGuys ssl::server_name parameters("/missing/goodGuys.acl")
> Squid Cache (Version 3.5.26-BZR): Terminated abnormally.

Please note that configuration_includes_quoted_values is required.
Without it, Squid interprets the string
`parameters("/missing/goodGuys.acl")` as a single domain name.


HTH,

Alex.


--
Mockingbird Consulting
Connecting you with your environment

w: www.mockingbirdconsulting.co.uk
e: [hidden email]
t: +44 (0) 1600 717142

Bridges Centre,
Drybridge House,
Monmouth,
NP25 5AS

Registered in England and Wales, Company Number 10488438

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users