Squid logging to UDP logs multiple lines at the same time

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid logging to UDP logs multiple lines at the same time

Matt Pson
Hello,
I have set up several Squid servers running 3.5.27 on Ubuntu 16.04 LTS (behind keepalived and haproxy to create a ha/load balanced setup) and having are some problems with logging. In order to have a single logfile I decided to send the log directly using UDP to a separate server running rsyslogd.

Each Squid is configured with:

access_log     udp://squid-logserver.domain.com:10514

The rsyslogd server is configured with:

Module (load="imudp")
# rsyslog Input Modules
input(type="imudp" port="10514" ruleset="Squid")
# Squid RulesSet
# make all squid input from the frontends end up here
ruleset(name="Squid") {
        action(type="omfile"
                file="/var/log/squid/access.log")
        }

This works quite well and logs are sent to the server which receives them and puts them in the log file as expected. But the logs are not sent line by line but instead multiple lines at the same time (the size is approx. a UDP packet of lines, ~1300 bytes) which makes the log file quite messed up.

What am I missing here?
I want my log to be readable ;)

/Matt Pson
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid logging to UDP logs multiple lines at the same time

Amos Jeffries
Administrator
On 04/05/18 20:44, Matt Pson wrote:

> Hello,
> I have set up several Squid servers running 3.5.27 on Ubuntu 16.04 LTS (behind keepalived and haproxy to create a ha/load balanced setup) and having are some problems with logging. In order to have a single logfile I decided to send the log directly using UDP to a separate server running rsyslogd.
>
> Each Squid is configured with:
>
> access_log     udp://squid-logserver.domain.com:10514
>
> The rsyslogd server is configured with:
>
> Module (load="imudp")
> # rsyslog Input Modules
> input(type="imudp" port="10514" ruleset="Squid")
> # Squid RulesSet
> # make all squid input from the frontends end up here
> ruleset(name="Squid") {
>         action(type="omfile"
>                 file="/var/log/squid/access.log")
>         }
>
> This works quite well and logs are sent to the server which receives them and puts them in the log file as expected. But the logs are not sent line by line but instead multiple lines at the same time (the size is approx. a UDP packet of lines, ~1300 bytes) which makes the log file quite messed up.

1400 bytes or your network UDP MSS - whichever is smaller. Squid tries
to fit liens in there until one does not fit then sends off the batch.

It's a bug if the buffered_logs directive and access_log buffer-size=
option do not work to control what Squid is *sending*, but UDP can also
be aggregated by the network stacks I/O buffers of both sender and
receiver machines - so make sure you check for that.

<http://www.squid-cache.org/Doc/config/buffered_logs/>
"
Note that even when buffered_logs are off, Squid may have to buffer
records if it cannot write/send them immediately due to pending I/Os
(e.g., the I/O writing the previous log record) or connectivity loss.
"

>
> What am I missing here?
> I want my log to be readable ;)

FWIW; The UDP module is intended for a logger which is capable of
handling the log as a raw data stream. syslog systems are not very good
at that.

You will probably find sending these to the Squid machines syslog and
having that relay them out to the remote server in the syslog format
works better than sending a raw UDP stream.

Or maybe using the TCP module and avoiding the complexity of syslog
entirely.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid logging to UDP logs multiple lines at the same time

Matt Pson


----- Original Message -----
> From: "Amos Jeffries" <[hidden email]>
> To: "squid-users" <[hidden email]>
> Sent: Friday, 4 May, 2018 11:45:50
> Subject: Re: [squid-users] Squid logging to UDP logs multiple lines at the same time

> On 04/05/18 20:44, Matt Pson wrote:

>>
>> This works quite well and logs are sent to the server which receives them and
>> puts them in the log file as expected. But the logs are not sent line by line
>> but instead multiple lines at the same time (the size is approx. a UDP packet
>> of lines, ~1300 bytes) which makes the log file quite messed up.
>
> 1400 bytes or your network UDP MSS - whichever is smaller. Squid tries
> to fit liens in there until one does not fit then sends off the batch.

So in theory it could work if I set the buffer-size low enough, like 16 bytes, so the log line would never fit in the buffer and thus being sent asap?

> It's a bug if the buffered_logs directive and access_log buffer-size=
> option do not work to control what Squid is *sending*, but UDP can also
> be aggregated by the network stacks I/O buffers of both sender and
> receiver machines - so make sure you check for that.
>
> <http://www.squid-cache.org/Doc/config/buffered_logs/>
> "
> Note that even when buffered_logs are off, Squid may have to buffer
> records if it cannot write/send them immediately due to pending I/Os
> (e.g., the I/O writing the previous log record) or connectivity loss.
> "

Thanks for the clarification. Reading "To send each log line as text data to a UDP receiver." (at http://www.squid-cache.org/Doc/config/access_log/ ) made me think that each line would be sent individually without buffering.

>>
>> What am I missing here?
>> I want my log to be readable ;)
>
> FWIW; The UDP module is intended for a logger which is capable of
> handling the log as a raw data stream. syslog systems are not very good
> at that.

I see. It will be very handy when we get around to make our own logging/stats engine take the input from Squid like we do with syslog messages from our firewalls. But that is in the future of another project.

> You will probably find sending these to the Squid machines syslog and
> having that relay them out to the remote server in the syslog format
> works better than sending a raw UDP stream.
>
> Or maybe using the TCP module and avoiding the complexity of syslog
> entirely.

Yep, I will go via the syslog on the squid machines. I was trying to remove as many dependencies as possible but I guess syslog is one I can live with since it it required by other services.

Thanks for a quick and very helpful answer.

/Matt Pson
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid logging to UDP logs multiple lines at the same time

Amos Jeffries
Administrator
On 04/05/18 23:24, Matt Pson wrote:

>
>
> ----- Original Message -----
>> From: "Amos Jeffries"
>
>> On 04/05/18 20:44, Matt Pson wrote:
>
>>>
>>> This works quite well and logs are sent to the server which receives them and
>>> puts them in the log file as expected. But the logs are not sent line by line
>>> but instead multiple lines at the same time (the size is approx. a UDP packet
>>> of lines, ~1300 bytes) which makes the log file quite messed up.
>>
>> 1400 bytes or your network UDP MSS - whichever is smaller. Squid tries
>> to fit liens in there until one does not fit then sends off the batch.
>
> So in theory it could work if I set the buffer-size low enough, like 16 bytes, so the log line would never fit in the buffer and thus being sent asap?
>

In theory, probably. But that would require patching Squid. So it would
be better to add a patch that makes the buffer-size= option work for UDP
instead of having it hard-coded.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users