client_delay_pools doesn't work as expected

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

client_delay_pools doesn't work as expected

Ivan Larionov

We've recently had an incident where misbehaving cluster of clients started fetching 4MB file from squid cache with ~1200 RPS (slowed down to 600 RPS later) which resulted in up to 2Gb/s of traffic sent to clients from each of our squid hosts and quickly overloaded squid.

I'm trying to use client_delay_pools to limit bandwidth per client and prevent misbehaving actors from saturating client-side network / CPU on squid hosts.

However I can't get it to work reliably. It seems to be working as expected for cache MISS, e.g. getting a speed limit of 10MB/s. But it's completely broken for cache HIT, speed I'm getting is ~5KB/s!

The following configuration:

client_delay_pools 1
client_delay_access 1 allow localnet
client_delay_access 1 deny all
client_delay_parameters 1 10000000 20000000

Testing with an already cached big object (2GB ISO file).

client_delay_pools disabled MISS: 20MB/s (probably speed limit on origin side)
client_delay_pools disabled HIT: 110MB/s (probably EBS disk speed)

client_delay_pools enabled MISS: 10MB/s (limit from client_delay_parameters)
client_delay_pools enabled HIT: 5KB/s (what ???)

I retested with a smaller file (337MB) but it made no difference. Still got 5KB download speed on cache HIT.

Any ideas? Am I doing something wrong? Any other ways to limit client-side bandwidth?

Squid version:

Squid Cache: Version 4.14
Service Name: squid
configure options:  '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--libexecdir=/usr/lib/squid' '--includedir=/usr/include' '--datadir=/usr/share/squid' '--sharedstatedir=/usr/com' '--localstatedir=/var' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-epoll' '--enable-removal-policies=heap,lru' '--enable-storeio=aufs,rock' '--enable-delay-pools' '--with-pthreads' '--enable-cache-digests' '--with-large-files' '--with-maxfd=16384' '--enable-htcp'

With best regards, Ivan Larionov.

squid-users mailing list
[hidden email]