Squid performance in the tank.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid performance in the tank.

Jeff Honey

I don't know that I've ever had occasion to ask the squid group anything before but this one thing has me stumped. We just moved our infrastructure from one facility to another and our squid servers' performance has really gone down the tubes. Request processes have slowed to a crawl. Admittedly, we have made some changes to the routing of external requests (as we are no longer in a flat network) but all the systems in that same IP network have no trouble at all getting to the outside world.

When squid receives a page request, it just seems to sit on it for a few seconds before doing anything with it and the end user doesn't see any activity from squid for a minute or longer. This was a perfectly functioning squid setup prior to our move. The only thing that has changed is the path it takes to get to the Internet. How should I go about finding out if it is squid with the problem or if it just something boneheaded I've done somewhere else?

<snip>
Squid Cache: Version 2.5.STABLE1-20030206
configure options:  --prefix=/usr/local/squid25 --enable-dlmalloc --enable-ssl --enable-openssl --enable-useragent-log --enable-snmp --enable-kill-parent-hack --enable-time-hack --enable-delay-pools --enable-referer-log --enable-underscores '--enable-auth=basic digest ntlm'
</snip>

¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
¤ Jeff Honey, Network Administrator
¤ PS America, Inc.
¤ 4426 N. Orange Blossom Trl
¤ Orlando, FL  32804
¤ 407-521-1011 voice
¤ 407-521-1007 fax
¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

Reply | Threaded
Open this post in threaded view
|

Re: Squid performance in the tank.

Shekhar Gupta-2
What is the change in Network , are the WAN Ip's still the same or
they have changed .
something on routing like previously u were using default route and
now switched to BGP ??


On 7/6/07, Jeff Honey <[hidden email]> wrote:

>
> I don't know that I've ever had occasion to ask the squid group anything before but this one thing has me stumped. We just moved our infrastructure from one facility to another and our squid servers' performance has really gone down the tubes. Request processes have slowed to a crawl. Admittedly, we have made some changes to the routing of external requests (as we are no longer in a flat network) but all the systems in that same IP network have no trouble at all getting to the outside world.
>
> When squid receives a page request, it just seems to sit on it for a few seconds before doing anything with it and the end user doesn't see any activity from squid for a minute or longer. This was a perfectly functioning squid setup prior to our move. The only thing that has changed is the path it takes to get to the Internet. How should I go about finding out if it is squid with the problem or if it just something boneheaded I've done somewhere else?
>
> <snip>
> Squid Cache: Version 2.5.STABLE1-20030206
> configure options:  --prefix=/usr/local/squid25 --enable-dlmalloc --enable-ssl --enable-openssl --enable-useragent-log --enable-snmp --enable-kill-parent-hack --enable-time-hack --enable-delay-pools --enable-referer-log --enable-underscores '--enable-auth=basic digest ntlm'
> </snip>
>
> ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
> ¤ Jeff Honey, Network Administrator
> ¤ PS America, Inc.
> ¤ 4426 N. Orange Blossom Trl
> ¤ Orlando, FL  32804
> ¤ 407-521-1011 voice
> ¤ 407-521-1007 fax
> ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Squid performance in the tank.

Jeff Honey
> What is the change in Network , are the WAN Ip's still the
> same or they have changed .
> something on routing like previously u were using default
> route and now switched to BGP ??

Where there was only just a firewall between the proxy and the Internet now there is a routing switch a router and a firewall between the server and the Internet. There is a specific rule on the new firewall, much as on the old one, that allows HTTP(S) traffic to/from our proxy so that should not be an issue. Honestly, I'm just looking for a way to dig into the guts of squid a little deeper to get some performance information and unearth where the slowdown is occurring.

¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
¤ Jeff Honey, Network Administrator
¤ PS America, Inc.
¤ 4426 N. Orange Blossom Trl
¤ Orlando, FL  32804
¤ 407-521-1011 voice
¤ 407-521-1007 fax
¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

 
Reply | Threaded
Open this post in threaded view
|

Re: Squid performance in the tank.

D & E Radel
Jeff Honey wrote:
>> What is the change in Network , are the WAN Ip's still the
>> same or they have changed .
>> something on routing like previously u were using default
>> route and now switched to BGP ??
>>    
>
> Where there was only just a firewall between the proxy and the Internet now there is a routing switch a router and a firewall between the server and the Internet. There is a specific rule on the new firewall, much as on the old one, that allows HTTP(S) traffic to/from our proxy so that should not be an issue. Honestly, I'm just looking for a way to dig into the guts of squid a little deeper to get some performance information and unearth where the slowdown is occurring.
>
>  

When you ping and traceroute your squidbox from various points on your
network, does it show anything weird? What about ping and traceroute
from your squid box to sites on the internet?

cheers,
D.Radel.
Reply | Threaded
Open this post in threaded view
|

Re: Squid performance in the tank.

Adrian Chadd
In reply to this post by Jeff Honey
Firstly, upgrade from Squid-2.5 to Squid-2.6. If you're on Linux, FreeBSD
(or soon, Solaris) then you'll definitely notice the CPU drop.

I'd check that your DNS is functioning, that your MTU is consistent
everywhere, you're not filtering ICMP. Saying "the only thing that
changed is the path it takes to get to the internet" is basically
admitting you changed something major; being confused over why a network
application changed behaviour when your network has changed is a bit
silly. :)

I'd do some test requests through the proxy whilst using tcpdump
on the squid proxy to identify what its trying to do -during- that
request. You might spot something.



Adrian

On Thu, Jul 05, 2007, Jeff Honey wrote:

>
> I don't know that I've ever had occasion to ask the squid group anything before but this one thing has me stumped. We just moved our infrastructure from one facility to another and our squid servers' performance has really gone down the tubes. Request processes have slowed to a crawl. Admittedly, we have made some changes to the routing of external requests (as we are no longer in a flat network) but all the systems in that same IP network have no trouble at all getting to the outside world.
>
> When squid receives a page request, it just seems to sit on it for a few seconds before doing anything with it and the end user doesn't see any activity from squid for a minute or longer. This was a perfectly functioning squid setup prior to our move. The only thing that has changed is the path it takes to get to the Internet. How should I go about finding out if it is squid with the problem or if it just something boneheaded I've done somewhere else?
>
> <snip>
> Squid Cache: Version 2.5.STABLE1-20030206
> configure options:  --prefix=/usr/local/squid25 --enable-dlmalloc --enable-ssl --enable-openssl --enable-useragent-log --enable-snmp --enable-kill-parent-hack --enable-time-hack --enable-delay-pools --enable-referer-log --enable-underscores '--enable-auth=basic digest ntlm'
> </snip>
>
> ???????????????????????????????????
> ? Jeff Honey, Network Administrator
> ? PS America, Inc.
> ? 4426 N. Orange Blossom Trl
> ? Orlando, FL  32804
> ? 407-521-1011 voice
> ? 407-521-1007 fax
> ???????????????????????????????????

--
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level bandwidth-capped VPSes available in WA -
Reply | Threaded
Open this post in threaded view
|

Re: Squid performance in the tank.

Tek Bahadur Limbu
In reply to this post by Jeff Honey
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 5 Jul 2007 15:44:36 -0400
"Jeff Honey" <[hidden email]> wrote:

>
> I don't know that I've ever had occasion to ask the squid group anything before but this one thing has me stumped. We just moved our infrastructure from one facility to another and our squid servers' performance has really gone down the tubes. Request processes have slowed to a crawl. Admittedly, we have made some changes to the routing of external requests (as we are no longer in a flat network) but all the systems in that same IP network have no trouble at all getting to the outside world.
>
> When squid receives a page request, it just seems to sit on it for a few seconds before doing anything with it and the end user doesn't see any activity from squid for a minute or longer. This was a perfectly functioning squid setup prior to our move. The only thing that has changed is the path it takes to get to the Internet. How should I go about finding out if it is squid with the problem or if it just something boneheaded I've done somewhere else?

Hi Jeff,

I agree with Adrian regarding upgrading from your  Squid-2.5 to the current version of 2.6.13. You will appreciate the drop in CPU load among other things.

Since your Squid box was working fine yesterday, upgrading will probably not solve your problems however.

I suppose you are not running Squid in transparent mode?

DNS could also be the culprit as Adrian had mentioned. How fast can your Squid box resolve DNS queries?

Since the only thing that has changed is an additional router between Squid and the Internet. I would first run a tcpdump between with your Squid box and your router to make sure the firewalls are doing their jobs fine.

Doesn't access.log and cache.log complain about anything?

Thanking you...

>
> <snip>
> Squid Cache: Version 2.5.STABLE1-20030206
> configure options:  --prefix=/usr/local/squid25 --enable-dlmalloc --enable-ssl --enable-openssl --enable-useragent-log --enable-snmp --enable-kill-parent-hack --enable-time-hack --enable-delay-pools --enable-referer-log --enable-underscores '--enable-auth=basic digest ntlm'
> </snip>


>
>
> ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
> ¤ Jeff Honey, Network Administrator
> ¤ PS America, Inc.
> ¤ 4426 N. Orange Blossom Trl
> ¤ Orlando, FL  32804
> ¤ 407-521-1011 voice
> ¤ 407-521-1007 fax
> ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
>
>


- --


With best regards and good wishes,

Yours sincerely,

Tek Bahadur Limbu

(TAG/TDG Group)
Jwl Systems Department

Worldlink Communications Pvt. Ltd.

Jawalakhel, Nepal

http://www.wlink.com.np
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (FreeBSD)

iD8DBQFGjeoEVrOl+eVhOvYRAp8ZAJ9mzo/9g3fV/sr9BNNA1lFbVHE29QCfbPAc
9E45d/wObtv5niJ4czTwWSk=
=n9xa
-----END PGP SIGNATURE-----