measuring latency of squid in different scenarios

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

measuring latency of squid in different scenarios

washuu
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

Alex Rousskov
On 9/28/20 9:19 AM, Rafał Stanilewicz wrote:

> I'd like to get some numbers about squid-introduced latency of getting
> some particular web resource. Is there any benchmarking program I could
> use? I'd like to see what is the current latency of getting the resource
> without any proxying, then of getting the same resource with explicit
> proxy settings, then of implicit (intercepting) proxy option, as well as
> for different options of caching. 

What is the primary goal of collecting those measurements? Let's assume
that the measurements show that Squid adds X% to the median response
time in a particular test scenario. Now what?

* If the primary goal is to just record/report _some_ number and forget
about it, then you can use curl, wget, or ab to generate dumb test
traffic and measure overall response times of primary configurations.
This (mostly pointless from a purely technical point of view) exercise
should not take more than a few hours. It is useful, for example, in
cases where one needs to report some measurements to the management, but
everybody just wants to mark some checkbox on some list.

* If the primary goal is to verify some performance guarantees or tune
Squid performance, then you would need to invest a lot more into these
performance tests. You need stable, reproducible results and
representative traffic pattern(s). I use Web Polygraph
(http://www.web-polygraph.org) for such tests, but it has a steep
learning curve, may need some love to compile in your environment, and
it is a biased recommendation.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

neok
In reply to this post by washuu
Hi Rafal, if you wish I've a manual redacted in SPANISH for build a VM whit Debian 10.5 running SQUID compiled from source, with kerberos and LDAP authentication, plus AD groups authorizations.

I haven't had time to translate it into English yet.
Let me know if it works for you and I'll share it with you.

Best regards,
Gabriel




El lun., 28 sep. 2020 10:19, Rafał Stanilewicz <[hidden email]> escribió:
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

neok
Below I leave the link. I think that with this you could achieve your goal. In this project there are more things that you might not want to use or maybe you do. To begin I believe that it is well.

  • High availability load balancing frontend between users and backend proxy nodes.
  • VIP (floating IP) for the load balancers.
  • Automatic configuration script for internal routing.
  • Proxy pool with integrated Kerberos and LDAP authentication in Active Directory
  • Domain, IP, and port filtering
  • Active Directory group browsing permissions
  • Navigation reports by cost centers and/or individual users
  • Bandwidth usage control per user.

Any question you may have, please reply with a copy to SQUID's mailing list in order to share with the community of users information that they may find useful.

Best regards,
Gabriel

El mié., 30 de sep. de 2020 a la(s) 05:12, Rafał Stanilewicz ([hidden email]) escribió:
Hi Gabriel, 

although I do not know Spanish, a few of my friends do. Also, the most important pieces will be code samples, which do not need translation. So if you would be so kind and share the manual with me, I'd appreciate it very much!

Rafal 

On Tue, 29 Sep 2020 at 23:07, Service MV <[hidden email]> wrote:
Hi Rafal, if you wish I've a manual redacted in SPANISH for build a VM whit Debian 10.5 running SQUID compiled from source, with kerberos and LDAP authentication, plus AD groups authorizations.

I haven't had time to translate it into English yet.
Let me know if it works for you and I'll share it with you.

Best regards,
Gabriel




El lun., 28 sep. 2020 10:19, Rafał Stanilewicz <[hidden email]> escribió:
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Zanim wydrukujesz, pomyśl o środowisku.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

neok
Thanks Louis!
I'll be waiting for your configuration.

Along with this I believe that the spirit of the open-source community is to share. So I hope the information can be useful to the community (especially in this case to spanish speakers haha ;))

Best regards,
Gabriel

El mié., 30 sep. 2020 10:48, L.P.H. van Belle <[hidden email]> escribió:
Hai Gabriel,

Only one thing i dont like... Spanish? Portugees? ..  ;-) but google translater is my friend. 

But this looks great, very nice..  !!!

I have a simular setup here, but i have only samba running, 0 windows servers.
I've seen same parts to simplify this a bit and when im done with that, i'l make the english "debianized" version of this..
Its a nice addition for my (Work in progress) automated Small bussiness setup on linux.

When im done, i'll send you/the list a copy of the debianized and english version..

Big thanks for sharing this !


Greetz,
Louis



Van: squid-users [mailto:[hidden email]] Namens Service MV
Verzonden: woensdag 30 september 2020 15:23
Aan: Rafa?? Stanilewicz; Squid Users
Onderwerp: Re: [squid-users] measuring latency of squid in different scenarios

Below I leave the link. I think that with this you could achieve your goal. In this project there are more things that you might not want to use or maybe you do. To begin I believe that it is well.

  • High availability load balancing frontend between users and backend proxy nodes.
  • VIP (floating IP) for the load balancers.
  • Automatic configuration script for internal routing.
  • Proxy pool with integrated Kerberos and LDAP authentication in Active Directory
  • Domain, IP, and port filtering
  • Active Directory group browsing permissions
  • Navigation reports by cost centers and/or individual users
  • Bandwidth usage control per user.

Any question you may have, please reply with a copy to SQUID's mailing list in order to share with the community of users information that they may find useful.

Best regards,
Gabriel

El mié., 30 de sep. de 2020 a la(s) 05:12, Rafał Stanilewicz ([hidden email]) escribió:
Hi Gabriel, 

although I do not know Spanish, a few of my friends do. Also, the most important pieces will be code samples, which do not need translation. So if you would be so kind and share the manual with me, I'd appreciate it very much!

Rafal 

On Tue, 29 Sep 2020 at 23:07, Service MV <[hidden email]> wrote:
Hi Rafal, if you wish I've a manual redacted in SPANISH for build a VM whit Debian 10.5 running SQUID compiled from source, with kerberos and LDAP authentication, plus AD groups authorizations.

I haven't had time to translate it into English yet.
Let me know if it works for you and I'll share it with you.

Best regards,
Gabriel




El lun., 28 sep. 2020 10:19, Rafał Stanilewicz <[hidden email]> escribió:
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Zanim wydrukujesz, pomyśl o środowisku.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

washuu
In reply to this post by neok
 Hi Gabriel, 

thank you very much, I confirm I downloaded successfully the document, and I'm going to read it carefully, although it will take me some time. 

Still, my second question remains: is there any way of measuring the time of getting some resource through squid? 

Best regards, 

Rafal Stanilewicz

On Wed, 30 Sep 2020 at 14:23, Service MV <[hidden email]> wrote:
Below I leave the link. I think that with this you could achieve your goal. In this project there are more things that you might not want to use or maybe you do. To begin I believe that it is well.

  • High availability load balancing frontend between users and backend proxy nodes.
  • VIP (floating IP) for the load balancers.
  • Automatic configuration script for internal routing.
  • Proxy pool with integrated Kerberos and LDAP authentication in Active Directory
  • Domain, IP, and port filtering
  • Active Directory group browsing permissions
  • Navigation reports by cost centers and/or individual users
  • Bandwidth usage control per user.

Any question you may have, please reply with a copy to SQUID's mailing list in order to share with the community of users information that they may find useful.

Best regards,
Gabriel

El mié., 30 de sep. de 2020 a la(s) 05:12, Rafał Stanilewicz ([hidden email]) escribió:
Hi Gabriel, 

although I do not know Spanish, a few of my friends do. Also, the most important pieces will be code samples, which do not need translation. So if you would be so kind and share the manual with me, I'd appreciate it very much!

Rafal 

On Tue, 29 Sep 2020 at 23:07, Service MV <[hidden email]> wrote:
Hi Rafal, if you wish I've a manual redacted in SPANISH for build a VM whit Debian 10.5 running SQUID compiled from source, with kerberos and LDAP authentication, plus AD groups authorizations.

I haven't had time to translate it into English yet.
Let me know if it works for you and I'll share it with you.

Best regards,
Gabriel




El lun., 28 sep. 2020 10:19, Rafał Stanilewicz <[hidden email]> escribió:
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Zanim wydrukujesz, pomyśl o środowisku.


--
Zanim wydrukujesz, pomyśl o środowisku.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: measuring latency of squid in different scenarios

Mike Rumph
Hello Rafal,

I have run some performance tests with WRK for Squid running as a proxy to a backend Apache httpd server.
This gives an example of latency measurements for Squid.

Maybe this will be useful for you.

Thanks,

Mike Rumph

On Thu, Oct 1, 2020 at 2:45 AM Rafał Stanilewicz <[hidden email]> wrote:
 Hi Gabriel, 

thank you very much, I confirm I downloaded successfully the document, and I'm going to read it carefully, although it will take me some time. 

Still, my second question remains: is there any way of measuring the time of getting some resource through squid? 

Best regards, 

Rafal Stanilewicz

On Wed, 30 Sep 2020 at 14:23, Service MV <[hidden email]> wrote:
Below I leave the link. I think that with this you could achieve your goal. In this project there are more things that you might not want to use or maybe you do. To begin I believe that it is well.

  • High availability load balancing frontend between users and backend proxy nodes.
  • VIP (floating IP) for the load balancers.
  • Automatic configuration script for internal routing.
  • Proxy pool with integrated Kerberos and LDAP authentication in Active Directory
  • Domain, IP, and port filtering
  • Active Directory group browsing permissions
  • Navigation reports by cost centers and/or individual users
  • Bandwidth usage control per user.

Any question you may have, please reply with a copy to SQUID's mailing list in order to share with the community of users information that they may find useful.

Best regards,
Gabriel

El mié., 30 de sep. de 2020 a la(s) 05:12, Rafał Stanilewicz ([hidden email]) escribió:
Hi Gabriel, 

although I do not know Spanish, a few of my friends do. Also, the most important pieces will be code samples, which do not need translation. So if you would be so kind and share the manual with me, I'd appreciate it very much!

Rafal 

On Tue, 29 Sep 2020 at 23:07, Service MV <[hidden email]> wrote:
Hi Rafal, if you wish I've a manual redacted in SPANISH for build a VM whit Debian 10.5 running SQUID compiled from source, with kerberos and LDAP authentication, plus AD groups authorizations.

I haven't had time to translate it into English yet.
Let me know if it works for you and I'll share it with you.

Best regards,
Gabriel




El lun., 28 sep. 2020 10:19, Rafał Stanilewicz <[hidden email]> escribió:
Hello, 

I'm planning the deployment of web proxy in my environment. It's not very big, around 80 typical windows 10 workstations, active directory, plus some DMZ servers. For now, there is very basic L7 inspection on the edge firewall. 

I plan to use two separate squid instances, one for explicit proxy traffic, forced by AD GPO settings, and second for traffic still being sent directly to the Internet (as several applications we use tend to ignore the system proxy settings). The first instance will use (hopefully) AD authentication, while the second will use only srcIP-based rules. I will be grateful for any comments, what should I focus on, or some quirks - I've never deployed squid from scratch. 

But my main point of writing is:

I'd like to get some numbers about squid-introduced latency of getting some particular web resource. Is there any benchmarking program I could use? I'd like to see what is the current latency of getting the resource without any proxying, then of getting the same resource with explicit proxy settings, then of implicit (intercepting) proxy option, as well as for different options of caching. 

How should I start? Is there any software I can use to measure that, besides analysis of HAR files? 

So far, I used squid only in home environment, and without a need for granular measurement. 

Best regards, 

Rafal Stanilewicz

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


--
Zanim wydrukujesz, pomyśl o środowisku.


--
Zanim wydrukujesz, pomyśl o środowisku.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users