Switch cache peer Parent server for every 30 minutes

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Switch cache peer Parent server for every 30 minutes

Prem Chand
Hi ,

My squid cache peer has 3 parent IP’s configured. I need to send HTTPS requests to the first parent IP for 30 minutes and after to the 2nd parent IP for 30 minutes and then to 3rd IP for 30 minutes and this switching needs to happen continuously .I tried round robin but the requests are equally distributed among 3 parents instead of sending requests to each parent for 30 mins .Could you please let us know how I can achieve this?


cache_peer first_IP_Address parent 3218 0 no-query  connect-fail-limit=2
cache_peer second_IP_Address parent 3218 0 no-query connect-fail-limit=2
cache_peer third_IP_Address  parent 3218 0 no-query connect-fail-limit=2

Thanks & Regards
Premchand

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Alex Rousskov
On 6/10/20 6:09 AM, Prem Chand wrote:

> My squid cache peer has 3 parent IP’s configured. I need to send HTTPS
> requests to the first parent IP for 30 minutes and after to the 2nd
> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
> switching needs to happen continuously .Could you please let us know how I
> can achieve this?

If you are OK with hard-coded usage time slots for each peer, then I
would use two[1] "time" ACLs and cache_peer_access rules. Look for
"aclname time" in squid.conf.documented. You will have to generate a
list of (24*2/3=16) staggered time slots for each of the two ACLs, but
it should work. This may be the simplest solution.

[1] You need two ACLs for three peers because the third peer should get
the requests that the first two peers were not allowed to get.

----

With a modern Squid, you could also implement this using a more flexible
(and more expensive, on several layers!) architecture with two ACLs:

1. An external ACL that returns the right cache peer name to use via a
keyword=value annotation API. This always-matching ACL should be
attached to http_access or a similar directive that supports slow ACLs.
Its goal is to annotate the request. You will need to write a
script/program that will compute the right annotations based on time or
some other factors. This is where the flexibility of this solution is
coming from.

2. A "note" ACL attached to cache_peer_access directives, allowing
access to peer X if the external ACL in item 1 returned
use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
reliably used with cache_peer_access.

If you already have another external ACL, you may be able to piggyback
annotations in item 1 to whatever that ACL is already doing.

For more information, search for "keyword=value" and "acl aclname note"
in your squid.conf.documented and see
https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.29


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Prem Chand
Hi Alex,

Thanks for responding to my issue  . I didn't get how the math was done(why it's multiplied by 2) to get 16 slots if possible could you please elaborate with an example.

Regards
Premchand

On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov <[hidden email]> wrote:
On 6/10/20 6:09 AM, Prem Chand wrote:

> My squid cache peer has 3 parent IP’s configured. I need to send HTTPS
> requests to the first parent IP for 30 minutes and after to the 2nd
> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
> switching needs to happen continuously .Could you please let us know how I
> can achieve this?

If you are OK with hard-coded usage time slots for each peer, then I
would use two[1] "time" ACLs and cache_peer_access rules. Look for
"aclname time" in squid.conf.documented. You will have to generate a
list of (24*2/3=16) staggered time slots for each of the two ACLs, but
it should work. This may be the simplest solution.

[1] You need two ACLs for three peers because the third peer should get
the requests that the first two peers were not allowed to get.

----

With a modern Squid, you could also implement this using a more flexible
(and more expensive, on several layers!) architecture with two ACLs:

1. An external ACL that returns the right cache peer name to use via a
keyword=value annotation API. This always-matching ACL should be
attached to http_access or a similar directive that supports slow ACLs.
Its goal is to annotate the request. You will need to write a
script/program that will compute the right annotations based on time or
some other factors. This is where the flexibility of this solution is
coming from.

2. A "note" ACL attached to cache_peer_access directives, allowing
access to peer X if the external ACL in item 1 returned
use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
reliably used with cache_peer_access.

If you already have another external ACL, you may be able to piggyback
annotations in item 1 to whatever that ACL is already doing.

For more information, search for "keyword=value" and "acl aclname note"
in your squid.conf.documented and see
https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.29


HTH,

Alex.


--
prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Antony Stone
On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:

> Hi Alex,
>
> Thanks for responding to my issue  . I didn't get how the math was done(why
> it's multiplied by 2) to get 16 slots if possible could you please elaborate
> with an example.

I believe what Alex meant was:

You want 30 minute timeslots for each of 3 peers, which is 48 half-hour
timeslots throughout the day.

However, you only need to define 48/3 of these for peer A, and 48/3 of them for
peer B, and then let peer C deal with anything not already handled (so it
doesn't need its own definitions).

48/3 = 16, therefore you define 16 half-hour periods when you want peer A to do
the work, 16 half-hour periods for peer B, and then just say "peer C, handle
anything left over".


Regards,


Antony.

> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
> > On 6/10/20 6:09 AM, Prem Chand wrote:
> > > My squid cache peer has 3 parent IP’s configured. I need to send HTTPS
> > > requests to the first parent IP for 30 minutes and after to the 2nd
> > > parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
> > > switching needs to happen continuously .Could you please let us know
> > > how I can achieve this?
> >
> > If you are OK with hard-coded usage time slots for each peer, then I
> > would use two[1] "time" ACLs and cache_peer_access rules. Look for
> > "aclname time" in squid.conf.documented. You will have to generate a
> > list of (24*2/3=16) staggered time slots for each of the two ACLs, but
> > it should work. This may be the simplest solution.
> >
> > [1] You need two ACLs for three peers because the third peer should get
> > the requests that the first two peers were not allowed to get.
> >
> > ----
> >
> > With a modern Squid, you could also implement this using a more flexible
> > (and more expensive, on several layers!) architecture with two ACLs:
> >
> > 1. An external ACL that returns the right cache peer name to use via a
> > keyword=value annotation API. This always-matching ACL should be
> > attached to http_access or a similar directive that supports slow ACLs.
> > Its goal is to annotate the request. You will need to write a
> > script/program that will compute the right annotations based on time or
> > some other factors. This is where the flexibility of this solution is
> > coming from.
> >
> > 2. A "note" ACL attached to cache_peer_access directives, allowing
> > access to peer X if the external ACL in item 1 returned
> > use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
> > reliably used with cache_peer_access.
> >
> > If you already have another external ACL, you may be able to piggyback
> > annotations in item 1 to whatever that ACL is already doing.
> >
> > For more information, search for "keyword=value" and "acl aclname note"
> > in your squid.conf.documented and see
> > https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
> > 29
> >
> >
> > HTH,
> >
> > Alex.

--
Neurotics build castles in the sky;
Psychotics live in them;
Psychiatrists collect the rent.


                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Alex Rousskov
On 6/10/20 12:20 PM, Antony Stone wrote:

> On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>
>> Hi Alex,
>>
>> Thanks for responding to my issue  . I didn't get how the math was done(why
>> it's multiplied by 2) to get 16 slots if possible could you please elaborate
>> with an example.
>
> I believe what Alex meant was:
>
> You want 30 minute timeslots for each of 3 peers, which is 48 half-hour
> timeslots throughout the day.
>
> However, you only need to define 48/3 of these for peer A, and 48/3 of them for
> peer B, and then let peer C deal with anything not already handled (so it
> doesn't need its own definitions).
>
> 48/3 = 16, therefore you define 16 half-hour periods when you want peer A to do
> the work, 16 half-hour periods for peer B, and then just say "peer C, handle
> anything left over".

Thank you, Antony! Here is an untested sketch:

  acl usePeerA time 00:00-00:29
  acl usePeerA time 01:30-01:59
  ... a total of 16 ORed lines for the first peer ...
  ... each line matches a unique 30 minute period ...


  acl usePeerB time 00:30-00:59
  acl usePeerB time 02:00-02:29
  ... a total of 16 ORed lines for the second peer ...
  ... each line matches a unique 30 minute period ...

  # and now match peer to its time slots
  cache_peer_access peerA allow usePeerA
  cache_peer_access peerB allow usePeerB
  cache_peer_access peerC allow !usePeerA !userPeerB


The above may need further adjustments and polishing. For example, I am
not sure how Squid will round these time values. The above assumes that
00:29 limit includes all 60 seconds up to (but excluding) 00:30:00.


HTH,

Alex.


>> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>>> On 6/10/20 6:09 AM, Prem Chand wrote:
>>>> My squid cache peer has 3 parent IP’s configured. I need to send HTTPS
>>>> requests to the first parent IP for 30 minutes and after to the 2nd
>>>> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
>>>> switching needs to happen continuously .Could you please let us know
>>>> how I can achieve this?
>>>
>>> If you are OK with hard-coded usage time slots for each peer, then I
>>> would use two[1] "time" ACLs and cache_peer_access rules. Look for
>>> "aclname time" in squid.conf.documented. You will have to generate a
>>> list of (24*2/3=16) staggered time slots for each of the two ACLs, but
>>> it should work. This may be the simplest solution.
>>>
>>> [1] You need two ACLs for three peers because the third peer should get
>>> the requests that the first two peers were not allowed to get.
>>>
>>> ----
>>>
>>> With a modern Squid, you could also implement this using a more flexible
>>> (and more expensive, on several layers!) architecture with two ACLs:
>>>
>>> 1. An external ACL that returns the right cache peer name to use via a
>>> keyword=value annotation API. This always-matching ACL should be
>>> attached to http_access or a similar directive that supports slow ACLs.
>>> Its goal is to annotate the request. You will need to write a
>>> script/program that will compute the right annotations based on time or
>>> some other factors. This is where the flexibility of this solution is
>>> coming from.
>>>
>>> 2. A "note" ACL attached to cache_peer_access directives, allowing
>>> access to peer X if the external ACL in item 1 returned
>>> use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
>>> reliably used with cache_peer_access.
>>>
>>> If you already have another external ACL, you may be able to piggyback
>>> annotations in item 1 to whatever that ACL is already doing.
>>>
>>> For more information, search for "keyword=value" and "acl aclname note"
>>> in your squid.conf.documented and see
>>> https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>>> 29
>>>
>>>
>>> HTH,
>>>
>>> Alex.
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Prem Chand
Hi Alex,

It's working as expected. I tried to allow only specific domains during the time by adding below acl but I'm getting HTTP status code 503 in my access.log, below is my configuration. Could you please let me know what I'm missing here.

acl usePeerB time 00:30-00:59
acl usePeerB time 02:00-02:29
acl alloweddomains dstdomain google.com facebook.com


cache_peer_access peerA allow usePeerA allowedomains
cache_peer_access peerB allow usePeerB allowedomains
cache_peer_access peerC allow !usePeerA !userPeerB alloweddomains

On Thu, Jun 11, 2020 at 4:54 AM Alex Rousskov <[hidden email]> wrote:
On 6/10/20 12:20 PM, Antony Stone wrote:
> On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>
>> Hi Alex,
>>
>> Thanks for responding to my issue  . I didn't get how the math was done(why
>> it's multiplied by 2) to get 16 slots if possible could you please elaborate
>> with an example.
>
> I believe what Alex meant was:
>
> You want 30 minute timeslots for each of 3 peers, which is 48 half-hour
> timeslots throughout the day.
>
> However, you only need to define 48/3 of these for peer A, and 48/3 of them for
> peer B, and then let peer C deal with anything not already handled (so it
> doesn't need its own definitions).
>
> 48/3 = 16, therefore you define 16 half-hour periods when you want peer A to do
> the work, 16 half-hour periods for peer B, and then just say "peer C, handle
> anything left over".

Thank you, Antony! Here is an untested sketch:

  acl usePeerA time 00:00-00:29
  acl usePeerA time 01:30-01:59
  ... a total of 16 ORed lines for the first peer ...
  ... each line matches a unique 30 minute period ...


  acl usePeerB time 00:30-00:59
  acl usePeerB time 02:00-02:29
  ... a total of 16 ORed lines for the second peer ...
  ... each line matches a unique 30 minute period ...

  # and now match peer to its time slots
  cache_peer_access peerA allow usePeerA
  cache_peer_access peerB allow usePeerB
  cache_peer_access peerC allow !usePeerA !userPeerB


The above may need further adjustments and polishing. For example, I am
not sure how Squid will round these time values. The above assumes that
00:29 limit includes all 60 seconds up to (but excluding) 00:30:00.


HTH,

Alex.


>> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>>> On 6/10/20 6:09 AM, Prem Chand wrote:
>>>> My squid cache peer has 3 parent IP’s configured. I need to send HTTPS
>>>> requests to the first parent IP for 30 minutes and after to the 2nd
>>>> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
>>>> switching needs to happen continuously .Could you please let us know
>>>> how I can achieve this?
>>>
>>> If you are OK with hard-coded usage time slots for each peer, then I
>>> would use two[1] "time" ACLs and cache_peer_access rules. Look for
>>> "aclname time" in squid.conf.documented. You will have to generate a
>>> list of (24*2/3=16) staggered time slots for each of the two ACLs, but
>>> it should work. This may be the simplest solution.
>>>
>>> [1] You need two ACLs for three peers because the third peer should get
>>> the requests that the first two peers were not allowed to get.
>>>
>>> ----
>>>
>>> With a modern Squid, you could also implement this using a more flexible
>>> (and more expensive, on several layers!) architecture with two ACLs:
>>>
>>> 1. An external ACL that returns the right cache peer name to use via a
>>> keyword=value annotation API. This always-matching ACL should be
>>> attached to http_access or a similar directive that supports slow ACLs.
>>> Its goal is to annotate the request. You will need to write a
>>> script/program that will compute the right annotations based on time or
>>> some other factors. This is where the flexibility of this solution is
>>> coming from.
>>>
>>> 2. A "note" ACL attached to cache_peer_access directives, allowing
>>> access to peer X if the external ACL in item 1 returned
>>> use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
>>> reliably used with cache_peer_access.
>>>
>>> If you already have another external ACL, you may be able to piggyback
>>> annotations in item 1 to whatever that ACL is already doing.
>>>
>>> For more information, search for "keyword=value" and "acl aclname note"
>>> in your squid.conf.documented and see
>>> https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>>> 29
>>>
>>>
>>> HTH,
>>>
>>> Alex.
>



--
prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Alex Rousskov
On 6/11/20 11:52 PM, Prem Chand wrote:

> It's working as expected. I tried to allow only specific domains during
> the time by adding below acl but I'm getting HTTP status code 503

> acl usePeerB time 00:30-00:59
> acl usePeerB time 02:00-02:29
> acl alloweddomains dstdomain google.com facebook.com

> cache_peer_access peerA allow usePeerA allowedomains
> cache_peer_access peerB allow usePeerB allowedomains
> cache_peer_access peerC allow !usePeerA !userPeerB alloweddomains

Assuming there are no other cache peers, the above rules leave no
forwarding path for a request to a banned domain. If you want to ban
such requests, http_access instead of cache_peer_access.


HTH,

Alex.


> On Thu, Jun 11, 2020 at 4:54 AM Alex Rousskov wrote:
>
>     On 6/10/20 12:20 PM, Antony Stone wrote:
>     > On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>     >
>     >> Hi Alex,
>     >>
>     >> Thanks for responding to my issue  . I didn't get how the math
>     was done(why
>     >> it's multiplied by 2) to get 16 slots if possible could you
>     please elaborate
>     >> with an example.
>     >
>     > I believe what Alex meant was:
>     >
>     > You want 30 minute timeslots for each of 3 peers, which is 48
>     half-hour
>     > timeslots throughout the day.
>     >
>     > However, you only need to define 48/3 of these for peer A, and
>     48/3 of them for
>     > peer B, and then let peer C deal with anything not already handled
>     (so it
>     > doesn't need its own definitions).
>     >
>     > 48/3 = 16, therefore you define 16 half-hour periods when you want
>     peer A to do
>     > the work, 16 half-hour periods for peer B, and then just say "peer
>     C, handle
>     > anything left over".
>
>     Thank you, Antony! Here is an untested sketch:
>
>       acl usePeerA time 00:00-00:29
>       acl usePeerA time 01:30-01:59
>       ... a total of 16 ORed lines for the first peer ...
>       ... each line matches a unique 30 minute period ...
>
>
>       acl usePeerB time 00:30-00:59
>       acl usePeerB time 02:00-02:29
>       ... a total of 16 ORed lines for the second peer ...
>       ... each line matches a unique 30 minute period ...
>
>       # and now match peer to its time slots
>       cache_peer_access peerA allow usePeerA
>       cache_peer_access peerB allow usePeerB
>       cache_peer_access peerC allow !usePeerA !userPeerB
>
>
>     The above may need further adjustments and polishing. For example, I am
>     not sure how Squid will round these time values. The above assumes that
>     00:29 limit includes all 60 seconds up to (but excluding) 00:30:00.
>
>
>     HTH,
>
>     Alex.
>
>
>     >> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>     >>> On 6/10/20 6:09 AM, Prem Chand wrote:
>     >>>> My squid cache peer has 3 parent IP’s configured. I need to
>     send HTTPS
>     >>>> requests to the first parent IP for 30 minutes and after to the 2nd
>     >>>> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
>     >>>> switching needs to happen continuously .Could you please let us
>     know
>     >>>> how I can achieve this?
>     >>>
>     >>> If you are OK with hard-coded usage time slots for each peer, then I
>     >>> would use two[1] "time" ACLs and cache_peer_access rules. Look for
>     >>> "aclname time" in squid.conf.documented. You will have to generate a
>     >>> list of (24*2/3=16) staggered time slots for each of the two
>     ACLs, but
>     >>> it should work. This may be the simplest solution.
>     >>>
>     >>> [1] You need two ACLs for three peers because the third peer
>     should get
>     >>> the requests that the first two peers were not allowed to get.
>     >>>
>     >>> ----
>     >>>
>     >>> With a modern Squid, you could also implement this using a more
>     flexible
>     >>> (and more expensive, on several layers!) architecture with two ACLs:
>     >>>
>     >>> 1. An external ACL that returns the right cache peer name to use
>     via a
>     >>> keyword=value annotation API. This always-matching ACL should be
>     >>> attached to http_access or a similar directive that supports
>     slow ACLs.
>     >>> Its goal is to annotate the request. You will need to write a
>     >>> script/program that will compute the right annotations based on
>     time or
>     >>> some other factors. This is where the flexibility of this
>     solution is
>     >>> coming from.
>     >>>
>     >>> 2. A "note" ACL attached to cache_peer_access directives, allowing
>     >>> access to peer X if the external ACL in item 1 returned
>     >>> use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
>     >>> reliably used with cache_peer_access.
>     >>>
>     >>> If you already have another external ACL, you may be able to
>     piggyback
>     >>> annotations in item 1 to whatever that ACL is already doing.
>     >>>
>     >>> For more information, search for "keyword=value" and "acl
>     aclname note"
>     >>> in your squid.conf.documented and see
>     >>>
>     https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>     >>> 29
>     >>>
>     >>>
>     >>> HTH,
>     >>>
>     >>> Alex.
>     >
>
>
>
> --
> prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Prem Chand
Hi Alex,

I stopped the peerA(purposefully)  and noticed that requests are failing for the time slots that are going through peerA. I used "connect-fail-limit" in cache_peer  but it's not working. Is there any way we can address this issue using the same solution considering how to handle the requests if any of the parent  peer goes down?

Thanks & Regards
Premchand .

On Fri, Jun 12, 2020 at 6:47 PM Alex Rousskov <[hidden email]> wrote:
On 6/11/20 11:52 PM, Prem Chand wrote:

> It's working as expected. I tried to allow only specific domains during
> the time by adding below acl but I'm getting HTTP status code 503

> acl usePeerB time 00:30-00:59
> acl usePeerB time 02:00-02:29
> acl alloweddomains dstdomain google.com facebook.com

> cache_peer_access peerA allow usePeerA allowedomains
> cache_peer_access peerB allow usePeerB allowedomains
> cache_peer_access peerC allow !usePeerA !userPeerB alloweddomains

Assuming there are no other cache peers, the above rules leave no
forwarding path for a request to a banned domain. If you want to ban
such requests, http_access instead of cache_peer_access.


HTH,

Alex.


> On Thu, Jun 11, 2020 at 4:54 AM Alex Rousskov wrote:
>
>     On 6/10/20 12:20 PM, Antony Stone wrote:
>     > On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>     >
>     >> Hi Alex,
>     >>
>     >> Thanks for responding to my issue  . I didn't get how the math
>     was done(why
>     >> it's multiplied by 2) to get 16 slots if possible could you
>     please elaborate
>     >> with an example.
>     >
>     > I believe what Alex meant was:
>     >
>     > You want 30 minute timeslots for each of 3 peers, which is 48
>     half-hour
>     > timeslots throughout the day.
>     >
>     > However, you only need to define 48/3 of these for peer A, and
>     48/3 of them for
>     > peer B, and then let peer C deal with anything not already handled
>     (so it
>     > doesn't need its own definitions).
>     >
>     > 48/3 = 16, therefore you define 16 half-hour periods when you want
>     peer A to do
>     > the work, 16 half-hour periods for peer B, and then just say "peer
>     C, handle
>     > anything left over".
>
>     Thank you, Antony! Here is an untested sketch:
>
>       acl usePeerA time 00:00-00:29
>       acl usePeerA time 01:30-01:59
>       ... a total of 16 ORed lines for the first peer ...
>       ... each line matches a unique 30 minute period ...
>
>
>       acl usePeerB time 00:30-00:59
>       acl usePeerB time 02:00-02:29
>       ... a total of 16 ORed lines for the second peer ...
>       ... each line matches a unique 30 minute period ...
>
>       # and now match peer to its time slots
>       cache_peer_access peerA allow usePeerA
>       cache_peer_access peerB allow usePeerB
>       cache_peer_access peerC allow !usePeerA !userPeerB
>
>
>     The above may need further adjustments and polishing. For example, I am
>     not sure how Squid will round these time values. The above assumes that
>     00:29 limit includes all 60 seconds up to (but excluding) 00:30:00.
>
>
>     HTH,
>
>     Alex.
>
>
>     >> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>     >>> On 6/10/20 6:09 AM, Prem Chand wrote:
>     >>>> My squid cache peer has 3 parent IP’s configured. I need to
>     send HTTPS
>     >>>> requests to the first parent IP for 30 minutes and after to the 2nd
>     >>>> parent IP for 30 minutes and then to 3rd IP for 30 minutes and this
>     >>>> switching needs to happen continuously .Could you please let us
>     know
>     >>>> how I can achieve this?
>     >>>
>     >>> If you are OK with hard-coded usage time slots for each peer, then I
>     >>> would use two[1] "time" ACLs and cache_peer_access rules. Look for
>     >>> "aclname time" in squid.conf.documented. You will have to generate a
>     >>> list of (24*2/3=16) staggered time slots for each of the two
>     ACLs, but
>     >>> it should work. This may be the simplest solution.
>     >>>
>     >>> [1] You need two ACLs for three peers because the third peer
>     should get
>     >>> the requests that the first two peers were not allowed to get.
>     >>>
>     >>> ----
>     >>>
>     >>> With a modern Squid, you could also implement this using a more
>     flexible
>     >>> (and more expensive, on several layers!) architecture with two ACLs:
>     >>>
>     >>> 1. An external ACL that returns the right cache peer name to use
>     via a
>     >>> keyword=value annotation API. This always-matching ACL should be
>     >>> attached to http_access or a similar directive that supports
>     slow ACLs.
>     >>> Its goal is to annotate the request. You will need to write a
>     >>> script/program that will compute the right annotations based on
>     time or
>     >>> some other factors. This is where the flexibility of this
>     solution is
>     >>> coming from.
>     >>>
>     >>> 2. A "note" ACL attached to cache_peer_access directives, allowing
>     >>> access to peer X if the external ACL in item 1 returned
>     >>> use_cache_peer_=X. The "note" ACL is a fast ACL and, hence, can be
>     >>> reliably used with cache_peer_access.
>     >>>
>     >>> If you already have another external ACL, you may be able to
>     piggyback
>     >>> annotations in item 1 to whatever that ACL is already doing.
>     >>>
>     >>> For more information, search for "keyword=value" and "acl
>     aclname note"
>     >>> in your squid.conf.documented and see
>     >>>
>     https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>     >>> 29
>     >>>
>     >>>
>     >>> HTH,
>     >>>
>     >>> Alex.
>     >
>
>
>
> --
> prem



--
prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Alex Rousskov
On 6/15/20 3:26 AM, Prem Chand wrote:

> I stopped the peerA(purposefully)  and noticed that requests are failing
> for the time slots that are going through peerA. I used
> "connect-fail-limit" in cache_peer  but it's not working. Is there any
> way we can address this issue using the same solution considering how to
> handle the requests if any of the parent  peer goes down?

I am not sure, but I think it should be possible to always give Squid
three peers to use, in the right order. There is no peer selection
algorithm that will do that automatically, but I suspect that a clever
combination of annotate_transaction and "note" ACLs in cache_peer_access
rules can be used to force a particular cache peer selection order.

https://wiki.squid-cache.org/Features/LoadBalance#Go_through_a_peer

The trick is to place one "best" peer into the first group (your rules
already do that!), but then stop banning peers so that the other two
peers are added to the "All Alive Parents" group (your rules currently
deny those two peers from being considered). It may be possible to stop
banning peers while the peer selection code is running its second pass
by changing request annotation.

I am sorry that I do not have enough time to sketch an example.

Alex.



> On Fri, Jun 12, 2020 at 6:47 PM Alex Rousskov wrote:
>
>     On 6/11/20 11:52 PM, Prem Chand wrote:
>
>     > It's working as expected. I tried to allow only specific domains
>     during
>     > the time by adding below acl but I'm getting HTTP status code 503
>
>     > acl usePeerB time 00:30-00:59
>     > acl usePeerB time 02:00-02:29
>     > acl alloweddomains dstdomain google.com <http://google.com>
>     facebook.com <http://facebook.com>
>
>     > cache_peer_access peerA allow usePeerA allowedomains
>     > cache_peer_access peerB allow usePeerB allowedomains
>     > cache_peer_access peerC allow !usePeerA !userPeerB alloweddomains
>
>     Assuming there are no other cache peers, the above rules leave no
>     forwarding path for a request to a banned domain. If you want to ban
>     such requests, http_access instead of cache_peer_access.
>
>
>     HTH,
>
>     Alex.
>
>
>     > On Thu, Jun 11, 2020 at 4:54 AM Alex Rousskov wrote:
>     >
>     >     On 6/10/20 12:20 PM, Antony Stone wrote:
>     >     > On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>     >     >
>     >     >> Hi Alex,
>     >     >>
>     >     >> Thanks for responding to my issue  . I didn't get how the math
>     >     was done(why
>     >     >> it's multiplied by 2) to get 16 slots if possible could you
>     >     please elaborate
>     >     >> with an example.
>     >     >
>     >     > I believe what Alex meant was:
>     >     >
>     >     > You want 30 minute timeslots for each of 3 peers, which is 48
>     >     half-hour
>     >     > timeslots throughout the day.
>     >     >
>     >     > However, you only need to define 48/3 of these for peer A, and
>     >     48/3 of them for
>     >     > peer B, and then let peer C deal with anything not already
>     handled
>     >     (so it
>     >     > doesn't need its own definitions).
>     >     >
>     >     > 48/3 = 16, therefore you define 16 half-hour periods when
>     you want
>     >     peer A to do
>     >     > the work, 16 half-hour periods for peer B, and then just say
>     "peer
>     >     C, handle
>     >     > anything left over".
>     >
>     >     Thank you, Antony! Here is an untested sketch:
>     >
>     >       acl usePeerA time 00:00-00:29
>     >       acl usePeerA time 01:30-01:59
>     >       ... a total of 16 ORed lines for the first peer ...
>     >       ... each line matches a unique 30 minute period ...
>     >
>     >
>     >       acl usePeerB time 00:30-00:59
>     >       acl usePeerB time 02:00-02:29
>     >       ... a total of 16 ORed lines for the second peer ...
>     >       ... each line matches a unique 30 minute period ...
>     >
>     >       # and now match peer to its time slots
>     >       cache_peer_access peerA allow usePeerA
>     >       cache_peer_access peerB allow usePeerB
>     >       cache_peer_access peerC allow !usePeerA !userPeerB
>     >
>     >
>     >     The above may need further adjustments and polishing. For
>     example, I am
>     >     not sure how Squid will round these time values. The above
>     assumes that
>     >     00:29 limit includes all 60 seconds up to (but excluding)
>     00:30:00.
>     >
>     >
>     >     HTH,
>     >
>     >     Alex.
>     >
>     >
>     >     >> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>     >     >>> On 6/10/20 6:09 AM, Prem Chand wrote:
>     >     >>>> My squid cache peer has 3 parent IP’s configured. I need to
>     >     send HTTPS
>     >     >>>> requests to the first parent IP for 30 minutes and after
>     to the 2nd
>     >     >>>> parent IP for 30 minutes and then to 3rd IP for 30
>     minutes and this
>     >     >>>> switching needs to happen continuously .Could you please
>     let us
>     >     know
>     >     >>>> how I can achieve this?
>     >     >>>
>     >     >>> If you are OK with hard-coded usage time slots for each
>     peer, then I
>     >     >>> would use two[1] "time" ACLs and cache_peer_access rules.
>     Look for
>     >     >>> "aclname time" in squid.conf.documented. You will have to
>     generate a
>     >     >>> list of (24*2/3=16) staggered time slots for each of the two
>     >     ACLs, but
>     >     >>> it should work. This may be the simplest solution.
>     >     >>>
>     >     >>> [1] You need two ACLs for three peers because the third peer
>     >     should get
>     >     >>> the requests that the first two peers were not allowed to get.
>     >     >>>
>     >     >>> ----
>     >     >>>
>     >     >>> With a modern Squid, you could also implement this using a
>     more
>     >     flexible
>     >     >>> (and more expensive, on several layers!) architecture with
>     two ACLs:
>     >     >>>
>     >     >>> 1. An external ACL that returns the right cache peer name
>     to use
>     >     via a
>     >     >>> keyword=value annotation API. This always-matching ACL
>     should be
>     >     >>> attached to http_access or a similar directive that supports
>     >     slow ACLs.
>     >     >>> Its goal is to annotate the request. You will need to write a
>     >     >>> script/program that will compute the right annotations
>     based on
>     >     time or
>     >     >>> some other factors. This is where the flexibility of this
>     >     solution is
>     >     >>> coming from.
>     >     >>>
>     >     >>> 2. A "note" ACL attached to cache_peer_access directives,
>     allowing
>     >     >>> access to peer X if the external ACL in item 1 returned
>     >     >>> use_cache_peer_=X. The "note" ACL is a fast ACL and,
>     hence, can be
>     >     >>> reliably used with cache_peer_access.
>     >     >>>
>     >     >>> If you already have another external ACL, you may be able to
>     >     piggyback
>     >     >>> annotations in item 1 to whatever that ACL is already doing.
>     >     >>>
>     >     >>> For more information, search for "keyword=value" and "acl
>     >     aclname note"
>     >     >>> in your squid.conf.documented and see
>     >     >>>
>     >   
>      https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>     >     >>> 29
>     >     >>>
>     >     >>>
>     >     >>> HTH,
>     >     >>>
>     >     >>> Alex.
>     >     >
>     >
>     >
>     >
>     > --
>     > prem
>
>
>
> --
> prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Switch cache peer Parent server for every 30 minutes

Prem Chand
Hi Alex,

Could you please share with me a rough sketch example for the below statement.
"but I suspect that a clever
combination of annotate_transaction and "note" ACLs in cache_peer_access
rules can be used to force a particular cache peer selection order."

On Mon, Jun 15, 2020 at 7:14 PM Alex Rousskov <[hidden email]> wrote:
On 6/15/20 3:26 AM, Prem Chand wrote:

> I stopped the peerA(purposefully)  and noticed that requests are failing
> for the time slots that are going through peerA. I used
> "connect-fail-limit" in cache_peer  but it's not working. Is there any
> way we can address this issue using the same solution considering how to
> handle the requests if any of the parent  peer goes down?

I am not sure, but I think it should be possible to always give Squid
three peers to use, in the right order. There is no peer selection
algorithm that will do that automatically, but I suspect that a clever
combination of annotate_transaction and "note" ACLs in cache_peer_access
rules can be used to force a particular cache peer selection order.

https://wiki.squid-cache.org/Features/LoadBalance#Go_through_a_peer

The trick is to place one "best" peer into the first group (your rules
already do that!), but then stop banning peers so that the other two
peers are added to the "All Alive Parents" group (your rules currently
deny those two peers from being considered). It may be possible to stop
banning peers while the peer selection code is running its second pass
by changing request annotation.

I am sorry that I do not have enough time to sketch an example.

Alex.



> On Fri, Jun 12, 2020 at 6:47 PM Alex Rousskov wrote:
>
>     On 6/11/20 11:52 PM, Prem Chand wrote:
>
>     > It's working as expected. I tried to allow only specific domains
>     during
>     > the time by adding below acl but I'm getting HTTP status code 503
>
>     > acl usePeerB time 00:30-00:59
>     > acl usePeerB time 02:00-02:29
>     > acl alloweddomains dstdomain google.com <http://google.com>
>     facebook.com <http://facebook.com>
>
>     > cache_peer_access peerA allow usePeerA allowedomains
>     > cache_peer_access peerB allow usePeerB allowedomains
>     > cache_peer_access peerC allow !usePeerA !userPeerB alloweddomains
>
>     Assuming there are no other cache peers, the above rules leave no
>     forwarding path for a request to a banned domain. If you want to ban
>     such requests, http_access instead of cache_peer_access.
>
>
>     HTH,
>
>     Alex.
>
>
>     > On Thu, Jun 11, 2020 at 4:54 AM Alex Rousskov wrote:
>     >
>     >     On 6/10/20 12:20 PM, Antony Stone wrote:
>     >     > On Wednesday 10 June 2020 at 18:11:03, Prem Chand wrote:
>     >     >
>     >     >> Hi Alex,
>     >     >>
>     >     >> Thanks for responding to my issue  . I didn't get how the math
>     >     was done(why
>     >     >> it's multiplied by 2) to get 16 slots if possible could you
>     >     please elaborate
>     >     >> with an example.
>     >     >
>     >     > I believe what Alex meant was:
>     >     >
>     >     > You want 30 minute timeslots for each of 3 peers, which is 48
>     >     half-hour
>     >     > timeslots throughout the day.
>     >     >
>     >     > However, you only need to define 48/3 of these for peer A, and
>     >     48/3 of them for
>     >     > peer B, and then let peer C deal with anything not already
>     handled
>     >     (so it
>     >     > doesn't need its own definitions).
>     >     >
>     >     > 48/3 = 16, therefore you define 16 half-hour periods when
>     you want
>     >     peer A to do
>     >     > the work, 16 half-hour periods for peer B, and then just say
>     "peer
>     >     C, handle
>     >     > anything left over".
>     >
>     >     Thank you, Antony! Here is an untested sketch:
>     >
>     >       acl usePeerA time 00:00-00:29
>     >       acl usePeerA time 01:30-01:59
>     >       ... a total of 16 ORed lines for the first peer ...
>     >       ... each line matches a unique 30 minute period ...
>     >
>     >
>     >       acl usePeerB time 00:30-00:59
>     >       acl usePeerB time 02:00-02:29
>     >       ... a total of 16 ORed lines for the second peer ...
>     >       ... each line matches a unique 30 minute period ...
>     >
>     >       # and now match peer to its time slots
>     >       cache_peer_access peerA allow usePeerA
>     >       cache_peer_access peerB allow usePeerB
>     >       cache_peer_access peerC allow !usePeerA !userPeerB
>     >
>     >
>     >     The above may need further adjustments and polishing. For
>     example, I am
>     >     not sure how Squid will round these time values. The above
>     assumes that
>     >     00:29 limit includes all 60 seconds up to (but excluding)
>     00:30:00.
>     >
>     >
>     >     HTH,
>     >
>     >     Alex.
>     >
>     >
>     >     >> On Wed, Jun 10, 2020 at 7:12 PM Alex Rousskov wrote:
>     >     >>> On 6/10/20 6:09 AM, Prem Chand wrote:
>     >     >>>> My squid cache peer has 3 parent IP’s configured. I need to
>     >     send HTTPS
>     >     >>>> requests to the first parent IP for 30 minutes and after
>     to the 2nd
>     >     >>>> parent IP for 30 minutes and then to 3rd IP for 30
>     minutes and this
>     >     >>>> switching needs to happen continuously .Could you please
>     let us
>     >     know
>     >     >>>> how I can achieve this?
>     >     >>>
>     >     >>> If you are OK with hard-coded usage time slots for each
>     peer, then I
>     >     >>> would use two[1] "time" ACLs and cache_peer_access rules.
>     Look for
>     >     >>> "aclname time" in squid.conf.documented. You will have to
>     generate a
>     >     >>> list of (24*2/3=16) staggered time slots for each of the two
>     >     ACLs, but
>     >     >>> it should work. This may be the simplest solution.
>     >     >>>
>     >     >>> [1] You need two ACLs for three peers because the third peer
>     >     should get
>     >     >>> the requests that the first two peers were not allowed to get.
>     >     >>>
>     >     >>> ----
>     >     >>>
>     >     >>> With a modern Squid, you could also implement this using a
>     more
>     >     flexible
>     >     >>> (and more expensive, on several layers!) architecture with
>     two ACLs:
>     >     >>>
>     >     >>> 1. An external ACL that returns the right cache peer name
>     to use
>     >     via a
>     >     >>> keyword=value annotation API. This always-matching ACL
>     should be
>     >     >>> attached to http_access or a similar directive that supports
>     >     slow ACLs.
>     >     >>> Its goal is to annotate the request. You will need to write a
>     >     >>> script/program that will compute the right annotations
>     based on
>     >     time or
>     >     >>> some other factors. This is where the flexibility of this
>     >     solution is
>     >     >>> coming from.
>     >     >>>
>     >     >>> 2. A "note" ACL attached to cache_peer_access directives,
>     allowing
>     >     >>> access to peer X if the external ACL in item 1 returned
>     >     >>> use_cache_peer_=X. The "note" ACL is a fast ACL and,
>     hence, can be
>     >     >>> reliably used with cache_peer_access.
>     >     >>>
>     >     >>> If you already have another external ACL, you may be able to
>     >     piggyback
>     >     >>> annotations in item 1 to whatever that ACL is already doing.
>     >     >>>
>     >     >>> For more information, search for "keyword=value" and "acl
>     >     aclname note"
>     >     >>> in your squid.conf.documented and see
>     >     >>>
>     >   
>      https://wiki.squid-cache.org/Features/AddonHelpers#Access_Control_.28ACL.
>     >     >>> 29
>     >     >>>
>     >     >>>
>     >     >>> HTH,
>     >     >>>
>     >     >>> Alex.
>     >     >
>     >
>     >
>     >
>     > --
>     > prem
>
>
>
> --
> prem



--
prem

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users