annotation and fast / slow acl

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

annotation and fast / slow acl

FUSTE Emmanuel
Hello,

I need to select a cache peer based on the user group.
As cache_peer_access need a fast acl to have predicable result, I tried to
- annotate transactions with "note"
- match the annotation with a fast acl
- use the acl in the cache_peer_access directive

But I still got warning about slow acl in use where fast are required.
I am missing something ?
I saw a proper configuration for something like that in the mailing list
but no longer find it.

Log:

2017/06/20 12:13:37.024 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test ACCESINTERNET") = lookup needed
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test ACCESINTERNET": queueing a call.
2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test ACCESINTERNET": no async support!
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test ACCESINTERNET": return -1.
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test ACCESCHARGEDECOM") = lookup needed
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test ACCESCHARGEDECOM": queueing a call.
2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test ACCESCHARGEDECOM": no async support!
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test ACCESCHARGEDECOM": return -1.
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test INITIAL") = lookup needed
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test INITIAL": queueing a call.
2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test INITIAL": no async support!
2017/06/20 12:13:37.026 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test INITIAL": return -1.

conf:

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 8002        # multiling http
acl Safe_ports port 8080        # multiling http
acl CONNECT method CONNECT
acl AuthorizedUsers proxy_auth REQUIRED
acl StandardUser external ldap_group ACCESINTERNET
acl VIPUser external ldap_group ACCESCHARGEDECOM
acl NoNetUser external ldap_group INITIAL
acl hostnoauth src "/etc/squid/hosts_noauth"
acl urlnoauth url_regex "/etc/squid/urls_noauth"

note profil StdUser StandardUser
note profil VIP VIPUser
note profil NoNet NoNetUser
acl match-StandardUser note profil StdUser
acl match-VIPUser note profil VIP
acl match-NoNetUser note profil NoNet

http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost
http_access allow urlnoauth hostnoauth
http_access allow AuthorizedUsers
http_access deny all
http_port 3128
http_port 10.10.10.10:8080
http_port 10.10.10.10:8002
http_port 10.10.10.10:8001

nonhierarchical_direct off

cache_peer 10.10.10.10         parent   8080     0  name=server_std
cache_peer 10.10.10.10         parent   8002     0  name=server_vip
cache_peer 10.10.10.10         parent   8002     0  name=server_urlnoauth
cache_peer 127.0.0.1             parent     80     0  name=server_nonet

never_direct allow all
always_direct deny all

cache_peer_access server_std allow match-StandardUser
cache_peer_access server_std deny all
cache_peer_access server_vip allow match-VIPUser
cache_peer_access server_vip deny all
cache_peer_access server_nonet allow match-NoNetUser
cache_peer_access server_nonet deny all
cache_peer_access server_urlnoauth allow urlnoauth
cache_peer_access server_urlnoauth deny all
cache_mem 2048 MB

maximum_object_size_in_memory 50 MB
logformat squid [%tl] %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt

debug_options ALL,2

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

FUSTE Emmanuel
Le 20/06/2017 à 12:55, FUSTE Emmanuel a écrit :

> Hello,
>
> I need to select a cache peer based on the user group.
> As cache_peer_access need a fast acl to have predicable result, I tried to
> - annotate transactions with "note"
> - match the annotation with a fast acl
> - use the acl in the cache_peer_access directive
>
> But I still got warning about slow acl in use where fast are required.
> I am missing something ?
> I saw a proper configuration for something like that in the mailing list
> but no longer find it.
>
> Log:
>
> 2017/06/20 12:13:37.024 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test ACCESINTERNET") = lookup needed
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test ACCESINTERNET": queueing a call.
> 2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test ACCESINTERNET": no async support!
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test ACCESINTERNET": return -1.
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test ACCESCHARGEDECOM") = lookup needed
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test ACCESCHARGEDECOM": queueing a call.
> 2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test ACCESCHARGEDECOM": no async support!
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test ACCESCHARGEDECOM": return -1.
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(788) aclMatchExternal: ldap_group("anne.test INITIAL") = lookup needed
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(791) aclMatchExternal: "anne.test INITIAL": queueing a call.
> 2017/06/20 12:13:37.025 kid1| 28,2| Checklist.cc(123) goAsync: 0x7ffde8afc0e0 a fast-only directive uses a slow ACL!
> 2017/06/20 12:13:37.025 kid1| 82,2| external_acl.cc(793) aclMatchExternal: "anne.test INITIAL": no async support!
> 2017/06/20 12:13:37.026 kid1| 82,2| external_acl.cc(794) aclMatchExternal: "anne.test INITIAL": return -1.
>
> conf:
>
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl Safe_ports port 8002        # multiling http
> acl Safe_ports port 8080        # multiling http
> acl CONNECT method CONNECT
> acl AuthorizedUsers proxy_auth REQUIRED
> acl StandardUser external ldap_group ACCESINTERNET
> acl VIPUser external ldap_group ACCESCHARGEDECOM
> acl NoNetUser external ldap_group INITIAL
> acl hostnoauth src "/etc/squid/hosts_noauth"
> acl urlnoauth url_regex "/etc/squid/urls_noauth"
>
> note profil StdUser StandardUser
> note profil VIP VIPUser
> note profil NoNet NoNetUser
> acl match-StandardUser note profil StdUser
> acl match-VIPUser note profil VIP
> acl match-NoNetUser note profil NoNet
>
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost
> http_access allow urlnoauth hostnoauth
> http_access allow AuthorizedUsers
> http_access deny all
> http_port 3128
--------
> http_port 10.10.10.10:8080
> http_port 10.10.10.10:8002
> http_port 10.10.10.10:8001
Forget this block : anonymization  error....
-------

>
> nonhierarchical_direct off
>
> cache_peer 10.10.10.10         parent   8080     0  name=server_std
> cache_peer 10.10.10.10         parent   8002     0  name=server_vip
> cache_peer 10.10.10.10         parent   8002     0  name=server_urlnoauth
> cache_peer 127.0.0.1             parent     80     0  name=server_nonet
>
> never_direct allow all
> always_direct deny all
>
> cache_peer_access server_std allow match-StandardUser
> cache_peer_access server_std deny all
> cache_peer_access server_vip allow match-VIPUser
> cache_peer_access server_vip deny all
> cache_peer_access server_nonet allow match-NoNetUser
> cache_peer_access server_nonet deny all
> cache_peer_access server_urlnoauth allow urlnoauth
> cache_peer_access server_urlnoauth deny all
> cache_mem 2048 MB
>
> maximum_object_size_in_memory 50 MB
> logformat squid [%tl] %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt
>
> debug_options ALL,2
>
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

Amos Jeffries
Administrator
In reply to this post by FUSTE Emmanuel
On 20/06/17 22:55, FUSTE Emmanuel wrote:

> Hello,
>
> I need to select a cache peer based on the user group.
> As cache_peer_access need a fast acl to have predicable result, I tried to
> - annotate transactions with "note"
> - match the annotation with a fast acl
> - use the acl in the cache_peer_access directive
>
> But I still got warning about slow acl in use where fast are required.
> I am missing something ?

The 'note' directive (different from the note ACL type) itself is a
"fast" access control whose purpose is to add things into the log file.
It only does its thing at the termination of a transaction right before
logging.


What you are wanting is to alter the external_acl_type helper (or write
a script wrapper for it that changes the output). Such that when Squid
sends it a lookup it generates an response to Squid saying something
like this:

  OK profil="$group_name"

(where $group_name, is the group which matched)


When that is working you can also vastly simplify your squid.conf by
replacing all these:

   acl StandardUser external ldap_group ACCESINTERNET
   acl VIPUser external ldap_group ACCESCHARGEDECOM
   acl NoNetUser external ldap_group INITIAL

... with a single helper ACL test:
   acl group external ldap_group ACCESINTERNET ACCESCHARGEDECOM INITIAL

... which gets run only for authenticated users:
   http_access deny !AuthorizedUsers
   http_access allow group

... and use the note ACLs to do all your other access controls:
   acl StandardUser note profil ACCESINTERNET
   acl VIPUser note profil ACCESCHARGEDECOM
   acl NoNetUser note profil INITIAL



PS.
>
> maximum_object_size_in_memory 50 MB
> logformat squid [%tl] %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt

FYI: please do not try to define that "squid" log format in squid.conf.
Squid does not follow that instruction, and may do unexpected things as
a result. The latest releases will refuse to start if squid.conf
contains these.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

FUSTE Emmanuel
Hello,

Thank you, it help a lot and clarify  things.


Emmanuel.

Le 20/06/2017 à 14:46, Amos Jeffries a écrit :

> On 20/06/17 22:55, FUSTE Emmanuel wrote:
>> Hello,
>>
>> I need to select a cache peer based on the user group.
>> As cache_peer_access need a fast acl to have predicable result, I tried to
>> - annotate transactions with "note"
>> - match the annotation with a fast acl
>> - use the acl in the cache_peer_access directive
>>
>> But I still got warning about slow acl in use where fast are required.
>> I am missing something ?
> The 'note' directive (different from the note ACL type) itself is a
> "fast" access control whose purpose is to add things into the log file.
> It only does its thing at the termination of a transaction right before
> logging.
>
>
> What you are wanting is to alter the external_acl_type helper (or write
> a script wrapper for it that changes the output). Such that when Squid
> sends it a lookup it generates an response to Squid saying something
> like this:
>
>    OK profil="$group_name"
>
> (where $group_name, is the group which matched)
>
>
> When that is working you can also vastly simplify your squid.conf by
> replacing all these:
>
>     acl StandardUser external ldap_group ACCESINTERNET
>     acl VIPUser external ldap_group ACCESCHARGEDECOM
>     acl NoNetUser external ldap_group INITIAL
>
> ... with a single helper ACL test:
>     acl group external ldap_group ACCESINTERNET ACCESCHARGEDECOM INITIAL
>
> ... which gets run only for authenticated users:
>     http_access deny !AuthorizedUsers
>     http_access allow group
>
> ... and use the note ACLs to do all your other access controls:
>     acl StandardUser note profil ACCESINTERNET
>     acl VIPUser note profil ACCESCHARGEDECOM
>     acl NoNetUser note profil INITIAL
>
>
>
> PS.
>> maximum_object_size_in_memory 50 MB
>> logformat squid [%tl] %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt
> FYI: please do not try to define that "squid" log format in squid.conf.
> Squid does not follow that instruction, and may do unexpected things as
> a result. The latest releases will refuse to start if squid.conf
> contains these.
>
>
> Amos
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

FUSTE Emmanuel
Hello,

One more question to be sure to understand some details:

> Le 20/06/2017 à 14:46, Amos Jeffries a écrit :
>> On 20/06/17 22:55, FUSTE Emmanuel wrote:
>>> Hello,
>>>
>>> I need to select a cache peer based on the user group.
>>> As cache_peer_access need a fast acl to have predicable result, I tried to
>>> - annotate transactions with "note"
>>> - match the annotation with a fast acl
>>> - use the acl in the cache_peer_access directive
>>>
>>> But I still got warning about slow acl in use where fast are required.
>>> I am missing something ?
>> The 'note' directive (different from the note ACL type) itself is a
>> "fast" access control whose purpose is to add things into the log file.
>> It only does its thing at the termination of a transaction right before
>> logging.
>>
>>
>> What you are wanting is to alter the external_acl_type helper (or write
>> a script wrapper for it that changes the output). Such that when Squid
>> sends it a lookup it generates an response to Squid saying something
>> like this:
>>
>>     OK profil="$group_name"
>>
>> (where $group_name, is the group which matched)
>>
>>
>> When that is working you can also vastly simplify your squid.conf by
>> replacing all these:
>>
>>      acl StandardUser external ldap_group ACCESINTERNET
>>      acl VIPUser external ldap_group ACCESCHARGEDECOM
>>      acl NoNetUser external ldap_group INITIAL
>>
>> ... with a single helper ACL test:
>>      acl group external ldap_group ACCESINTERNET ACCESCHARGEDECOM INITIAL
>>
>> ... which gets run only for authenticated users:
>>      http_access deny !AuthorizedUsers
>>      http_access allow group
>>
>> ... and use the note ACLs to do all your other access controls:
>>      acl StandardUser note profil ACCESINTERNET
>>      acl VIPUser note profil ACCESCHARGEDECOM
>>      acl NoNetUser note profil INITIAL
So arbitrary k- v pair not used by the acl helper protocol could be
matched against with the note acl ?
How it relate to the defined/reserved tag= and clt_conn_tag= keywords of
the acl helper protocol ?

The helper is modified to return profil="$group_name" when the group
match. It work.
Will test it on a squid instance with note acl tomorrow.

Emmanuel.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

Amos Jeffries
Administrator
On 22/06/17 04:51, FUSTE Emmanuel wrote:
>
> So arbitrary k- v pair not used by the acl helper protocol could be
> matched against with the note acl ?
> How it relate to the defined/reserved tag= and clt_conn_tag= keywords of
> the acl helper protocol ?

They are all attached as annotations on the transaction. The only ones
excluded are the security tokens in Kerberos/NTLM/Digest auth.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: annotation and fast / slow acl

FUSTE Emmanuel
Le 21/06/2017 à 19:17, Amos Jeffries a écrit :

> On 22/06/17 04:51, FUSTE Emmanuel wrote:
>> So arbitrary k- v pair not used by the acl helper protocol could be
>> matched against with the note acl ?
>> How it relate to the defined/reserved tag= and clt_conn_tag= keywords of
>> the acl helper protocol ?
> They are all attached as annotations on the transaction. The only ones
> excluded are the security tokens in Kerberos/NTLM/Digest auth.
>
> Amos
>
Hello,

One more time, thank you!
All is working beautifully now with annotation  and note acl for peer
selection.

Emmanuel.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...