Quantcast

ext_wbinfo_group_acl is not working

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ext_wbinfo_group_acl is not working

Verónica Ovando

Hi, everybody!


I have my Squid 3.4.8 running in Debian Jessie. It has been working with Active Directory authentication for more than a year without any kind of problem. But since a couple of weeks ago, suddenly, it stopped authenticate users, asking for credentials (username and pass) and they are not able to browse. I am getting this messages in /var/log/cache.log:


2017/03/04 12:04:25.806 kid1| WARNING: external ACL 'Grupos_AD' queue overload. Request rejected 'user1 it_group'.


After some research I found this thread http://www.squid-cache.org/mail-archive/squid-users/200902/0386.html and followed the suggestions posted by Amos. But nothing happened.

I tried rejoining the server to domain. Everything was fine in that way: wbinfo -u, wbinfo -g and wbinfo -P correctly returns all the users, groups and information of the domain.


After restart Squid service, I noticed that neither helper ext_wbinfo_group_acl nor pinger are started:


12:04:01 [root@server ]# systemctl status squid3.service -l
● squid3.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid3)
   Active: active (running) since sáb 2017-03-04 12:04:01 ART; 3s ago
  Process: 4537 ExecStop=/etc/init.d/squid3 stop (code=exited, status=0/SUCCESS)
  Process: 4560 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/squid3.service
           ├─4593 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
           ├─4595 (squid-1) -YC -f /etc/squid3/squid.conf
           └─4596 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
mar 04 12:04:01 server.mydomain.com squid3[4560]: Starting Squid HTTP Proxy 3.x: squid32017/03/04 12:04:01| WARNING: external_acl_type option children=N has been deprecated in favor of children-max=N and children-startup=N
mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: will start 1 kids
mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: (squid-1) process 4595 started
mar 04 12:04:01 server.mydomain.com squid3[4560]: .


12:04:30 [root@server ]# ps fax | grep ext_wbinfo_group_acl
 1418 pts/0    S+     0:00              \_ grep ext_wbinfo_group_acl

If I run echo "mydomain\user1 it_group" | /usr/lib/squid3/ext_wbinfo_group_acl -d, it returns

Debugging mode ON.
Got mydomain\user1 it_group from squid
User:  -mydomain\user1-
Group: -it_group-
SID:   -S-1-5-21-2290000000-711000000-3300000000-3949-
GID:   -10006-
Sending OK to squid
OK

What it's a good, because that user belongs to that group. If I change the group name, it returns an ERR.

Here is my squid.conf:

#===========================================================================
http_port 3128
visible_hostname proxy.squid
cache_mgr [hidden email]
cache_effective_user proxy
error_directory /usr/share/squid3/errors/es
err_page_stylesheet /etc/squid3/estilo.css

####################################################
#******************************Ports*************************************#
####################################################

#acl manager proto cache_object
#acl all src 0.0.0.0/0.0.0.0
#acl localhost src 127.0.0.1/32
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70 #prot gopher
acl Safe_ports port 210 #whais
acl Safe_ports port 280 #http-mgmt
acl Safe_ports port 488 #gss-http
acl Safe_ports port 591 #filemaker
acl Safe_ports port 8080
acl Safe_ports port 2481
acl Safe_ports port 20010
acl Safe_ports port 777 #multi http
#acl purge method PURGE
acl CONNECT method CONNECT

acl_uses_indirect_client on
delay_pool_uses_indirect_client on
log_uses_indirect_client on


##############################################################
#*******************Active Directory HELPERS**************************#
##############################################################

auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
auth_param ntlm children 100
auth_param ntlm keep_alive off

auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
auth_param basic children 100
auth_param basic realm Servidor proxy-cache
auth_param basic credentialsttl 2 hours


#######################################################################
#****************************ACL******************************************#
###########################################################################

#---------------------------ACL Active Directory------------------------#
external_acl_type Grupos_AD ttl=10 negative_ttl=10 children=100 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d
acl it_group external Grupos_AD it_group

------------------Acceso sólo a usuarios autenticados--------------------#
acl auth proxy_auth REQUIRED
http_access deny !auth

#-----------------------------Grupo *it_group*----------------------------#
http_access allow it_group allow

http_access allow manager localhost
http_access deny manager
#http_access allow purge localhost
#http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_PORTS

http_access deny all

dead_peer_timeout 20 seconds
strip_query_terms on
debug_options ALL,1 33,2 28,9
coredump_dir /var/spool/squid3
ftp_passive on
ftp_sanitycheck off
ftp_telnet_protocol off
read_ahead_gap 1 MB
positive_dns_ttl 6 hours
forward_max_tries 25


############################################################################
#*************************Log********************************#
############################################################################

logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
cache_access_log /var/log/squid3/access.log
cache_log /var/log/squid3/cache.log
logfile_rotate 0

############################################################################
#******************Cache and memory***************************#
############################################################################

cache_mem 1024 MB
maximum_object_size_in_memory 1024 KB
memory_cache_mode always
cache_dir aufs /var/spool/squid3 15000 16 256
maximum_object_size 96 MB
minimum_object_size 10 KB
#cache_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
#memory_replacement_policy lru
cache_store_log none
#log_fqdn off
log_icp_queries off
buffered_logs off
#emulate_httpd_log off
redirect_rewrites_host_header off
cache_swap_low 80
cache_swap_high 95

#===========================================================================

It is really weird, I really don't know how to solve this. I hope my explanation was clear.

For testing purposes, I have another Squid working with the same AD server, and it is going fine: the helper and pinger are executed as you can see here:

root@debian-test-server:/etc/squid3# systemctl status squid3.service
● squid3.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid3)
   Active: active (running) since lun 2017-02-13 07:35:01 ART; 2 weeks 5 days ago
  Process: 570 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/squid3.service
           ├─ 1017 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
           ├─ 1020 (squid-1) -YC -f /etc/squid3/squid.conf
           ├─ 1945 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─ 1968 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1969 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1970 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1971 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1972 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1973 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1974 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1993 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─ 2029 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─63477 (pinger)
           ├─63478 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─63479 (ntlm_auth) --helper-protocol=squid-2.5-basic
           └─63480 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d

I will appreciate your help!

Thanks!


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ext_wbinfo_group_acl is not working

Amos Jeffries
Administrator
On 8/03/2017 2:35 a.m., Verónica Ovando wrote:
> Hi, everybody!
>
>
> I have my Squid 3.4.8 running in Debian Jessie. It has been working with Active Directory authentication for more than a year without any kind of problem. But since a couple of weeks ago, suddenly, it stopped authenticate users, asking for credentials (username and pass) and they are not able to browse. I am getting this messages in /var/log/cache.log:
>
>
> 2017/03/04 12:04:25.806 kid1| WARNING: external ACL 'Grupos_AD' queue overload. Request rejected 'user1 it_group'.
>

This means that your AD is not keeping up with the traffic through your
proxy.
Since your Squid has children=100 it will queue up to 200 transactions
waiting for the helper before this message is shown.


Dis cache.log have anything else from the external helper? you have
debug mode enabled (-d) so it should be reporting if there are any
issues with AD other than simply slowness.


>
> After some research I found this thread http://www.squid-cache.org/mail-archive/squid-users/200902/0386.html and followed the suggestions posted by Amos. But nothing happened.
>
> I tried rejoining the server to domain. Everything was fine in that way: wbinfo -u, wbinfo -g and wbinfo -P correctly returns all the users, groups and information of the domain.
>
>
> After restart Squid service, I noticed that neither helper ext_wbinfo_group_acl nor pinger are started:
>
>
> 12:04:01 [root@server ]# systemctl status squid3.service -l

NOTE: do not trust systemd information about Squid-3. The two are not
compatible and systemd often says incorrect things because it makes
incorrect assumptions about the squid process(es). Especially if there
has been a process crash and auto-restart at any point during Squid
operation.


> ● squid3.service - LSB: Squid HTTP Proxy version 3.x
>    Loaded: loaded (/etc/init.d/squid3)
>    Active: active (running) since sáb 2017-03-04 12:04:01 ART; 3s ago
>   Process: 4537 ExecStop=/etc/init.d/squid3 stop (code=exited, status=0/SUCCESS)
>   Process: 4560 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
>    CGroup: /system.slice/squid3.service
>            ├─4593 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
>            ├─4595 (squid-1) -YC -f /etc/squid3/squid.conf
>            └─4596 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
> mar 04 12:04:01 server.mydomain.com squid3[4560]: Starting Squid HTTP Proxy 3.x: squid3
> 2017/03/04 12:04:01| WARNING: external_acl_type option children=N has been deprecated in favor of children-max=N and children-startup=N

Please note the warning and update your config file.

> mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: will start 1 kids
> mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: (squid-1) process 4595 started
> mar 04 12:04:01 server.mydomain.com squid3[4560]: .
>
>
> 12:04:30 [root@server ]# ps fax | grep ext_wbinfo_group_acl
>  1418 pts/0    S+     0:00              \_ grep ext_wbinfo_group_acl
>
> If I run echo "mydomain\user1 it_group" | /usr/lib/squid3/ext_wbinfo_group_acl -d, it returns
>
> Debugging mode ON.
> Got mydomain\user1 it_group from squid
> User:  -mydomain\user1-
> Group: -it_group-
> SID:   -S-1-5-21-2290000000-711000000-3300000000-3949-
> GID:   -10006-
> Sending OK to squid
> OK
>
> What it's a good, because that user belongs to that group. If I change the group name, it returns an ERR.
>
> Here is my squid.conf:
>
> #===========================================================================
> http_port 3128
> visible_hostname proxy.squid
> cache_mgr [hidden email]
> cache_effective_user proxy
> error_directory /usr/share/squid3/errors/es
> err_page_stylesheet /etc/squid3/estilo.css
>
> ####################################################
> #******************************Ports*************************************#
> ####################################################
>
> #acl manager proto cache_object
> #acl all src 0.0.0.0/0.0.0.0
> #acl localhost src 127.0.0.1/32
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 21
> acl Safe_ports port 443
> acl Safe_ports port 70 #prot gopher
> acl Safe_ports port 210 #whais
> acl Safe_ports port 280 #http-mgmt
> acl Safe_ports port 488 #gss-http
> acl Safe_ports port 591 #filemaker
> acl Safe_ports port 8080
> acl Safe_ports port 2481
> acl Safe_ports port 20010
> acl Safe_ports port 777 #multi http
> #acl purge method PURGE
> acl CONNECT method CONNECT
>
> acl_uses_indirect_client on
> delay_pool_uses_indirect_client on
> log_uses_indirect_client on
>
>
> ##############################################################
> #*******************Active Directory HELPERS**************************#
> ##############################################################
>
> auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
> auth_param ntlm children 100
> auth_param ntlm keep_alive off
>
> auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
> auth_param basic children 100
> auth_param basic realm Servidor proxy-cache
> auth_param basic credentialsttl 2 hours
>
>
> #######################################################################
> #****************************ACL******************************************#
> ###########################################################################
>
> #---------------------------ACL Active Directory------------------------#
> external_acl_type Grupos_AD ttl=10 negative_ttl=10 children=100 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d
> acl it_group external Grupos_AD it_group
>
> ------------------Acceso sólo a usuarios autenticados--------------------#
> acl auth proxy_auth REQUIRED
> http_access deny !auth
>
> #-----------------------------Grupo *it_group*----------------------------#
> http_access allow it_group allow

What is this extra "allow" on the end of the line for?

I dont see any ACL named "allow" in the above config. So that may be
preventing Squid from restarting, which would confuse systemd.


>
> http_access allow manager localhost
> http_access deny manager
> #http_access allow purge localhost
> #http_access deny purge

Please move the below two lines up to be the very first http_access
lines in your config. Part of their purpose is to protect against some
DoS conditions which can cause exactly this type of overload on headers.

> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_PORTS
>
> http_access deny all
>
> dead_peer_timeout 20 seconds
> strip_query_terms on
> debug_options ALL,1 33,2 28,9
> coredump_dir /var/spool/squid3
> ftp_passive on
> ftp_sanitycheck off
> ftp_telnet_protocol off
> read_ahead_gap 1 MB
> positive_dns_ttl 6 hours
> forward_max_tries 25
>
>
> ############################################################################
> #*************************Log********************************#
> ############################################################################
>
> logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
> cache_access_log /var/log/squid3/access.log
> cache_log /var/log/squid3/cache.log
> logfile_rotate 0
>
> ############################################################################
> #******************Cache and memory***************************#
> ############################################################################
>
> cache_mem 1024 MB
> maximum_object_size_in_memory 1024 KB
> memory_cache_mode always
> cache_dir aufs /var/spool/squid3 15000 16 256
> maximum_object_size 96 MB
> minimum_object_size 10 KB
> #cache_replacement_policy heap LFUDA
> cache_replacement_policy heap GDSF
> memory_replacement_policy heap GDSF
> #memory_replacement_policy lru
> cache_store_log none
> #log_fqdn off
> log_icp_queries off
> buffered_logs off
> #emulate_httpd_log off
> redirect_rewrites_host_header off
> cache_swap_low 80
> cache_swap_high 95
>
> #===========================================================================
>
> It is really weird, I really don't know how to solve this. I hope my explanation was clear.
>
> For testing purposes, I have another Squid working with the same AD server, and it is going fine: the helper and pinger are executed as you can see here:
>
> root@debian-test-server:/etc/squid3# systemctl status squid3.service
> ● squid3.service - LSB: Squid HTTP Proxy version 3.x
>    Loaded: loaded (/etc/init.d/squid3)
>    Active: active (running) since lun 2017-02-13 07:35:01 ART; 2 weeks 5 days ago
>   Process: 570 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
>    CGroup: /system.slice/squid3.service
>            ├─ 1017 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
>            ├─ 1020 (squid-1) -YC -f /etc/squid3/squid.conf
>            ├─ 1945 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>            ├─ 1968 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1969 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1970 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1971 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1972 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1973 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1974 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─ 1993 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>            ├─ 2029 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>            ├─63477 (pinger)
>            ├─63478 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>            ├─63479 (ntlm_auth) --helper-protocol=squid-2.5-basic
>            └─63480 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>

As configured your Squid should be starting exactly 100 of each - no
more, no less. I suspect from both these traces that you dont actually
need 100 of each helper running, or systemd is confused already.

The current Squid versions can auto-start helpers as needed. See the
auth_param and external_acl_type documentation for the max=, startup=
and idle= options. That may help a little, or at least allow you to
configure higher max limits to cope with slow AD periods.



Another possibility is converting to the LDAP group lookup instead of
using the wbinfo tool to do lookups. I know that LDAP does not suffer
from wbind connection limits, which might be part of your issue.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ext_wbinfo_group_acl is not working

Verónica Ovando
In reply to this post by Verónica Ovando

Hi, everybody!


I have my Squid 3.4.8 running in Debian Jessie. It has been working with Active Directory authentication for more than a year without any kind of problem. But since a couple of weeks ago, suddenly, it stopped authenticate users, asking for credentials (username and pass) and they are not able to browse. I am getting this messages in /var/log/cache.log:


2017/03/04 12:04:25.806 kid1| WARNING: external ACL 'Grupos_AD' queue overload. Request rejected 'user1 it_group'.


After some research I found this thread http://www.squid-cache.org/mail-archive/squid-users/200902/0386.html and followed the suggestions posted by Amos. But nothing happened.

I tried rejoining the server to domain. Everything was fine in that way: wbinfo -u, wbinfo -g and wbinfo -P correctly returns all the users, groups and information of the domain.


After restart Squid service, I noticed that neither helper ext_wbinfo_group_acl nor pinger are started:


12:04:01 [root@server ]# systemctl status squid3.service -l
● squid3.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid3)
   Active: active (running) since sáb 2017-03-04 12:04:01 ART; 3s ago
  Process: 4537 ExecStop=/etc/init.d/squid3 stop (code=exited, status=0/SUCCESS)
  Process: 4560 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/squid3.service
           ├─4593 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
           ├─4595 (squid-1) -YC -f /etc/squid3/squid.conf
           └─4596 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
mar 04 12:04:01 server.mydomain.com squid3[4560]: Starting Squid HTTP Proxy 3.x: squid32017/03/04 12:04:01| WARNING: external_acl_type option children=N has been deprecated in favor of children-max=N and children-startup=N
mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: will start 1 kids
mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: (squid-1) process 4595 started
mar 04 12:04:01 server.mydomain.com squid3[4560]: .


12:04:30 [root@server ]# ps fax | grep ext_wbinfo_group_acl
 1418 pts/0    S+     0:00              \_ grep ext_wbinfo_group_acl

If I run echo "mydomain\user1 it_group" | /usr/lib/squid3/ext_wbinfo_group_acl -d, it returns

Debugging mode ON.
Got mydomain\user1 it_group from squid
User:  -mydomain\user1-
Group: -it_group-
SID:   -S-1-5-21-2290000000-711000000-3300000000-3949-
GID:   -10006-
Sending OK to squid
OK

What it's a good, because that user belongs to that group. If I change the group name, it returns an ERR.

Here is my squid.conf:

#===========================================================================
http_port 3128
visible_hostname proxy.squid
cache_mgr [hidden email]
cache_effective_user proxy
error_directory /usr/share/squid3/errors/es
err_page_stylesheet /etc/squid3/estilo.css

####################################################
#******************************Ports*************************************#
####################################################

#acl manager proto cache_object
#acl all src 0.0.0.0/0.0.0.0
#acl localhost src 127.0.0.1/32
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70 #prot gopher
acl Safe_ports port 210 #whais
acl Safe_ports port 280 #http-mgmt
acl Safe_ports port 488 #gss-http
acl Safe_ports port 591 #filemaker
acl Safe_ports port 8080
acl Safe_ports port 2481
acl Safe_ports port 20010
acl Safe_ports port 777 #multi http
#acl purge method PURGE
acl CONNECT method CONNECT

acl_uses_indirect_client on
delay_pool_uses_indirect_client on
log_uses_indirect_client on


##############################################################
#*******************Active Directory HELPERS**************************#
##############################################################

auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
auth_param ntlm children 100
auth_param ntlm keep_alive off

auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
auth_param basic children 100
auth_param basic realm Servidor proxy-cache
auth_param basic credentialsttl 2 hours


#######################################################################
#****************************ACL******************************************#
###########################################################################

#---------------------------ACL Active Directory------------------------#
external_acl_type Grupos_AD ttl=10 negative_ttl=10 children=100 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d
acl it_group external Grupos_AD it_group

------------------Acceso sólo a usuarios autenticados--------------------#
acl auth proxy_auth REQUIRED
http_access deny !auth

#-----------------------------Grupo *it_group*----------------------------#
http_access allow it_group allow

http_access allow manager localhost
http_access deny manager
#http_access allow purge localhost
#http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_PORTS

http_access deny all

dead_peer_timeout 20 seconds
strip_query_terms on
debug_options ALL,1 33,2 28,9
coredump_dir /var/spool/squid3
ftp_passive on
ftp_sanitycheck off
ftp_telnet_protocol off
read_ahead_gap 1 MB
positive_dns_ttl 6 hours
forward_max_tries 25


############################################################################
#*************************Log********************************#
############################################################################

logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
cache_access_log /var/log/squid3/access.log
cache_log /var/log/squid3/cache.log
logfile_rotate 0

############################################################################
#******************Cache and memory***************************#
############################################################################

cache_mem 1024 MB
maximum_object_size_in_memory 1024 KB
memory_cache_mode always
cache_dir aufs /var/spool/squid3 15000 16 256
maximum_object_size 96 MB
minimum_object_size 10 KB
#cache_replacement_policy heap LFUDA
cache_replacement_policy heap GDSF
memory_replacement_policy heap GDSF
#memory_replacement_policy lru
cache_store_log none
#log_fqdn off
log_icp_queries off
buffered_logs off
#emulate_httpd_log off
redirect_rewrites_host_header off
cache_swap_low 80
cache_swap_high 95

#===========================================================================

It is really weird, I really don't know how to solve this. I hope my explanation was clear.

For testing purposes, I have another Squid working with the same AD server, and it is going fine: the helper and pinger are executed as you can see here:

root@debian-test-server:/etc/squid3# systemctl status squid3.service
● squid3.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid3)
   Active: active (running) since lun 2017-02-13 07:35:01 ART; 2 weeks 5 days ago
  Process: 570 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/squid3.service
           ├─ 1017 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
           ├─ 1020 (squid-1) -YC -f /etc/squid3/squid.conf
           ├─ 1945 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─ 1968 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1969 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1970 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1971 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1972 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1973 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1974 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─ 1993 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─ 2029 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
           ├─63477 (pinger)
           ├─63478 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
           ├─63479 (ntlm_auth) --helper-protocol=squid-2.5-basic
           └─63480 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d

I also tried reinstalling Squid. Nothing happened.

I will appreciate your help!

Thanks!


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ext_wbinfo_group_acl is not working

Verónica Ovando
In reply to this post by Amos Jeffries
>> Hi, everybody!
>>
>>
>> I have my Squid 3.4.8 running in Debian Jessie. It has been working with Active Directory authentication for more than a year without any kind of problem. But since a couple of weeks ago, suddenly, it stopped authenticate users, asking for credentials (username and pass) and they are not able to browse. I am getting this messages in /var/log/cache.log:
>>
>>
>> 2017/03/04 12:04:25.806 kid1| WARNING: external ACL 'Grupos_AD' queue overload. Request rejected 'user1 it_group'.
>>
>
>This means that your AD is not keeping up with the traffic through your
>proxy.
>Since your Squid has children=100 it will queue up to 200 transactions
>waiting for the helper before this message is shown.
>
>
>Dis cache.log have anything else from the external helper? you have
>debug mode enabled (-d) so it should be reporting if there are any
>issues with AD other than simply slowness.
>
>

Yes. I found this:
14:53:48 [root@server squid3]# tail -f /var/log/squid3/cache.log | grep helper
2017/03/16 14:54:19.527 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.
2017/03/16 14:54:19.532 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.
2017/03/16 14:54:20.743 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.

And this:
2017/03/16 14:53:47.887 kid1| Acl.cc(118) FindByName: ACL::FindByName 'it_group'
2017/03/16 14:53:47.887 kid1| Gadgets.cc(71) aclGetDenyInfoPage: got called for it_group
2017/03/16 14:53:48.028 kid1| Acl.cc(157) matches: checking it_group
2017/03/16 14:53:48.028 kid1| Acl.cc(177) matches: checked: it_group = -1
2017/03/16 14:53:48.028 kid1| Gadgets.cc(103) aclIsProxyAuth: aclIsProxyAuth: called for it_group
2017/03/16 14:53:48.028 kid1| Acl.cc(118) FindByName: ACL::FindByName 'it_group'

>>
>> After some research I found this thread http://www.squid-cache.org/mail-archive/squid-users/200902/0386.html and followed the suggestions posted by Amos. But nothing happened.
>>
>> I tried rejoining the server to domain. Everything was fine in that way: wbinfo -u, wbinfo -g and wbinfo -P correctly returns all the users, groups and information of the domain.
>>
>>
>> After restart Squid service, I noticed that neither helper ext_wbinfo_group_acl nor pinger are started:
>>
>>
>> 12:04:01 [root at server ]# systemctl status squid3.service -l
>
>NOTE: do not trust systemd information about Squid-3. The two are not
>compatible and systemd often says incorrect things because it makes
>incorrect assumptions about the squid process(es). Especially if there
>has been a process crash and auto-restart at any point during Squid
>operation.
>
>
>> ● squid3.service - LSB: Squid HTTP Proxy version 3.x
>>    Loaded: loaded (/etc/init.d/squid3)
>>    Active: active (running) since sáb 2017-03-04 12:04:01 ART; 3s ago
>>   Process: 4537 ExecStop=/etc/init.d/squid3 stop (code=exited, status=0/SUCCESS)
>>   Process: 4560 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
>>    CGroup: /system.slice/squid3.service
>>            ├─4593 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
>>            ├─4595 (squid-1) -YC -f /etc/squid3/squid.conf
>>            └─4596 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>> mar 04 12:04:01 server.mydomain.com squid3[4560]: Starting Squid HTTP Proxy 3.x: squid3
>> 2017/03/04 12:04:01| WARNING: external_acl_type option children=N has been deprecated in favor of children-max=N and children-startup=N
>
>Please note the warning and update your config file.
>
>> mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: will start 1 kids
>> mar 04 12:04:01 server.mydomain.com squid3[4593]: Squid Parent: (squid-1) process 4595 started
>> mar 04 12:04:01 server.mydomain.com squid3[4560]: .
>>
>>
>> 12:04:30 [root at server ]# ps fax | grep ext_wbinfo_group_acl
>>  1418 pts/0    S+     0:00              \_ grep ext_wbinfo_group_acl
>>
>> If I run echo "mydomain\user1 it_group" | /usr/lib/squid3/ext_wbinfo_group_acl -d, it returns
>>
>> Debugging mode ON.
>> Got mydomain\user1 it_group from squid
>> User:  -mydomain\user1-
>> Group: -it_group-
>> SID:   -S-1-5-21-2290000000-711000000-3300000000-3949-
>> GID:   -10006-
>> Sending OK to squid
>> OK
>>
>> What it's a good, because that user belongs to that group. If I change the group name, it returns an ERR.
>>
>> Here is my squid.conf:
>>
>> #===========================================================================
>> http_port 3128
>> visible_hostname proxy.squid
>> cache_mgr server at proxy.com
>> cache_effective_user proxy
>> error_directory /usr/share/squid3/errors/es
>> err_page_stylesheet /etc/squid3/estilo.css
>>
>> ####################################################
>> #******************************Ports*************************************#
>> ####################################################
>>
>> #acl manager proto cache_object
>> #acl all src 0.0.0.0/0.0.0.0
>> #acl localhost src 127.0.0.1/32
>> acl SSL_ports port 443
>> acl Safe_ports port 80
>> acl Safe_ports port 21
>> acl Safe_ports port 443
>> acl Safe_ports port 70 #prot gopher
>> acl Safe_ports port 210 #whais
>> acl Safe_ports port 280 #http-mgmt
>> acl Safe_ports port 488 #gss-http
>> acl Safe_ports port 591 #filemaker
>> acl Safe_ports port 8080
>> acl Safe_ports port 2481
>> acl Safe_ports port 20010
>> acl Safe_ports port 777 #multi http
>> #acl purge method PURGE
>> acl CONNECT method CONNECT
>>
>> acl_uses_indirect_client on
>> delay_pool_uses_indirect_client on
>> log_uses_indirect_client on
>>
>>
>> ##############################################################
>> #*******************Active Directory HELPERS**************************#
>> ##############################################################
>>
>> auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>> auth_param ntlm children 100
>> auth_param ntlm keep_alive off
>>
>> auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
>> auth_param basic children 100
>> auth_param basic realm Servidor proxy-cache
>> auth_param basic credentialsttl 2 hours
>>
>>
>> #######################################################################
>> #****************************ACL******************************************#
>> ###########################################################################
>>
>> #---------------------------ACL Active Directory------------------------#
>> external_acl_type Grupos_AD ttl=10 negative_ttl=10 children=100 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d
>> acl it_group external Grupos_AD it_group
>>
>> ------------------Acceso sólo a usuarios autenticados--------------------#
>> acl auth proxy_auth REQUIRED
>> http_access deny !auth
>>
>> #-----------------------------Grupo *it_group*----------------------------#
>> http_access allow it_group allow
>
>What is this extra "allow" on the end of the line for?
>
>I dont see any ACL named "allow" in the above config. So that may be
>preventing Squid from restarting, which would confuse systemd.
>
>
This is only a mistake. ACL "allow" does not exist. So it should be:
#-----------------------------Grupo *it_group*----------------------------#
http_access allow it_group
>>
>> http_access allow manager localhost
>> http_access deny manager
>> #http_access allow purge localhost
>> #http_access deny purge
>
>Please move the below two lines up to be the very first http_access
>lines in your config. Part of their purpose is to protect against some
>DoS conditions which can cause exactly this type of overload on headers.
>
I'll move it.
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_PORTS
>>
>> http_access deny all
>>
>> dead_peer_timeout 20 seconds
>> strip_query_terms on
>> debug_options ALL,1 33,2 28,9
>> coredump_dir /var/spool/squid3
>> ftp_passive on
>> ftp_sanitycheck off
>> ftp_telnet_protocol off
>> read_ahead_gap 1 MB
>> positive_dns_ttl 6 hours
>> forward_max_tries 25
>>
>>
>> ############################################################################
>> #*************************Log********************************#
>> ############################################################################
>>
>> logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
>> cache_access_log /var/log/squid3/access.log
>> cache_log /var/log/squid3/cache.log
>> logfile_rotate 0
>>
>> ############################################################################
>> #******************Cache and memory***************************#
>> ############################################################################
>>
>> cache_mem 1024 MB
>> maximum_object_size_in_memory 1024 KB
>> memory_cache_mode always
>> cache_dir aufs /var/spool/squid3 15000 16 256
>> maximum_object_size 96 MB
>> minimum_object_size 10 KB
>> #cache_replacement_policy heap LFUDA
>> cache_replacement_policy heap GDSF
>> memory_replacement_policy heap GDSF
>> #memory_replacement_policy lru
>> cache_store_log none
>> #log_fqdn off
>> log_icp_queries off
>> buffered_logs off
>> #emulate_httpd_log off
>> redirect_rewrites_host_header off
>> cache_swap_low 80
>> cache_swap_high 95
>>
>> #===========================================================================
>>
>> It is really weird, I really don't know how to solve this. I hope my explanation was clear.
>>
>> For testing purposes, I have another Squid working with the same AD server, and it is going fine: the helper and pinger are executed as you can see here:
>>
>> root at debian-test-server:/etc/squid3# systemctl status squid3.service
>> ● squid3.service - LSB: Squid HTTP Proxy version 3.x
>>    Loaded: loaded (/etc/init.d/squid3)
>>    Active: active (running) since lun 2017-02-13 07:35:01 ART; 2 weeks 5 days ago
>>   Process: 570 ExecStart=/etc/init.d/squid3 start (code=exited, status=0/SUCCESS)
>>    CGroup: /system.slice/squid3.service
>>            ├─ 1017 /usr/sbin/squid3 -YC -f /etc/squid3/squid.conf
>>            ├─ 1020 (squid-1) -YC -f /etc/squid3/squid.conf
>>            ├─ 1945 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>>            ├─ 1968 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1969 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1970 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1971 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1972 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1973 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1974 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─ 1993 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>>            ├─ 2029 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>>            ├─63477 (pinger)
>>            ├─63478 (ntlm_auth) --helper-protocol=squid-2.5-ntlmssp --DOMAIN=MYDOMAIN
>>            ├─63479 (ntlm_auth) --helper-protocol=squid-2.5-basic
>>            └─63480 /usr/bin/perl -w /usr/lib/squid3/ext_wbinfo_group_acl -d
>>
>
>As configured your Squid should be starting exactly 100 of each - no
>more, no less. I suspect from both these traces that you dont actually
>need 100 of each helper running, or systemd is confused already.
>
>The current Squid versions can auto-start helpers as needed. See the
>auth_param and external_acl_type documentation for the max=, startup=
>and idle= options. That may help a little, or at least allow you to
>configure higher max limits to cope with slow AD periods.
>
>
>

I tried with those params but nothing happens. The helper doesn't  auto-start.
external_acl_type Grupos_AD ttl=10 children-max=10 children-startup=10 children-idle=10 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d

And as suggested in the Squid wiki http://www.squid-cache.org/Doc/config/auth_param/, I used this values for:
auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=RENTAS
auth_param ntlm children 20 startup=0 idle=1
auth_param ntlm keep_alive off

auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
auth_param basic children 5 startup=5 idle=1
auth_param basic realm DPR-proxy
auth_param basic credentialsttl 2 hours

>Another possibility is converting to the LDAP group lookup instead of
>using the wbinfo tool to do lookups. I know that LDAP does not suffer
>from wbind connection limits, which might be part of your issue.
>
>Amos
>
Please Amos provide me further guidance. Cannot find a solution to this. Thanks!
--
Verónica Ovando

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ext_wbinfo_group_acl is not working

Amos Jeffries
Administrator
On 22/03/2017 1:04 a.m., Verónica Ovando wrote:

>>> Hi, everybody!
>>>
>>>
>>> I have my Squid 3.4.8 running in Debian Jessie. It has been working with Active Directory authentication for more than a year without any kind of problem. But since a couple of weeks ago, suddenly, it stopped authenticate users, asking for credentials (username and pass) and they are not able to browse. I am getting this messages in /var/log/cache.log:
>>>
>>>
>>> 2017/03/04 12:04:25.806 kid1| WARNING: external ACL 'Grupos_AD' queue overload. Request rejected 'user1 it_group'.
>>>
>>
>> This means that your AD is not keeping up with the traffic through your
>> proxy.
>> Since your Squid has children=100 it will queue up to 200 transactions
>> waiting for the helper before this message is shown.
>>
>>
>> Dis cache.log have anything else from the external helper? you have
>> debug mode enabled (-d) so it should be reporting if there are any
>> issues with AD other than simply slowness.
>>
>>
>
> Yes. I found this:
> 14:53:48 [root@server squid3]# tail -f /var/log/squid3/cache.log | grep helper
> 2017/03/16 14:54:19.527 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.
> 2017/03/16 14:54:19.532 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.
> 2017/03/16 14:54:20.743 kid1| Acl.cc(62) AuthenticateAcl: returning 2 sending credentials to helper.
>
> And this:
> 2017/03/16 14:53:47.887 kid1| Acl.cc(118) FindByName: ACL::FindByName 'it_group'
> 2017/03/16 14:53:47.887 kid1| Gadgets.cc(71) aclGetDenyInfoPage: got called for it_group
> 2017/03/16 14:53:48.028 kid1| Acl.cc(157) matches: checking it_group
> 2017/03/16 14:53:48.028 kid1| Acl.cc(177) matches: checked: it_group = -1
> 2017/03/16 14:53:48.028 kid1| Gadgets.cc(103) aclIsProxyAuth: aclIsProxyAuth: called for it_group
> 2017/03/16 14:53:48.028 kid1| Acl.cc(118) FindByName: ACL::FindByName 'it_group'
>

Hmm. These lines are still from Squid itself. Note the "kid" portion of
the line entry is only produced by Squid.
I'm not sure about debug on the Samba helper, but debug lines from
helpers will usually show up either no timestamp, maybe with a different
timestamp format, and maybe the helpers binary name where the kid part is.

Anyhow, the above lines seem to indicate the group checking it being
done. Or at least started. Which matches what you said earlier, they are
starting but may be overloading.


<snip>

>>
>> The current Squid versions can auto-start helpers as needed. See the
>> auth_param and external_acl_type documentation for the max=, startup=
>> and idle= options. That may help a little, or at least allow you to
>> configure higher max limits to cope with slow AD periods.
>>
>>
>>
>
> I tried with those params but nothing happens. The helper doesn't  auto-start.
> external_acl_type Grupos_AD ttl=10 children-max=10 children-startup=10 children-idle=10 %LOGIN /usr/lib/squid3/ext_wbinfo_group_acl -d
>
> And as suggested in the Squid wiki http://www.squid-cache.org/Doc/config/auth_param/, I used this values for:
> auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp --DOMAIN=RENTAS
> auth_param ntlm children 20 startup=0 idle=1
> auth_param ntlm keep_alive off
>
> auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
> auth_param basic children 5 startup=5 idle=1
> auth_param basic realm DPR-proxy
> auth_param basic credentialsttl 2 hours
>
>> Another possibility is converting to the LDAP group lookup instead of
>> using the wbinfo tool to do lookups. I know that LDAP does not suffer
>>from wbind connection limits, which might be part of your issue.
>>
>> Amos
>>
> Please Amos provide me further guidance. Cannot find a solution to this. Thanks!

I'm stuck myself now. The info you ave been able to provide does not
seem to contain any useful clues about what the problem might be other
than just the helper queue overloading. It has been a long time since I
went near the Windows stuff so memory has paged out sorry :-(.

Maybe raising the helpers max limit? with these 5 and 10 helper limits I
would expect Squid to only be able to handle 20-150 requests per second.
I usually advice using a limit of ~200 helpers for NTLM related things,
a bit more if you have a busy proxy. The point of the new
max/startup/idle feature is that you can set the startup value low and a
much higher maximum.


One other thing that might help is; with that starup=0 you should not
see any NTLM auth helpers immediately after starting Squid. The first
ntlm_auth helper is only started (from idle=1) when the first HTTP
request that needs credential checking is received. Your checks will
need to take that into consideration.
 (I suspect systemd may not be able to track helpers which are started
after the initial startup process is completed.)

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...