squid time out

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

squid time out

ANDRINANTENAINA Avo

Dear all,

 

I have the following setup :

# ../sbin/squid -v                                                                                                       /usr/local/squid/etc

Squid Cache: Version 5.0.0-VCS

Service Name: squid

configure options:  '--with-logdir=/var/log/squid' '--enable-auth-basic=LDAP,PAM,SMB,RADIUS' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-digest=LDAP,eDirectory' '--with-default-user=proxy'

#                                                                                                                        /usr/local/squid/etc

 

I have a huge range in terms of network, but awkwardly, the authentication/ACL and everything works well in one given subnet but not on the others. The users in the other subnets are not able to surf the internet, and this without any specific logs from the proxy side ( the most significant part of the config could be seen below). Any request from these users just times out.  

 

#debug_options 29,9

#dns_nameservers 192.168.0.9 192.168.0.4

#connect_timeout 1  minute

debug_options ALL,9 11,3 20,3

### negotiate kerberos and ntlm authentication

auth_param negotiate program /usr/local/squid/libexec/negotiate_wrapper_auth   -d --ntlm /usr/local/samba/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp  --domain=BCM --kerberos /usr/local/squid/libexec/ext_kerberos_sid_group_acl -d -s GSS_C_NO_NAME

auth_param negotiate children 60

auth_param negotiate keep_alive off

 

### pure ntlm authentication

auth_param ntlm program /usr/local/samba/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp  --domain=KATANA

auth_param ntlm children 60

auth_param ntlm keep_alive off

 

 

# warning: basic authentication sends passwords plaintext

# a network sniffer can and will discover passwords

auth_param basic program /usr/local/samba/bin/ntlm_auth --helper-protocol=squid-2.5-basic

auth_param basic children 60

auth_param basic credentialsttl 4 hours

 

##

auth_param basic program /usr/local/squid/libexec/basic_ldap_auth  -R -b "dc=KATANA,dc=LOCAL" -D [hidden email] -W /usr/local/squid/etc/pass.txt -f sAMAccountName=%s -h 192.168.111.4

auth_param basic children 60

auth_param basic realm Banky Foibe

auth_param basic credentialsttl 1 minute

 

 

acl local0  dst  172.16.0.0/12

acl local1  dst  192.168.0.0/16

http_access allow local0 all

http_access allow local1 all

cache deny local1

cache deny local0

redirector_access deny local0

redirector_access deny local1

 

http_access deny !auth

http_access allow auth

#http_access deny all

http_port 8080

 

I can’t really understand the issue, from the affected networks:

-          The user is able to ping the proxy and access its port 8080 (through telnet / netcat)

-          The request is able to reach the proxy but the in the access_log the “user” is missing

1563455060.396      1 192.168.230.195 TCP_DENIED/407 4714 GET http://api.bing.com/qsml.aspx? - HIER_NONE/- text/html

-          TCP_DENIED/407, requesting the user to go through the authentication phase is presented by the proxy to the user’s browser but nothing happens. I thought that if the timer set to Kerberos, NTLM expires, a pop up should appear but nothing (from wireshark)

GET http://www.bing.com/favicon.ico HTTP/1.1

Accept: */*

UA-CPU: AMD64

Accept-Encoding: gzip, deflate

User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko

Host: www.bing.com

Proxy-Connection: Keep-Alive

 

HTTP/1.1 407 Proxy Authentication Required

Server: squid/5.0.0-VCS

Mime-Version: 1.0

Date: Thu, 18 Jul 2019 10:01:53 GMT

Content-Type: text/html;charset=utf-8

Content-Length: 3733

X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0

Vary: Accept-Language

Content-Language: en

Proxy-Authenticate: Negotiate

Proxy-Authenticate: NTLM

Proxy-Authenticate: Basic realm="KATANA - PERIMETER"

X-Cache: MISS from katana_proxy

Via: 1.1 lichtquanta (squid/5.0.0-VCS)

Connection: close

 

-          On cache.log there is nothing that could mean something, just a bunch of ARP error. Tried to debug the section 29 for authentication … but nothing. Checked the IE internet options, just in case the windows authentication profile is no ticked … but it is there.

I am lost so any help would really be appreciated.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid time out

Amos Jeffries
Administrator
On 19/07/19 1:57 am, ANDRINANTENAINA Avo wrote:
>
> I have a huge range in terms of network, but awkwardly, the
> authentication/ACL and everything works well in one given subnet but not
> on the others. The users in the other subnets are not able to surf the
> internet, and this without any specific logs from the proxy side ( the
> most significant part of the config could be seen below). Any request
> from these users just times out.  ____
>
...

> __ __
>
> I can’t really understand the issue, from the affected networks:____
>
> __-          __The user is able to ping the proxy and access its port
> 8080 (through telnet / netcat) ____
>
> __-          __The request is able to reach the proxy but the in the
> access_log the /“user” /is missing ____
>
> /1563455060.396      1 192.168.230.195 TCP_DENIED/407 4714 GET
> http://api.bing.com/qsml.aspx? - HIER_NONE/- text/html____/
>
> __-          __TCP_DENIED/407, requesting the user to go through the
> authentication phase is presented by the proxy to the user’s browser but
> nothing happens. I thought that if the timer set to Kerberos, NTLM
> expires, a pop up should appear but nothing (from wireshark)____
>

Er. Not sure what you mean by a timer.

The log entry is a reasonable first-request from any client. No sane
client will broadcast user credentials until it knows the receiving
agent needs them - and in what form they are needed.
 That is why your log entry has no username, and the purpose of the 407
status.

Once that 407 is delivered to the Browser that HTTP transaction is over.
If nothing happens afterwards that is a Browser or network layer
problem, nothing to do with Squid. (There are exceptions, but I see no
sign of those being relevant in your config).

Browser popup is what happens if the Browser is _unable_ to find
appropriate user credentials to send the proxy or web server needing
login. If it is able to find any Kerberors, NTLM or Basic auth
credentials to use (in that order of priority) - it will start a new
HTTP transaction using those. Which will be logged as a separate HTTP
transaction.
 But, if those credentials are not able to validate there may not be any
resulting username to log. Your wireshark trace shows no
Proxy-Authorization header in the request, so of course there will be no
username on that transactions log entry.


Setting the timeouts on credentials usability between the DC and the
Browsers will only cause credential tokens to become invalid before they
arrive at the proxy. That can lead to loops of transactions with 407 and
no username logged, especially with NTLM credentials.

Setting any of the auth related TTL or timeouts in squid.conf to short
values will only cause extra work for the auth validation process.
Slowing everything down. It has no effect on whether credentials are
valid, nor what the Browser does.

Despite the PR and marketing MS have done about single-sign-on being a
NTLM thing, it is actually a regular part of all HTTP authentication.
Seeing the popup is a *bad* sign, something is going wrong with the
Browsers auth setup if it has to be bothering the user for details.
 On Windows particularly the Browser should have access to the users
machine login or Kerberos keytab and so use one of those to access the
proxy without bothering or even being noticed by the user at all.

>
> -          On cache.log there is nothing that could mean something, just
> a bunch of ARP error. Tried to debug the section 29 for authentication …
> but nothing. Checked the IE internet options, just in case the windows
> authentication profile is no ticked … but it is there.
>

ARP errors may be nothing, or it could be a sign that your routing needs
something fixed.
 A routing problem might be affecting background connectivity for NTLM
and Kerberos processes the Browser has to do to allocate auth tokens
with DC.
 It might also effect the proxy verifying those tokens, but that would
have a different more obvious error logged.


If the above does not help your troubleshooting, please consider posting
your whole squid.conf.  (Without the #comment lines, and obfuscate
anything like cachemgr_passwd which should not be made public - but in a
way which ensures we can still tell eg that two IPs are different numbers).

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid time out

ANDRINANTENAINA Avo
In reply to this post by ANDRINANTENAINA Avo
Hi Amos, 

Thank you for your prompt reply.

As you said, the first request is hitting the proxy with the "user" field empty, but there is no second request. And I was wrong about the "timer". 
Please find below the config 

auth_param negotiate program /usr/local/squid/libexec/negotiate_wrapper_auth   -d --ntlm /usr/local/samba/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp  --domain=KATANA --kerberos /usr/local/squid/libexec/ext_kerberos_sid_group_acl -d -s GSS_C_NO_NAME

auth_param negotiate children 60

auth_param negotiate keep_alive off

 

auth_param ntlm program /usr/local/samba/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp  --domain=KATANA

auth_param ntlm children 60

auth_param ntlm keep_alive off

 

auth_param basic program /usr/local/samba/bin/ntlm_auth --helper-protocol=squid-2.5-basic

auth_param basic children 60

auth_param basic credentialsttl 4 hours

 

auth_param basic program /usr/local/squid/libexec/basic_ldap_auth  -R -b "dc=KATANA,dc=LOCAL" -D [hidden email] -W /usr/local/squid/etc/pass.txt -f sAMAccountName=%s -h 192.168.111.40

auth_param basic children 60

auth_param basic realm Katana Local

auth_param basic credentialsttl 1 minute

 

acl auth proxy_auth REQUIRED

 

tcp_outgoing_address 0.0.0.0 all

dns_v4_first    on

 

acl mimeblock rep_mime_type ^application/x-shockwave-flash$

http_reply_access deny mimeblock

acl deny_rep_mime_flashvideo rep_mime_type video/flv

http_reply_access deny deny_rep_mime_flashvideo

 

acl local0  dst  172.16.0.0/12

acl local1  dst  192.168.0.0/16

http_access allow local0 all

http_access allow local1 all

cache deny local1

cache deny local0

redirector_access deny local0

redirector_access deny local1

 

http_access deny !auth

http_access allow auth

#http_access deny all

http_port 8080

 

debug_options 29,9

cache_swap_low 94

cache_swap_high 95

logfile_rotate 150

 

cache_dir aufs /media/STORAGE/cache 7000 16 256

cache_log  /media/STORAGE/ACCESS/cache.log

access_log /media/STORAGE/ACCESS/access.log

 

refresh_pattern ^ftp:    1440  20%  10080

refresh_pattern ^gopher:  1440  0%  1440

refresh_pattern -i (/cgi-bin/|\?) 0  0%  0

refresh_pattern .    0  20%  4320

 

acl allsrc src all

acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  8080 3129 1025-65535

acl sslports port 443 563

 

acl purge method PURGE

acl connect method CONNECT

 

acl HTTP proto HTTP

acl HTTPS proto HTTPS

acl allowed_subnets src 192.168.0.0/16

http_access allow allowed_subnets

http_access allow manager localhost

 

http_access deny manager

http_access allow purge localhost

http_access deny purge

http_access deny !safeports

http_access deny CONNECT !sslports

 

http_access allow localhost

 

request_body_max_size 0 KB

delay_pools 1

delay_class 1 2

delay_parameters 1 -1/-1 -1/-1

delay_initial_bucket_level 100

delay_access 1 allow allsrc

 

http_access deny allsrc

 

acl max_user_ip_conn max_user_ip -s 1

http_access deny max_user_ip_conn

deny_info https://192.168.111.111/index3.html  max_user_ip_conn

 

acl Java browser Java/1.4 Java/1.5 Java/1.6 Java/1.7 Java/1.8

http_access allow Java

 

url_rewrite_program /usr/local/ufdbguard/bin/ufdbgclient -l /var/log/squid

url_rewrite_children 64 startup=16 idle=4 concurrency=0

debug_options 28,9

url_rewrite_children 10

 

 

icap_enable on

icap_send_client_ip on

icap_send_client_username on

icap_client_username_encode off

icap_client_username_header X-Authenticated-User

icap_preview_enable on

icap_preview_size 1024

icap_service service_req reqmod_precache bypass=1 icap://127.0.0.1:1345/squidclamav

adaptation_access service_req allow all

icap_service service_resp respmod_precache bypass=1 icap://127.0.0.1:1345/squidclamav

adaptation_access service_resp allow all

 

Thank you 






Message: 1
Date: Fri, 19 Jul 2019 02:59:13 +1200
From: Amos Jeffries <[hidden email]>
To: [hidden email]
Subject: Re: [squid-users] squid time out
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=utf-8

On 19/07/19 1:57 am, ANDRINANTENAINA Avo wrote:
>
> I have a huge range in terms of network, but awkwardly, the
> authentication/ACL and everything works well in one given subnet but not
> on the others. The users in the other subnets are not able to surf the
> internet, and this without any specific logs from the proxy side ( the
> most significant part of the config could be seen below). Any request
> from these users just times out.  ____
>
...

> __ __
>
> I can’t really understand the issue, from the affected networks:____
>
> __-          __The user is able to ping the proxy and access its port
> 8080 (through telnet / netcat) ____
>
> __-          __The request is able to reach the proxy but the in the
> access_log the /“user” /is missing ____
>
> /1563455060.396      1 192.168.230.195 TCP_DENIED/407 4714 GET
> http://api.bing.com/qsml.aspx? - HIER_NONE/- text/html____/
>
> __-          __TCP_DENIED/407, requesting the user to go through the
> authentication phase is presented by the proxy to the user’s browser but
> nothing happens. I thought that if the timer set to Kerberos, NTLM
> expires, a pop up should appear but nothing (from wireshark)____
>

Er. Not sure what you mean by a timer.

The log entry is a reasonable first-request from any client. No sane
client will broadcast user credentials until it knows the receiving
agent needs them - and in what form they are needed.
 That is why your log entry has no username, and the purpose of the 407
status.

Once that 407 is delivered to the Browser that HTTP transaction is over.
If nothing happens afterwards that is a Browser or network layer
problem, nothing to do with Squid. (There are exceptions, but I see no
sign of those being relevant in your config).

Browser popup is what happens if the Browser is _unable_ to find
appropriate user credentials to send the proxy or web server needing
login. If it is able to find any Kerberors, NTLM or Basic auth
credentials to use (in that order of priority) - it will start a new
HTTP transaction using those. Which will be logged as a separate HTTP
transaction.
 But, if those credentials are not able to validate there may not be any
resulting username to log. Your wireshark trace shows no
Proxy-Authorization header in the request, so of course there will be no
username on that transactions log entry.


Setting the timeouts on credentials usability between the DC and the
Browsers will only cause credential tokens to become invalid before they
arrive at the proxy. That can lead to loops of transactions with 407 and
no username logged, especially with NTLM credentials.

Setting any of the auth related TTL or timeouts in squid.conf to short
values will only cause extra work for the auth validation process.
Slowing everything down. It has no effect on whether credentials are
valid, nor what the Browser does.

Despite the PR and marketing MS have done about single-sign-on being a
NTLM thing, it is actually a regular part of all HTTP authentication.
Seeing the popup is a *bad* sign, something is going wrong with the
Browsers auth setup if it has to be bothering the user for details.
 On Windows particularly the Browser should have access to the users
machine login or Kerberos keytab and so use one of those to access the
proxy without bothering or even being noticed by the user at all.

>
> -          On cache.log there is nothing that could mean something, just
> a bunch of ARP error. Tried to debug the section 29 for authentication …
> but nothing. Checked the IE internet options, just in case the windows
> authentication profile is no ticked … but it is there.
>

ARP errors may be nothing, or it could be a sign that your routing needs
something fixed.
 A routing problem might be affecting background connectivity for NTLM
and Kerberos processes the Browser has to do to allocate auth tokens
with DC.
 It might also effect the proxy verifying those tokens, but that would
have a different more obvious error logged.


If the above does not help your troubleshooting, please consider posting
your whole squid.conf.  (Without the #comment lines, and obfuscate
anything like cachemgr_passwd which should not be made public - but in a
way which ensures we can still tell eg that two IPs are different numbers).

Amos



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: squid time out

Amos Jeffries
Administrator
On 19/07/19 5:30 pm, ANDRINANTENAINA Avo wrote:
> Hi Amos, 
>
> Thank you for your prompt reply.
>
> As you said, the first request is hitting the proxy with the "user"
> field empty, but there is no second request. And I was wrong about the
> "timer". 
> Please find below the config

I'm not seeing anything obvious. Though its very hard to read black text
on very dark grey background.

Perhapse it is related to the delays necessary for clamav scanning to
occur or ufdbguard rules.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users