Cache peer help

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Cache peer help

Alejandro Delgado Moreno

Hi,

I need to set up a proxy server to filter the Gateway used by sites.

I’ve created a file called sites.txt, that contains the list of sites that our squid proxy should forward the request to another proxy outside our lan.

If the address typed is not in the list, it should be request by our proxy.

I’ve tried different configurations with peer_cache directive, but haven’t been able to route it successfully because all traffic is going by the peer proxy or by our own Gateway, without having into account the file contents.

 

This is a part of the configuration:

 

acl journals dstdomain "/etc/squid/xx_LIST.txt"

cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default

cache_peer_access proxy-inst.upf.edu allow journals

Does anybody has a similar configuration and share it with me?

 

Regards,

Alex.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Amos Jeffries
Administrator
On 07/06/17 01:19, Alejandro Delgado Moreno wrote:

>
> Hi,
>
> I need to set up a proxy server to filter the Gateway used by sites.
>
> I’ve created a file called sites.txt, that contains the list of sites
> that our squid proxy should forward the request to another proxy
> outside our lan.
>
> If the address typed is not in the list, it should be request by our
> proxy.
>
> I’ve tried different configurations with peer_cache directive, but
> haven’t been able to route it successfully because all traffic is
> going by the peer proxy or by our own Gateway, without having into
> account the file contents.
>
> This is a part of the configuration:
>
> acl journals dstdomain "/etc/squid/xx_LIST.txt"
>
> cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default
>
> cache_peer_access proxy-inst.upf.edu allow journals
>
> Does anybody has a similar configuration and share it with me?
>

In your lines above, you have a cache_peer named "xxx.xxx.xxx.xxx".

Your cache_peer_access rule is applied to a different cache_peer line
containing a peer named "proxy-inst.upf.edu".


Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Alejandro Delgado Moreno
Sorry for this mistake,

It's:

acl journals dstdomain "/etc/squid/xx_LIST.txt"

 cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default

 cache_peer_access xxx.xxx.xxx.xxx allow journals

and it's the same, in both lines.

Regards,

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Amos Jeffries
Sent: martes, 6 de junio de 2017 16:17
To: [hidden email]
Subject: Re: [squid-users] Cache peer help

On 07/06/17 01:19, Alejandro Delgado Moreno wrote:

>
> Hi,
>
> I need to set up a proxy server to filter the Gateway used by sites.
>
> I’ve created a file called sites.txt, that contains the list of sites
> that our squid proxy should forward the request to another proxy
> outside our lan.
>
> If the address typed is not in the list, it should be request by our
> proxy.
>
> I’ve tried different configurations with peer_cache directive, but
> haven’t been able to route it successfully because all traffic is
> going by the peer proxy or by our own Gateway, without having into
> account the file contents.
>
> This is a part of the configuration:
>
> acl journals dstdomain "/etc/squid/xx_LIST.txt"
>
> cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default
>
> cache_peer_access proxy-inst.upf.edu allow journals
>
> Does anybody has a similar configuration and share it with me?
>

In your lines above, you have a cache_peer named "xxx.xxx.xxx.xxx".

Your cache_peer_access rule is applied to a different cache_peer line containing a peer named "proxy-inst.upf.edu".


Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Amos Jeffries
Administrator
On 07/06/17 02:24, Alejandro Delgado Moreno wrote:

> Sorry for this mistake,
>
> It's:
>
> acl journals dstdomain "/etc/squid/xx_LIST.txt"
>
>   cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default
>
>   cache_peer_access xxx.xxx.xxx.xxx allow journals
>
> and it's the same, in both lines.

Okay then the issue is something else, those lines in isolation are
correct for allowing traffic to use that peer, but there are many other
things that may make other routes either required or preferred.

So what is the rest of your squid.conf and can you provide a sample of
the access.log for the traffic going wrong?

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Alejandro Delgado Moreno
Hi Amos,

Here is the squid.conf file:

acl localnet src 172.16.0.0/16

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT


acl journals dstdomain "/etc/squid/UPF_LIST.txt"

cache_peer proxy-inst.upf.edu parent 9090 0 no-query no-digest default

cache_peer_access proxy-inst.upf.edu allow journals
always_direct allow journals


# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS

# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 8881

coredump_dir /var/spool/squid

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320


And this is an extract of the log:

[Thu Jun  8 09:47:15 2017].269     57 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:16 2017].128     57 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:16 2017].331     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:20 2017].258    111 172.18.2.45 TCP_MISS/200 967 POST http://ocsp.usertrust.com/ - HIER_DIRECT/178.255.83.1 application/ocsp-response
[Thu Jun  8 09:47:21 2017].250     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:21 2017].459     47 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:23 2017].744    185 172.18.2.45 TCP_MISS/302 615 GET http://wos.fecyt.es/ - HIER_DIRECT/185.79.129.106 text/html
[Thu Jun  8 09:47:24 2017].005    104 172.18.2.45 TCP_MISS/200 2067 POST http://ss.symcd.com/ - HIER_DIRECT/23.37.171.27 application/ocsp-response
[Thu Jun  8 09:47:25 2017].902   5105 172.18.2.45 TCP_TUNNEL/200 5792 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:27 2017].980     65 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:28 2017].394    211 172.18.2.45 TCP_MISS/200 488 GET http://detectportal.firefox.com/success.txt - HIER_DIRECT/88.221.254.202 text/plain
[Thu Jun  8 09:47:28 2017].786     46 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:28 2017].809   8785 172.18.2.45 TCP_TUNNEL/200 54093 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].120   5106 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].144   5130 172.18.2.45 TCP_TUNNEL/200 332 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].147   5133 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].374   6567 172.18.2.45 TCP_TUNNEL/200 108115 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

As you can see, always is going direct, but when going to idp.fecyt.es should be going through the peer, as the file UPF_LIST.txt has:

https://idp.fecyt.es
https://idp.fecyt.es/
https://idp.fecyt.es/*
 
among other lines.

Regards,

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Amos Jeffries
Sent: martes, 6 de junio de 2017 18:18
To: [hidden email]
Subject: Re: [squid-users] Cache peer help

On 07/06/17 02:24, Alejandro Delgado Moreno wrote:

> Sorry for this mistake,
>
> It's:
>
> acl journals dstdomain "/etc/squid/xx_LIST.txt"
>
>   cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default
>
>   cache_peer_access xxx.xxx.xxx.xxx allow journals
>
> and it's the same, in both lines.

Okay then the issue is something else, those lines in isolation are correct for allowing traffic to use that peer, but there are many other things that may make other routes either required or preferred.

So what is the rest of your squid.conf and can you provide a sample of the access.log for the traffic going wrong?

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Cache peer help

Alejandro Delgado Moreno
In reply to this post by Amos Jeffries
Hi Amos,

Here is the squid.conf file:

acl localnet src 172.16.0.0/16

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT


acl journals dstdomain "/etc/squid/UPF_LIST.txt"

cache_peer proxy-inst.upf.edu parent 9090 0 no-query no-digest default

cache_peer_access proxy-inst.upf.edu allow journals always_direct allow journals


# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS

# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy http_access deny all

# Squid normally listens to port 3128
http_port 8881

coredump_dir /var/spool/squid

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320


And this is an extract of the log:

[Thu Jun  8 09:47:15 2017].269     57 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:16 2017].128     57 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:16 2017].331     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:20 2017].258    111 172.18.2.45 TCP_MISS/200 967 POST http://ocsp.usertrust.com/ - HIER_DIRECT/178.255.83.1 application/ocsp-response
[Thu Jun  8 09:47:21 2017].250     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.204.142 application/ocsp-response
[Thu Jun  8 09:47:21 2017].459     47 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:23 2017].744    185 172.18.2.45 TCP_MISS/302 615 GET http://wos.fecyt.es/ - HIER_DIRECT/185.79.129.106 text/html
[Thu Jun  8 09:47:24 2017].005    104 172.18.2.45 TCP_MISS/200 2067 POST http://ss.symcd.com/ - HIER_DIRECT/23.37.171.27 application/ocsp-response
[Thu Jun  8 09:47:25 2017].902   5105 172.18.2.45 TCP_TUNNEL/200 5792 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:27 2017].980     65 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:28 2017].394    211 172.18.2.45 TCP_MISS/200 488 GET http://detectportal.firefox.com/success.txt - HIER_DIRECT/88.221.254.202 text/plain
[Thu Jun  8 09:47:28 2017].786     46 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Thu Jun  8 09:47:28 2017].809   8785 172.18.2.45 TCP_TUNNEL/200 54093 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].120   5106 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].144   5130 172.18.2.45 TCP_TUNNEL/200 332 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].147   5133 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Thu Jun  8 09:47:30 2017].374   6567 172.18.2.45 TCP_TUNNEL/200 108115 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

As you can see, always is going direct, but when going to idp.fecyt.es should be going through the peer, as the file UPF_LIST.txt has:

https://idp.fecyt.es
https://idp.fecyt.es/
https://idp.fecyt.es/*
 
among other lines.

Regards,

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Amos Jeffries
Sent: martes, 6 de junio de 2017 18:18
To: [hidden email]
Subject: Re: [squid-users] Cache peer help

On 07/06/17 02:24, Alejandro Delgado Moreno wrote:

> Sorry for this mistake,
>
> It's:
>
> acl journals dstdomain "/etc/squid/xx_LIST.txt"
>
>   cache_peer xxx.xxx.xxx.xxx parent 9090 0 no-query no-digest default
>
>   cache_peer_access xxx.xxx.xxx.xxx allow journals
>
> and it's the same, in both lines.

Okay then the issue is something else, those lines in isolation are correct for allowing traffic to use that peer, but there are many other things that may make other routes either required or preferred.

So what is the rest of your squid.conf and can you provide a sample of the access.log for the traffic going wrong?

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Amos Jeffries
Administrator
In reply to this post by Alejandro Delgado Moreno
On 08/06/17 19:51, Alejandro Delgado Moreno wrote:

> Hi Amos,
>
> Here is the squid.conf file:
>
> acl localnet src 172.16.0.0/16
>
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl CONNECT method CONNECT
>
>
> acl journals dstdomain "/etc/squid/UPF_LIST.txt"
>
> cache_peer proxy-inst.upf.edu parent 9090 0 no-query no-digest default
>
> cache_peer_access proxy-inst.upf.edu allow journals
> always_direct allow journals

There you go. Problem #1:  "always_direct allow" prohibits any
cache_peer being used by that request (by requiring that DIRECT be used,
mandatory). Remove that and some of the journal traffic will start going
to the peer.

> And this is an extract of the log:
>
> [Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].120   5106 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].144   5130 172.18.2.45 TCP_TUNNEL/200 332 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].147   5133 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].374   6567 172.18.2.45 TCP_TUNNEL/200 108115 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

CONNECT and a few other things are normally sent DIRECT because that is
way faster than doing another hop.

To make those prefer going through the peer add this line:

   nonhierarchical_direct off

And if that is not enough, you can add "never_direct allow journals" to
forbid DIRECT being used. They will then fail completely if the peer is
not used for any reason.


> As you can see, always is going direct, but when going to idp.fecyt.es should be going through the peer, as the file UPF_LIST.txt has:
>
> https://idp.fecyt.es
> https://idp.fecyt.es/
> https://idp.fecyt.es/*

Your squid.conf said these were being loaded into a dstdomain ACL. But
the above lines are URLs, not domain names.

dstdomain syntax is a domain name with maybe a wildcard to match all
sub-domains. see
<http://wiki.squid-cache.org/SquidFaq/SquidAcl#Squid_doesn.27t_match_my_subdomains>


HTH
Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Alejandro Delgado Moreno
Hi Amos,

I've applied your suggestions, but still every request is sent directly, bypassing the peer proxy for sites specified on file UPF_List.txt:

[Tue Jun 13 13:25:58 2017].905    111 172.18.2.45 TCP_MISS/200 968 POST http://ocsp.usertrust.com/ - HIER_DIRECT/178.255.83.1 application/ocsp-response
[Tue Jun 13 13:26:00 2017].173     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.208.238 application/ocsp-response
[Tue Jun 13 13:26:00 2017].283     47 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
[Tue Jun 13 13:26:00 2017].618    211 172.18.2.45 TCP_TUNNEL/200 5147 CONNECT www.facebook.com:443 - HIER_DIRECT/31.13.90.36 -
[Tue Jun 13 13:26:01 2017].691  65863 172.18.2.43 TCP_TUNNEL/200 4946 CONNECT d.dropbox.com:443 - HIER_DIRECT/162.125.32.5 -
[Tue Jun 13 13:26:03 2017].821     68 172.18.2.45 TCP_MISS/302 615 GET http://wos.fecyt.es/ - HIER_DIRECT/185.79.129.106 text/html
[Tue Jun 13 13:26:04 2017].014     29 172.18.2.45 TCP_MISS/200 2068 POST http://ss.symcd.com/ - HIER_DIRECT/23.37.171.27 application/ocsp-response
[Tue Jun 13 13:26:05 2017].151   5079 172.18.2.45 TCP_TUNNEL/200 404 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:05 2017].239   5163 172.18.2.45 TCP_TUNNEL/200 404 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:08 2017].878  10313 172.18.2.45 TCP_TUNNEL/200 54835 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:10 2017].281   5202 172.18.2.45 TCP_TUNNEL/200 526 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:10 2017].365   5107 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:10 2017].372  10219 172.18.2.45 TCP_TUNNEL/200 38460 CONNECT platform.twitter.com:443 - HIER_DIRECT/199.96.57.6 -
[Tue Jun 13 13:26:10 2017].391   5135 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
[Tue Jun 13 13:26:10 2017].454   6580 172.18.2.45 TCP_TUNNEL/200 106738 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

This is the squid.conf file settings:

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
acl localnet src 172.17.0.0/16
acl localnet src 172.18.0.0/16
acl localnet src 172.16.0.0/16

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

acl journals dstdomain "/etc/squid/UPF_LIST.txt"

cache_peer proxy-inst.upf.edu parent 9090 0 no-query  no-digest default
cache_peer_access proxy-inst.upf.edu allow journals
#originserver name=proxyupf
# dstdomain "/etc/squid/UPF_LIST.txt"
#cache_peer_access server_upf allow upf
#cache_peer_access proxyupf allow upf
#cache_peer_access proxyupf deny all
nonhierarchical_direct off
#never_direct deny upf
never_direct allow journals

#never_direct allow upf

#never_direct deny !upf
#never_direct allow all
#cache_peer_access allow upf
#cache_peer_access deny all

#never_direct allow !upf
#never_direct deny all
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
http_access allow journals
#cache_peer_access proxyupf allow upf
#cache_peer_access proxyupf deny all
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
#http_port 3128
http_port 8881

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

Any other suggestions? Do you need the contents of UPF_LIST.txt?

Regards,

-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Amos Jeffries
Sent: jueves, 8 de junio de 2017 12:55
To: [hidden email]
Subject: Re: [squid-users] Cache peer help

On 08/06/17 19:51, Alejandro Delgado Moreno wrote:

> Hi Amos,
>
> Here is the squid.conf file:
>
> acl localnet src 172.16.0.0/16
>
> acl SSL_ports port 443
> acl Safe_ports port 80          # http
> acl Safe_ports port 21          # ftp
> acl Safe_ports port 443         # https
> acl Safe_ports port 70          # gopher
> acl Safe_ports port 210         # wais
> acl Safe_ports port 1025-65535  # unregistered ports
> acl Safe_ports port 280         # http-mgmt
> acl Safe_ports port 488         # gss-http
> acl Safe_ports port 591         # filemaker
> acl Safe_ports port 777         # multiling http
> acl CONNECT method CONNECT
>
>
> acl journals dstdomain "/etc/squid/UPF_LIST.txt"
>
> cache_peer proxy-inst.upf.edu parent 9090 0 no-query no-digest default
>
> cache_peer_access proxy-inst.upf.edu allow journals always_direct
> allow journals

There you go. Problem #1:  "always_direct allow" prohibits any cache_peer being used by that request (by requiring that DIRECT be used, mandatory). Remove that and some of the journal traffic will start going to the peer.

> And this is an extract of the log:
>
> [Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].094   5079 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].120   5106 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].144   5130 172.18.2.45 TCP_TUNNEL/200 332 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].147   5133 172.18.2.45 TCP_TUNNEL/200 333 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Thu Jun  8 09:47:30 2017].374   6567 172.18.2.45 TCP_TUNNEL/200 108115 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

CONNECT and a few other things are normally sent DIRECT because that is way faster than doing another hop.

To make those prefer going through the peer add this line:

   nonhierarchical_direct off

And if that is not enough, you can add "never_direct allow journals" to forbid DIRECT being used. They will then fail completely if the peer is not used for any reason.


> As you can see, always is going direct, but when going to idp.fecyt.es should be going through the peer, as the file UPF_LIST.txt has:
>
> https://idp.fecyt.es
> https://idp.fecyt.es/
> https://idp.fecyt.es/*

Your squid.conf said these were being loaded into a dstdomain ACL. But the above lines are URLs, not domain names.

dstdomain syntax is a domain name with maybe a wildcard to match all sub-domains. see <http://wiki.squid-cache.org/SquidFaq/SquidAcl#Squid_doesn.27t_match_my_subdomains>


HTH
Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cache peer help

Amos Jeffries
Administrator
On 13/06/17 23:30, Alejandro Delgado Moreno wrote:

> Hi Amos,
>
> I've applied your suggestions, but still every request is sent directly, bypassing the peer proxy for sites specified on file UPF_List.txt:
>
> [Tue Jun 13 13:25:58 2017].905    111 172.18.2.45 TCP_MISS/200 968 POST http://ocsp.usertrust.com/ - HIER_DIRECT/178.255.83.1 application/ocsp-response
> [Tue Jun 13 13:26:00 2017].173     56 172.18.2.45 TCP_MISS/200 874 POST http://clients1.google.com/ocsp - HIER_DIRECT/216.58.208.238 application/ocsp-response
> [Tue Jun 13 13:26:00 2017].283     47 172.18.2.45 TCP_MISS/200 924 POST http://ocsp.digicert.com/ - HIER_DIRECT/93.184.220.29 application/ocsp-response
> [Tue Jun 13 13:26:00 2017].618    211 172.18.2.45 TCP_TUNNEL/200 5147 CONNECT www.facebook.com:443 - HIER_DIRECT/31.13.90.36 -
> [Tue Jun 13 13:26:01 2017].691  65863 172.18.2.43 TCP_TUNNEL/200 4946 CONNECT d.dropbox.com:443 - HIER_DIRECT/162.125.32.5 -
> [Tue Jun 13 13:26:03 2017].821     68 172.18.2.45 TCP_MISS/302 615 GET http://wos.fecyt.es/ - HIER_DIRECT/185.79.129.106 text/html
> [Tue Jun 13 13:26:04 2017].014     29 172.18.2.45 TCP_MISS/200 2068 POST http://ss.symcd.com/ - HIER_DIRECT/23.37.171.27 application/ocsp-response
> [Tue Jun 13 13:26:05 2017].151   5079 172.18.2.45 TCP_TUNNEL/200 404 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:05 2017].239   5163 172.18.2.45 TCP_TUNNEL/200 404 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:08 2017].878  10313 172.18.2.45 TCP_TUNNEL/200 54835 CONNECT www.recursoscientificos.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:10 2017].281   5202 172.18.2.45 TCP_TUNNEL/200 526 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:10 2017].365   5107 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:10 2017].372  10219 172.18.2.45 TCP_TUNNEL/200 38460 CONNECT platform.twitter.com:443 - HIER_DIRECT/199.96.57.6 -
> [Tue Jun 13 13:26:10 2017].391   5135 172.18.2.45 TCP_TUNNEL/200 331 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -
> [Tue Jun 13 13:26:10 2017].454   6580 172.18.2.45 TCP_TUNNEL/200 106738 CONNECT idp.fecyt.es:443 - HIER_DIRECT/185.79.129.106 -

Hmm. Your squid.conf now looks fine to me.

> Any other suggestions? Do you need the contents of UPF_LIST.txt?

I think so, yes. It is the last bit of the config I can think of right
now as maybe problematic.

PS. If it contains anything you want to keep private or is bigger than
50KB then you may mail me off-list.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...