Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

minh hưng đỗ hoàng


Dear Squid-users,
I want to setup a Squid proxy in transparent mode http/https traffic without any config in Client site. 

I use Squid 3.5.20 on Centos7.I just install squid with default feature as yum install squid.

I just do that , but i have some problem with my output logging in access.log .
Specifically, my access.log only show ip_address_server:443 instead domain name of destination server like that :


1511525732.912    206 172.18.18.15 TAG_NONE/200 0 CONNECT 172.217.24.35:443 - ORIGINAL_DST/172.217.24.35 -

I know that i take some mistake in my squid.conf . But i can't find out how to fix it. Could you please show me how to improve my squid.conf .

Here is my squid.conf file in attact file

Waiting for your reply.
Thanks alot !

--
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115



--
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

squid.conf (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

Amos Jeffries
Administrator
On 25/11/17 02:04, minh hưng đỗ hoàng wrote:

>
>
> Dear Squid-users,
> I want to setup a Squid proxy in transparent mode http/https traffic
> without any config in Client site.
>
> I use Squid 3.5.20 on Centos7.I just install squid with default feature
> as *yum install squid.*
> *
> *
> I just do that , but i have some problem with my output logging in
> access.log .
> Specifically, my access.log only show ip_address_server:443 instead
> domain name of destination server like that :
>
>
> *1511525732.912    206 172.18.18.15 TAG_NONE/200 0 CONNECT
> 172.217.24.35:443 - ORIGINAL_DST/172.217.24.35 -*
> *
> *
> I know that i take some mistake in my squid.conf . But i can't find out
> how to fix it. Could you please show me how to improve my squid.conf .
>

You configured "ssl_bump none all".

<https://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions>
"do not use these with Squid-3.5 and newer"


Use this instead:

  acl step1 at_step SslBump1
  ssl_bump peek step1
  ssl_bump splice all


There should be two log entries per HTTPS connection. One before peek
happens with raw-IP:port details. And a second one after peek which may
have a _server_ name (*not* domain name) if and only if the client sends
TLS SNI extension data.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

minh hưng đỗ hoàng
Dear Amos, thank you so much for your quickly reply .
I have tried to replace my SSL config with your suggestion. But my squid get a error like this in cache.log:

2017/11/25 13:21:49 kid1| SECURITY ALERT: Host header forgery detected on local=216.58.199.110:443 remote=172.18.18.15:55704 FD 13 flags=33 (local IP does not match any domain IP)
2017/11/25 13:21:49 kid1| SECURITY ALERT: on URL: apis.google.com:443
2017/11/25 13:21:49 kid1| SECURITY ALERT: Host header forgery detected on local=172.217.25.3:443 remote=172.18.18.15:55705 FD 17 flags=33 (local IP does not match any domain IP)
2017/11/25 13:21:49 kid1| SECURITY ALERT: on URL: www.google.com.vn:443
2017/11/25 13:21:53 kid1| SECURITY ALERT: Host header forgery detected on local=157.240.13.35:443 remote=172.18.18.15:55720 FD 22 flags=33 (local IP does not match any domain IP)
2017/11/25 13:21:53 kid1| SECURITY ALERT: on URL: www.facebook.com:443
2017/11/25 13:21:54 kid1| SECURITY ALERT: Host header forgery detected on local=157.240.13.35:443 remote=172.18.18.15:55724 FD 22 flags=33 (local IP does not match any domain IP)
2017/11/25 13:21:54 kid1| SECURITY ALERT: on URL: www.facebook.com:443

So i can't access www.facebook.com. It's error on my browser : ERR_SSL_PROTOCOL_ERROR

I find out the same issue in this discussion : http://lists.squid-cache.org/pipermail/squid-users/2016-June/011014.html

And then i try to make my squid becomes a cache DNS itself using Unbound. But look like it does'nt work . I get same error before install cache DNS.
Here is my DNS test on my Squid:

[root@localhost ~]# nslookup
Server: 127.0.0.1
Address: 127.0.0.1#53

Non-authoritative answer:
Address: 216.58.203.46

And this is my dns config in squid.config :

# --------- DNS AND IP CACHES [4341]

dns_nameservers 127.0.0.1
dns_v4_first on
#original_dst off
client_dst_passthru off
host_verify_strict off
ignore_unknown_nameservers off
dns_timeout 120 seconds
ipcache_size 1024
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
positive_dns_ttl 6 hours
negative_dns_ttl 300 seconds

Could you help me please :(

2017-11-24 20:27 GMT+07:00 Amos Jeffries <[hidden email]>:
On 25/11/17 02:04, minh hưng đỗ hoàng wrote:


Dear Squid-users,
I want to setup a Squid proxy in transparent mode http/https traffic without any config in Client site.

I use Squid 3.5.20 on Centos7.I just install squid with default feature as *yum install squid.*
*
*
I just do that , but i have some problem with my output logging in access.log .
Specifically, my access.log only show ip_address_server:443 instead domain name of destination server like that :


*1511525732.912    206 172.18.18.15 TAG_NONE/200 0 CONNECT 172.217.24.35:443 - ORIGINAL_DST/172.217.24.35 -*
*
*
I know that i take some mistake in my squid.conf . But i can't find out how to fix it. Could you please show me how to improve my squid.conf .


You configured "ssl_bump none all".

<https://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions>
"do not use these with Squid-3.5 and newer"


Use this instead:

 acl step1 at_step SslBump1
 ssl_bump peek step1
 ssl_bump splice all


There should be two log entries per HTTPS connection. One before peek happens with raw-IP:port details. And a second one after peek which may have a _server_ name (*not* domain name) if and only if the client sends TLS SNI extension data.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users



--
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

Amos Jeffries
Administrator
On 25/11/17 19:40, minh hưng đỗ hoàng wrote:
> Dear Amos, thank you so much for your quickly reply .
> I have tried to replace my SSL config with your suggestion. But my squid
> get a error like this in cache.log:
>
> 2017/11/25 13:21:49 kid1| SECURITY ALERT: Host header forgery detected
> on local=216.58.199.110:443
> remote=172.18.18.15:55704 FD 13 flags=33
> (local IP does not match any domain IP)

...
>
> So i can't access www.facebook.com. It's error
> on my browser : *ERR_SSL_PROTOCOL_ERROR*
> *


> *
> I find out the same issue in this discussion :
> http://lists.squid-cache.org/pipermail/squid-users/2016-June/011014.html
>

The more complete info about that problem, the things to avoid, and the
workarounds that help reduce it can be found at
<https://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery>

Be aware that there is no full solution yet. The latest Squid-4 and
Squid-5 functionality is getting closer to coping with these services,
but still not complete.


> And then i try to make my squid becomes a cache DNS itself using
> Unbound. But look like it does'nt work . I get same error before install
> cache DNS.

Not just the Squid machine but *all* the clients going through your
Squid also have to be using the same DNS resolver for that workaround.
Any of them using other resolvers (eg 8.8.8.8 or similar services)
*will* hit these errors.


> Here is my DNS test on my Squid:
>
> [root@localhost ~]# nslookup google.com
> Server:127.0.0.1
> Address:127.0.0.1#53
>
> Non-authoritative answer:
> Name:google.com
> Address: 216.58.203.46
>

"google.com" is not your problem. The domain names in the log are:

  apis.google.com    != 216.58.199.110
  www.google.com.vn  != 172.217.25.3
  www.facebook.com   != 157.240.13.35

Also, be aware that the problem is extremely temporary. It can change
between failed and working in any random millisecond. So testing even a
few seconds later often shows different results.


> And this is my dns config in squid.config :
>
> # --------- DNS AND IP CACHES [4341]
>
> dns_nameservers 127.0.0.1
> dns_v4_first on
> #original_dst off
> client_dst_passthru off

The above setting is rejecting clients when the host verify fails.
TO let traffic through the proxy when host-verify fails set it back to
the default "client_dst_passthru on".

The Host verify failure is most dangerous when cached - so that is
always prohibited. But upstream routing is difficult for Squid to
determine - thus that config option. It is left up to you whether you
risk your clients getting infected by that mechanism - Squid just
minimizes the damage and risk by limiting it to the one client making
the suspicious request.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

minh hưng đỗ hoàng
Not just the Squid machine but *all* the clients going through your Squid also have to be using the same DNS resolver for that workaround. Any of them using other resolvers (eg 8.8.8.8 or similar services) *will* hit these errors.


And this is my dns config in squid.config :

# --------- DNS AND IP CACHES [4341]

dns_nameservers 127.0.0.1
dns_v4_first on
#original_dst off
client_dst_passthru off

The above setting is rejecting clients when the host verify fails.
TO let traffic through the proxy when host-verify fails set it back to the default "client_dst_passthru on".

The Host verify failure is most dangerous when cached - so that is always prohibited. But upstream routing is difficult for Squid to determine - thus that config option. It is left up to you whether you risk your clients getting infected by that mechanism - Squid just minimizes the damage and risk by limiting it to the one client making the suspicious request.


Thanks alot for your suggestion, i thought that i made some mistake in my DNS. I will try to find out and show you the result.
--
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

minh hưng đỗ hoàng
Dear Amos,
I solved my problem by following this :
1 - I used my Mikrotik router as a cache DNS
2 - Both Squid proxy and my client use Mikrotik' DNS

=> It no more take alert form cache.log

Thanks alot :)
-- 
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

minh hưng đỗ hoàng
Dear Amos,
Sorry for concluded hurriedly.
When i do a test with 1 user, it's seem ok, no more Aler from cache.log. But when i test with more users, the Alert log from cache.log happen again. And so i can't access some https page as chatwork.com , facebook.com

2017/11/29 18:06:41 kid1| SECURITY ALERT: Host header forgery detected on local=54.238.137.130:443 remote=172.16.255.10:61831 FD 131 flags=33 (local IP does not match any domain IP)
2017/11/29 18:06:41 kid1| SECURITY ALERT: on URL: www.chatwork.com:443
2017/11/29 18:06:48 kid1| SECURITY ALERT: Host header forgery detected on local=31.13.95.8:443 remote=172.16.255.51:54984 FD 173 flags=33 (local IP does not match any domain IP)
2017/11/29 18:06:48 kid1| SECURITY ALERT: on URL: api.facebook.com:443
2017/11/29 18:08:07 kid1| SECURITY ALERT: Host header forgery detected on local=31.13.95.12:443 remote=172.16.255.51:54990 FD 51 flags=33 (local IP does not match any domain IP)
2017/11/29 18:08:07 kid1| SECURITY ALERT: on URL: static.xx.fbcdn.net:443
2017/11/29 18:08:50 kid1| SECURITY ALERT: Host header forgery detected on local=172.217.24.197:443 remote=172.16.255.10:61866 FD 34 flags=33 (local IP does not match any domain IP)
2017/11/29 18:08:50 kid1| SECURITY ALERT: on URL: mail.google.com:443
2017/11/29 18:09:43 kid1| SECURITY ALERT: Host header forgery detected on local=13.113.80.172:443 remote=172.16.255.10:61890 FD 124 flags=33 (local IP does not match any domain IP)
2017/11/29 18:09:43 kid1| SECURITY ALERT: on URL: ws-chatwork.pusher.com:443
2017/11/29 18:10:59 kid1| WARNING: 1 swapin MD5 mismatches
2017/11/29 18:11:00 kid1| SECURITY ALERT: Host header forgery detected on local=157.240.15.22:443 remote=172.16.255.51:55032 FD 93 flags=33 (local IP does not match any domain IP)
2017/11/29 18:11:00 kid1| SECURITY ALERT: on URL: connect.facebook.net:443
2017/11/29 18:13:15 kid1| SECURITY ALERT: Host header forgery detected on local=31.13.95.36:443 remote=172.16.255.12:33158 FD 25 flags=33 (local IP does not match any domain IP)
2017/11/29 18:13:15 kid1| SECURITY ALERT: on URL: www.facebook.com:443
2017/11/29 18:14:00 kid1| SECURITY ALERT: Host header forgery detected on local=31.13.95.34:443 remote=172.16.255.59:39526 FD 74 flags=33 (local IP does not match any domain IP)
2017/11/29 18:14:00 kid1| SECURITY ALERT: on URL: mqtt-mini.facebook.com:443


I have a Mikrotik router (172.16.1.1), and some Lan Local. With every Lan, my DHCP allocates DNS, gateway to my LAN. Ext : 172.16.255.0/24 with gateway : 172.16.255.254 and DNS 172.16.255.254
- Mikrotik config with Cache DNS from 8.8.8.8
- Squid use DNS 172.16.1.1 ( Mikrotik DNS)
- Squid config DNS to 172.16.1.1
- Client use DNS allocated by DHCP (but there is still Mikrotik router)

Here is my full squid.conf :

#Allollow LAN Network

# Allow Network ACL Allow/Deny Section#
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl Safe_ports port 1025-65535

acl CONNECT method CONNECT
acl fb dstdomain .facebook.com

#http_access deny CONNECT fb

http_access allow localhost
http_access allow all


# Transparent Proxy Parameters
http_port 3130
http_port 3128 intercept
https_port 3129 intercept ssl-bump generate-host-certificates=off cert=/etc/squid/ssl_cert/squid-3.5.27.pem

### SSL config ###
#-Start-#
#ssl_bump none all
 acl step1 at_step SslBump1
 ssl_bump peek step1
 ssl_bump splice all
#-End-#

# --------- Add X-Forwarded-for in headers [0]? 
#-Start-#
forwarded_for transparent
#-End-#

debug_options ALL,1

log_fqdn on
emulate_httpd_log on
icap_enable on

global_internal_static on
short_icon_urls on
log_uses_indirect_client         on


# --------- DNS AND IP CACHES [4341]

dns_nameservers 172.16.1.1
dns_v4_first on
host_verify_strict off
ignore_unknown_nameservers off
dns_timeout 120 seconds
ipcache_size 1024
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
positive_dns_ttl 6 hours
negative_dns_ttl 300 seconds
---------------------------------------------------------

Could you please help me . Thanks & Best Regards,

2017-11-28 17:32 GMT+07:00 minh hưng đỗ hoàng <[hidden email]>:
Dear Amos,
I solved my problem by following this :
1 - I used my Mikrotik router as a cache DNS
2 - Both Squid proxy and my client use Mikrotik' DNS

=> It no more take alert from cache.log

Thanks alot :)
-- 
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115



--
Thanks & Best Regards,
--------------
Đỗ Hoàng Minh Hưng
Gmail : [hidden email]
SĐT : 01234454115

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: [Squid-3.5.20]Squid transparent proxy http/https without client site config

Amos Jeffries
Administrator
On 30/11/17 01:34, minh hưng đỗ hoàng wrote:
> Dear Amos,
> Sorry for concluded hurriedly.
> When i do a test with 1 user, it's seem ok, no more Aler from cache.log.
> But when i test with more users, the Alert log from cache.log happen
> again. And so i can't access some https page as chatwork.com , facebook.com.


You are understanding that this is a log entry that cannot be completely
removed right? the problem can only be reduced in how much damage is
done, not fixed.

Also be aware that the cache.log records every security event. Even when
the user does not see anything unusual because Squid sends them
transparently to the server they were trying to contact as if the proxy
was not there (real transparency).

You seem to be doing everything that can be done about the connectivity
issues related to that log message.


I suspect that any remaining issues you are now having with those HTTPS
sites is a separate problem with the Squid-3 SSL-Bump code or TLS
protocol itself. You need to take a closer look at the exact
transactions that are going on with those remaining problem sites.

If the problem turns out to be anything in the TLS protocol messages the
'splice' action that your Squid is currently doing means that type of
problem has nothing to do with Squid. It is the client and server
endpoints having the issue between themselves.

You could also try out Squid 3.5.27 or Squid-4 code for a more up to
date SSL-Bump implementation. There are a few changes to how the
connection management works that might show up as weird problems in
Squid-3 despite the splice. Even the 7 months between your 3.5.20 and
3.5.27 has a few of those.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users