Allow some domains to bypass Squid

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Allow some domains to bypass Squid

Nicolas Kovacs
Hi,

I have Squid setup as a transparent HTTP+HTTPS proxy in my local
network, using SSL-Bump.

The configuration works quite nicely, according to
/var/log/squid/cache.log and /var/log/squid/access.log.

This being said, I am having trouble with a handful of domains like
Github, or my OwnCloud installation. I have an OwnCloud server installed
at https://cloud.microlinux.fr, and everytime I fire up a client, I have
to confirm the use of an untrusted certificate. And on my workstation, I
can't connect to my Github repository anymore. Here's the error I get.

  # git pull
  fatal: unable to access 'https://github.com/kikinovak/centos-
  7-desktop-kde/': Peer's certificate issuer has been marked as not
  trusted by the user.

So I thought the best thing to do is to create an exception for this
handful of domains with issues.

Can I configure some domains to simply bypass the proxy in my current
(transparent) setup? Ideally, the configuration should be able to read a
simple text file containing said domains, something like
/etc/squid/bypass-these-domains.txt. And then these bypass the proxy and
get treated regularly, as if there was no proxy?

Cheers,

Niki
--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Amos Jeffries
Administrator
On 11/03/18 21:07, Nicolas Kovacs wrote:

> Hi,
>
> I have Squid setup as a transparent HTTP+HTTPS proxy in my local
> network, using SSL-Bump.
>
> The configuration works quite nicely, according to
> /var/log/squid/cache.log and /var/log/squid/access.log.
>
> This being said, I am having trouble with a handful of domains like
> Github, or my OwnCloud installation. I have an OwnCloud server installed
> at https://cloud.microlinux.fr, and everytime I fire up a client, I have
> to confirm the use of an untrusted certificate. And on my workstation, I
> can't connect to my Github repository anymore. Here's the error I get.
>
>   # git pull
>   fatal: unable to access 'https://github.com/kikinovak/centos-
>   7-desktop-kde/': Peer's certificate issuer has been marked as not
>   trusted by the user.
>
> So I thought the best thing to do is to create an exception for this
> handful of domains with issues.
>
> Can I configure some domains to simply bypass the proxy in my current
> (transparent) setup? Ideally, the configuration should be able to read a
> simple text file containing said domains, something like
> /etc/squid/bypass-these-domains.txt. And then these bypass the proxy and
> get treated regularly, as if there was no proxy?
>

What you need to start with is switch your thinking from "domains" to
considering things in terms of connections and individual servers. Since
"domain" is a URL concept, and URLs are all hidden inside the encrypted
part of the traffic there is no knowing what that really is until after
decryption.

However when dealing with servers and connections, the connections TLS
SNI can tell you which *server* a client is connecting to and you can
decide to do the splice action based on which servers you are having
trouble with (not domains).

Or better yet, decide even earlier in your NAT system not to send that
traffic to the proxy at all.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
Le 11/03/2018 à 09:24, Amos Jeffries a écrit :

> What you need to start with is switch your thinking from "domains" to
> considering things in terms of connections and individual servers. Since
> "domain" is a URL concept, and URLs are all hidden inside the encrypted
> part of the traffic there is no knowing what that really is until after
> decryption.
>
> However when dealing with servers and connections, the connections TLS
> SNI can tell you which *server* a client is connecting to and you can
> decide to do the splice action based on which servers you are having
> trouble with (not domains).
>
> Or better yet, decide even earlier in your NAT system not to send that
> traffic to the proxy at all.

I'm sorry, but I don't understand what you're saying.

Here's what I want, It's very simple.

Create a text file that contains a list of domains. For example:

  google.com
  hotmail.com
  github.com
  credit-cooperatif.fr

And then all connections that go to anyone of these domains don't get
cached, but simply pass through Squid.

Thanks,

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
In reply to this post by Amos Jeffries
Le 11/03/2018 à 09:24, Amos Jeffries a écrit :

> What you need to start with is switch your thinking from "domains" to
> considering things in terms of connections and individual servers. Since
> "domain" is a URL concept, and URLs are all hidden inside the encrypted
> part of the traffic there is no knowing what that really is until after
> decryption.
>
> However when dealing with servers and connections, the connections TLS
> SNI can tell you which *server* a client is connecting to and you can
> decide to do the splice action based on which servers you are having
> trouble with (not domains).
>
> Or better yet, decide even earlier in your NAT system not to send that
> traffic to the proxy at all.

I tried to formulate your suggestion in my own words and sent it to the
CentOS mailing list, where I'm a regular, since this seems more to be of
an iptables-related problem ("earlier in the NAT system").

Here's my message:

--8<---------------------------------------------------------

Hi,

I'm currently facing a quite tricky problem. Here goes.

I have setup Squid as a transparent HTTP+HTTPS proxy in my local
network. All web traffic gets handed over to Squid by an iptables script
on the server. Here's the relevant section in /etc/squid/squid.conf:

--8<-------------------------------------------------------------
# Ports du proxy
http_port 3130
http_port 3128 intercept
https_port 3129 intercept ssl-bump \
  cert=/etc/squid/ssl_cert/amandine.sandbox.lan.pem \
  generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
--8<-------------------------------------------------------------

And here's the corresponding section of my firewall script:

--8<-------------------------------------------------------------
# Commandes
IPT=/usr/sbin/iptables
SYS=/usr/sbin/sysctl
SERVICE=/usr/sbin/service

# Internet
IFACE_INET=enp2s0

# Réseau local
IFACE_LAN=virbr0
IFACE_LAN_IP=192.168.2.0/24

# Serveur
SERVER_IP=192.168.2.1

...

# Squid
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3128 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3128 -j ACCEPT
$IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \
  --dport 80 -j REDIRECT --to-port 3128
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3129 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3129 -j ACCEPT
$IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \
  --dport 443 -j REDIRECT --to-port 3129
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3130 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3130 -j ACCEPT
--8<-------------------------------------------------------------

This setup works nicely for the vast majority of web sites.

BUT: a handful of sites has some trouble with my local certificate. For
example, I can't sync my local Github repo anymore. Or my local OwnCloud
client spews back a warning message on every startup.

I asked on the Squid mailing list if there was a possibility to create
an exception for a list of domains, so that these can simply bypass the
proxy. The problem is, according to one of the developers, I have to
tackle that problem earlier in the process, e. g. in the firewall setup.

So here's what I want to do, in plain words:

1. Redirect all HTTP traffic (port 80) to port 3128. So far so good.

2. Redirect all HTTPS traffic (port 443) to port 3129. Equally OK.

AND...

3. DO NOT REDIRECT traffic that goes to certain domains, like:

  github.com
  credit-cooperatif.coop
  cloud.microlinux.fr
  squid-cache.org
  etc.

Ideally, these domains should be read from a simple text file.

Any idea how I could do that? I don't even know if this is theoretically
possible.

Cheers,

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Amos Jeffries
Administrator
In reply to this post by Nicolas Kovacs
On 11/03/18 21:33, Nicolas Kovacs wrote:

> Le 11/03/2018 à 09:24, Amos Jeffries a écrit :
>> What you need to start with is switch your thinking from "domains" to
>> considering things in terms of connections and individual servers. Since
>> "domain" is a URL concept, and URLs are all hidden inside the encrypted
>> part of the traffic there is no knowing what that really is until after
>> decryption.
>>
>> However when dealing with servers and connections, the connections TLS
>> SNI can tell you which *server* a client is connecting to and you can
>> decide to do the splice action based on which servers you are having
>> trouble with (not domains).
>>
>> Or better yet, decide even earlier in your NAT system not to send that
>> traffic to the proxy at all.
>
> I'm sorry, but I don't understand what you're saying.
>

Once the traffic arrives at the proxy it MUST be handled. It is too late
to send it elsewhere.

So to actually bypass the proxy you have to not send the TCP packets to
it at all. But the NAT system only works with raw-IPs.


There are many ways to "handle" traffic though. Rejection is one. When
TLS is involved relaying without doing anything (aka splice) is another.

But TLS is a point-to-point security protocol. Its handshakes are
dealing with origin server names. The domain name is a secondary detail
only sometimes available at all.

see
<https://superuser.com/questions/59093/difference-between-host-name-and-domain-name/59094>



> Here's what I want, It's very simple.
>
> Create a text file that contains a list of domains. For example:
>
>   google.com

Talking this as an example;

"google.com" is the public domain that users enter into their browsers
to view a certain website. The browser than does a lot of stuff, and
eventually contacts one of the origin servers for "google.com".

But "google.com" is not actually the name for any of those origin
servers. The server names for Google machines are all inside the
*.1e100.net TLD name, and all their machines answer to many different
"domain names" at the HTTP level (gmail.com, google.com, youtube.com,
googlevideo.com, 1e100.net, ... and many others including all the
country-specific ccTLDs variations on those names, and a lot of common
typos eg "gogle.com").


Your MITM proxy does not receive a URL straight off. It receives a TCP
SYN+ACK packet details. Which contains only the raw-IP for that Google
server. It can lookup the DNS to find out the servers name
 ... something.1e100.net.

If you configured it to handle the TLS handshake (with ssl-bump) it will
receive various representations of that server name in TLS messages.
Which should still be something.1e100.net, usually not "google.com" -
but that depends on whether the client software (Browser or non-Browser)
is properly obeying the requirement that it indicate *server name* in
TLS SNI.
 And also on whether the Google company servers specify the "google.com"
domain as an alias (SubjectAltName) for their TLS certificate from any
particular server (some do, some do not).


All of the above has to happen and be acceptable to the proxy access
controls you have configured before it gets a chance to decrypt the HTTP
message inside the TLS encryption ... and finally find out what URL for
that message is with its domain name.
 Only then can it re-process those access controls for the HTTP(S)
message itself using the actual domain name the client wants.

The above is why we have different "dstdomain" and "ssl::server_name"
ACL types to deal with the different name information available.



>   hotmail.com
>   github.com
>   credit-cooperatif.fr
>
> And then all connections that go to anyone of these domains don't get
> cached, but simply pass through Squid.

The process is not getting anywhere close to caching being relevant. The
error you mentioned earlier is in the TLS handshake part of the process.


HTH
Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
Le 11/03/2018 à 11:17, Amos Jeffries a écrit :
> The process is not getting anywhere close to caching being relevant. The
> error you mentioned earlier is in the TLS handshake part of the process.

I've experimented some more, and I have a partial success. Here, I'm
redirecting all HTTPS traffic *except* the one that goes to my bank:

iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d
www.credit-cooperatif.coop --dport 443 -j REDIRECT --to-port 3129

This works because my bank is hosted on a single IP. As soon as I
replace that with a domain that's hosted on multiple IP's, I get this:

iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.google.com
--dport 443 -j REDIRECT --to-port 3129

# firewall.sh
iptables v1.4.21: ! not allowed with multiple source or destination IP
addresses

So my question is: how can I write an iptables rule (or series of rules)
that redirect all traffic to my proxy, *except* the one going to
<list_of_domains> ?

Cheers,

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Amos Jeffries
Administrator
On 11/03/18 23:54, Nicolas Kovacs wrote:

> Le 11/03/2018 à 11:17, Amos Jeffries a écrit :
>> The process is not getting anywhere close to caching being relevant. The
>> error you mentioned earlier is in the TLS handshake part of the process.
>
> I've experimented some more, and I have a partial success. Here, I'm
> redirecting all HTTPS traffic *except* the one that goes to my bank:
>
> iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d
> www.credit-cooperatif.coop --dport 443 -j REDIRECT --to-port 3129
>
> This works because my bank is hosted on a single IP. As soon as I
> replace that with a domain that's hosted on multiple IP's, I get this:
>
> iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.google.com
> --dport 443 -j REDIRECT --to-port 3129
>
> # firewall.sh
> iptables v1.4.21: ! not allowed with multiple source or destination IP
> addresses
>
> So my question is: how can I write an iptables rule (or series of rules)
> that redirect all traffic to my proxy, *except* the one going to
> <list_of_domains> ?

The whois system can provide info on the IP ranges owned by the
companies like Google which own their own ranges.


The alternative for ssl-bump is the splice action. For that you only
need to know the server names each company uses.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
Le 11/03/2018 à 12:31, Amos Jeffries a écrit :
> The whois system can provide info on the IP ranges owned by the
> companies like Google which own their own ranges.
>
>
> The alternative for ssl-bump is the splice action. For that you only
> need to know the server names each company uses.

OK, I got something that's starting to work.

# Exceptions
EXCEPTIONS=$(egrep -v '(^\#)|(^\s+$)' /usr/local/sbin/no-proxy.txt)
for EXCEPTION in $EXCEPTIONS; do
  $IPT -A PREROUTING -t nat -i $IFACE_LAN -d $EXCEPTION -j ACCEPT
done

# Squid
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3128 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3128 -j ACCEPT
$IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \
  --dport 80 -j REDIRECT --to-port 3128
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3129 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3129 -j ACCEPT
$IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \
  --dport 443 -j REDIRECT --to-port 3129
$IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3130 -j ACCEPT
$IPT -A INPUT -p udp -i $IFACE_LAN --dport 3130 -j ACCEPT


And here's what the no-proxy.txt file looks like:

# Ne pas utiliser le proxy pour les domaines suivants
#
# Crédit Coopératif
www.credit-cooperatif.coop
# Github
github.com
# Microlinux
microlinux.fr
microlinux.eu
# Squid
squid-cache.org
# Thunderbird
start.thunderbird.net

So far, it works fine.

Any suggestions ?

Niki


--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
In reply to this post by Amos Jeffries
Le 11/03/2018 à 12:31, Amos Jeffries a écrit :
> The whois system can provide info on the IP ranges owned by the
> companies like Google which own their own ranges.
>
>
> The alternative for ssl-bump is the splice action. For that you only
> need to know the server names each company uses.

I'd say the problem is solved.

I wrote a little blog article to wrap it up.

https://blog.microlinux.fr/squid-exceptions/

Cheers !

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Alex Crow
In reply to this post by Nicolas Kovacs
.
>>
>>
>> The alternative for ssl-bump is the splice action. For that you only
>> need to know the server names each company uses.
>

OP,

It would be a lot easier to just create exceptions on the squid device
for sites where bumping doesn't work which cause then to be tunnelled or
spliced rather then bumped. You can then at least use dstdomain or
ssl:servername rules. dstdomain will let you tunnel or splice, whereas
ssl servername you will only be able to splice as an SSL connection must
already have been started AFAIK. Your firewall will probably need
restarting every time one of the IP addresses behind those hostnames
changes. Squid will at least do a lookup every request for dstdomain
(you need a good DNS server nearby or on the squid box).

BTW, peek/splice/bump is not just install and forget. It needs
maintenance and care in deployment.

Adding transparent into the mix makes it more difficult, as I can see
you have found.

Try to keep the architecture as simple as you can and use each part to
its best ability. Simple firewalls using hostnames for rules is a path
to severe pain where round-robin is in place. Might be OK with a big,
expensive FW appliance that has the ability to DNS lookup for every
connection.

Cheers

Alex


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
Le 11/03/2018 à 16:48, Alex Crow a écrit :

>
> It would be a lot easier to just create exceptions on the squid device
> for sites where bumping doesn't work which cause then to be tunnelled or
> spliced rather then bumped. You can then at least use dstdomain or
> ssl:servername rules. dstdomain will let you tunnel or splice, whereas
> ssl servername you will only be able to splice as an SSL connection must
> already have been started AFAIK. Your firewall will probably need
> restarting every time one of the IP addresses behind those hostnames
> changes. Squid will at least do a lookup every request for dstdomain
> (you need a good DNS server nearby or on the squid box).

What would this configuration look like? Do you have a working example?

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Yuri Voinov
Alex would like to say, splice, when implemented, more easy to
maintenance than iptables/firewall rules.

It's trivial to implement. Here is my config snippet:

# SSL bump rules
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex
"/usr/local/squid/etc/acl.url.nobump"
ssl_bump peek DiscoverSNIHost
ssl_bump splice NoSSLIntercept
ssl_bump bump all

acl.ur.nobump fragment:

# Adobe updates (web installation)
# This requires to splice due to SSL-pinned web-downloader
(get|platformdl|fpdownload|ardownload[0-9])\.adobe\.com
....

As Alex said, splice list require to maintenance all time.

Common rule is:

- Each SSL Pinning site must be spliced.

- Each OCSP stapling site must be spliced.

- Each site could not be bumped should spliced.

Feel free to make RTFM first:

https://wiki.squid-cache.org/Features/SslPeekAndSplice


12.03.2018 00:39, Nicolas Kovacs пишет:

> Le 11/03/2018 à 16:48, Alex Crow a écrit :
>> It would be a lot easier to just create exceptions on the squid device
>> for sites where bumping doesn't work which cause then to be tunnelled or
>> spliced rather then bumped. You can then at least use dstdomain or
>> ssl:servername rules. dstdomain will let you tunnel or splice, whereas
>> ssl servername you will only be able to splice as an SSL connection must
>> already have been started AFAIK. Your firewall will probably need
>> restarting every time one of the IP addresses behind those hostnames
>> changes. Squid will at least do a lookup every request for dstdomain
>> (you need a good DNS server nearby or on the squid box).
> What would this configuration look like? Do you have a working example?
>
> Niki
>
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Yuri Voinov
In reply to this post by Nicolas Kovacs

Also,
feel free to read our config examples here:

https://wiki.squid-cache.org/ConfigExamples


12.03.2018 00:39, Nicolas Kovacs пишет:

> Le 11/03/2018 à 16:48, Alex Crow a écrit :
>> It would be a lot easier to just create exceptions on the squid device
>> for sites where bumping doesn't work which cause then to be tunnelled or
>> spliced rather then bumped. You can then at least use dstdomain or
>> ssl:servername rules. dstdomain will let you tunnel or splice, whereas
>> ssl servername you will only be able to splice as an SSL connection must
>> already have been started AFAIK. Your firewall will probably need
>> restarting every time one of the IP addresses behind those hostnames
>> changes. Squid will at least do a lookup every request for dstdomain
>> (you need a good DNS server nearby or on the squid box).
> What would this configuration look like? Do you have a working example?
>
> Niki
>
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nicolas Kovacs
In reply to this post by Yuri Voinov
Le 11/03/2018 à 19:44, Yuri a écrit :

> It's trivial to implement. Here is my config snippet:
>
> # SSL bump rules
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/acl.url.nobump"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
>
> acl.ur.nobump fragment:
>
> # Adobe updates (web installation)
> # This requires to splice due to SSL-pinned web-downloader
> (get|platformdl|fpdownload|ardownload[0-9])\.adobe\.com

I gave this configuration a spin on my local proxy, and it works great,
without special firewall rules.

Thanks very much! You made my day!

Niki

--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Yuri Voinov
You're welcome ;)

This config works several years on my servers :)


12.03.2018 02:17, Nicolas Kovacs пишет:

> Le 11/03/2018 à 19:44, Yuri a écrit :
>> It's trivial to implement. Here is my config snippet:
>>
>> # SSL bump rules
>> acl DiscoverSNIHost at_step SslBump1
>> acl NoSSLIntercept ssl::server_name_regex
>> "/usr/local/squid/etc/acl.url.nobump"
>> ssl_bump peek DiscoverSNIHost
>> ssl_bump splice NoSSLIntercept
>> ssl_bump bump all
>>
>> acl.ur.nobump fragment:
>>
>> # Adobe updates (web installation)
>> # This requires to splice due to SSL-pinned web-downloader
>> (get|platformdl|fpdownload|ardownload[0-9])\.adobe\.com
> I gave this configuration a spin on my local proxy, and it works great,
> without special firewall rules.
>
> Thanks very much! You made my day!
>
> Niki
>
--
"C++ seems like a language suitable for firing other people's legs."

*****************************
* C++20 : Bug to the future *
*****************************



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (673 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Eliezer Croitoru
In reply to this post by Nicolas Kovacs
Hey Nicolas,

If you are running a squid which doesn't have a mandatory rule of "Block first and then allow" or what in the security industry will be named "up-tight" then Yuri solution is the right path.
But... as a rule of thumb, if you don't need to pass the traffic into the proxy software don’t and allow or block whatever you can on the OS firewall level.
I wrote couple example bypass scripts:
https://gist.github.com/elico/e0faadf0cc63942c5aaade808a87deef
https://gist.github.com/elico/a54c2c8f8e1a2407b42210896b960f4b

For a non router\proxy linux system:
https://gist.github.com/elico/f21dae7a34e1736f56a1995977852460

The above examples are good for pre-known domains similar to the script you wrote in your blog but it gives some form of dynamics to the firewall rules.
I believe that the best formula is to combine both squid splice with ipset and domains resolution and the bypass rules.
Using  squid you will be able to splice domains automatically and with a daily log analysis of squid access.log files you might be able to find new domains that you can add into your firewall level bypassed domains.

Let me know if it sounds good and it worth a wiki article.
Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Nicolas Kovacs
Sent: Sunday, March 11, 2018 10:07
To: [hidden email]
Subject: [squid-users] Allow some domains to bypass Squid

Hi,

I have Squid setup as a transparent HTTP+HTTPS proxy in my local
network, using SSL-Bump.

The configuration works quite nicely, according to
/var/log/squid/cache.log and /var/log/squid/access.log.

This being said, I am having trouble with a handful of domains like
Github, or my OwnCloud installation. I have an OwnCloud server installed
at https://cloud.microlinux.fr, and everytime I fire up a client, I have
to confirm the use of an untrusted certificate. And on my workstation, I
can't connect to my Github repository anymore. Here's the error I get.

  # git pull
  fatal: unable to access 'https://github.com/kikinovak/centos-
  7-desktop-kde/': Peer's certificate issuer has been marked as not
  trusted by the user.

So I thought the best thing to do is to create an exception for this
handful of domains with issues.

Can I configure some domains to simply bypass the proxy in my current
(transparent) setup? Ideally, the configuration should be able to read a
simple text file containing said domains, something like
/etc/squid/bypass-these-domains.txt. And then these bypass the proxy and
get treated regularly, as if there was no proxy?

Cheers,

Niki
--
Microlinux - Solutions informatiques durables
7, place de l'église - 30730 Montpezat
Site : https://www.microlinux.fr
Blog : https://blog.microlinux.fr
Mail : [hidden email]
Tél. : 04 66 63 10 32
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Allow some domains to bypass Squid

Nishant Sharma
In reply to this post by Nicolas Kovacs
Hi Nicolas,

On Sunday 11 March 2018 05:35 PM, Nicolas Kovacs wrote:
> Le 11/03/2018 à 12:31, Amos Jeffries a écrit :
> OK, I got something that's starting to work.
>
> # Exceptions
> EXCEPTIONS=$(egrep -v '(^\#)|(^\s+$)' /usr/local/sbin/no-proxy.txt)
> for EXCEPTION in $EXCEPTIONS; do
>    $IPT -A PREROUTING -t nat -i $IFACE_LAN -d $EXCEPTION -j ACCEPT
> done

The problem with this approach might be that domains are looked up for
their IPs at the time of rule creation and not at the time of request.
Since destinations like github.com, google.com, facebook etc use many
large pools of IPs, your rule might not match later in the day or after
a few days.

Better to use "ipset" along with dnsmasq and refer that ipset in the
iptables rule to match dst.

1. ipset create _ipsetname_ bitmap:ip

2. Configure dnsmasq to populate _ipsetname_ by adding following lines
for each domain to dnsmasq.conf:

ipset=/google.com/_ipsetname_
ipset=/github.com/_ipsetname_
...
...

3. Use dnsmasq as resolver-cache on your proxy machine and ensure that
squid uses your dnsmasq for DNS queries.

4. Add intercept iptables rules to not NAT the traffic  to destination
ipset:

iptables -A PREROUTING -t nat -i $IFACE_LAN -m set --match-set
_ipsetname_ dst -j ACCEPT

Dnsmasq will keep populating the ipset as and when a resolution request
is received for the matched domains. An ipset can hold 65534 entries.

I use this approach extensively to allow Anti-Virus and Windows updates
to the machines which otherwise are not allowed to access Internet
directly without configuring explicit proxy or through proxy.pac/wpad.

Regards,
Nishant
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users