source spoofing without tproxy?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

source spoofing without tproxy?

David Kewley
I want my clients to explicitly address squid as a proxy (not use tproxy), but have squid spoof the source addresses in the forwarded connection, so that further hops know the original source address from the IPv4 headers.

I could find no indication that anyone else has done this, and when I tried various things, I could not get it working.

Is this possible today? If not, is it worth considering as a future feature? Or am I overlooking a reason that this cannot work even in theory?

I got the nearly-equivalent functionality working for reverse proxying using nginx, but so far I've found no way to do it with forward proxying. Nginx doesn't do https forward proxying (no handling of CONNECT).

If squid can't do what I'm looking for today, I would welcome pointers to other possible approaches.

Thanks,
David

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Amos Jeffries
Administrator
On 13/06/17 13:48, David Kewley wrote:

> I want my clients to explicitly address squid as a proxy (not use
> tproxy), but have squid spoof the source addresses in the forwarded
> connection, so that further hops know the original source address from
> the IPv4 headers.
>
> I could find no indication that anyone else has done this, and when I
> tried various things, I could not get it working.
>
> Is this possible today? If not, is it worth considering as a future
> feature? Or am I overlooking a reason that this cannot work even in
> theory?

It is not possible.

No, it is a terrible idea.

It is prohibited by the OS kernel as part of the anti-malware
protections, in this case to prevent the local machine being used to
attack its surrounding network nodes. And by Squid to make it harder to
use Squid as viral payload and damage the brand reputation.


Also, HTTP contains multiplexing and persistent connections. So there is
no particular relation between one incoming/client connection and the
outgoing/server connection(s) the traffic from that client goes out on.
Added to that, a client request may generate multiple outgoing requests
of various types, or Squid may itself generate traffic for its own needs
without any client interaction.

So doing this just degrades the proxy performance. And not in a small
way - intercepted traffic pinning everything as this would need comes
out about 10% nominal (90% reduction), and at the extreme end proxies
with NTLM going through to an origin see only 1% of nominal performance.
Nominal for me being what I clocked a big clients network doing in
real-world traffic a few years back: ~20000 requests per second a few
years back (Squid Project got approx 2x that in controlled lab tests).

>
> I got the nearly-equivalent functionality working for reverse proxying
> using nginx, but so far I've found no way to do it with forward
> proxying. Nginx doesn't do https forward proxying (no handling of
> CONNECT).

So Nginx can be used to attack networks from inside. Good no know we now
have to watch out for that in viral payloads too.

>
> If squid can't do what I'm looking for today, I would welcome pointers
> to other possible approaches.

Squid supports X-Forwarded-For fully - it was invented by Squid devs
back in the day, and Squid is still the authoritative implementation for
how it is supposed to work. As an old feature just about all other HTTP
server and intermediary software have support for that too so you should
have no issue pulling the data out at the receiving end, or in HTTP
processing DPI software / firewalls etc. It is sent on all outgoing
Squid messages unless you explicitly configure something else to happen
with the forwarded_for directive.
  <http://www.squid-cache.org/Doc/config/forwarded_for/>


There is also newer HTTP "Forwarded" header which supercedes
X-Forwarded-For and some very newly written servers might only support
that. Squid lacks the built-in support for that directive so its no good
on received traffic. But if you have to it can be sent to an upstream
server fine with the request_header_add directive, like so:
   request_header_add Forwarded for=%>a


HTH
Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

David Kewley
Thanks for your reply, Amos.

On Mon, Jun 12, 2017 at 9:50 PM, Amos Jeffries <[hidden email]> wrote:
On 13/06/17 13:48, David Kewley wrote:
I want my clients to explicitly address squid as a proxy (not use tproxy), but have squid spoof the source addresses in the forwarded connection, so that further hops know the original source address from the IPv4 headers.

I could find no indication that anyone else has done this, and when I tried various things, I could not get it working.

Is this possible today? If not, is it worth considering as a future feature? Or am I overlooking a reason that this cannot work even in theory?

It is not possible.

No, it is a terrible idea.

It is prohibited by the OS kernel as part of the anti-malware protections, in this case to prevent the local machine being used to attack its surrounding network nodes. And by Squid to make it harder to use Squid as viral payload and damage the brand reputation.

What exactly is the "it" that you're saying is prohibited by the OS kernel? Source spoofing alone, or something else?

Also, HTTP contains multiplexing and persistent connections. So there is no particular relation between one incoming/client connection and the outgoing/server connection(s) the traffic from that client goes out on. Added to that, a client request may generate multiple outgoing requests of various types, or Squid may itself generate traffic for its own needs without any client interaction.

So doing this just degrades the proxy performance. And not in a small way - intercepted traffic pinning everything as this would need comes out about 10% nominal (90% reduction), and at the extreme end proxies with NTLM going through to an origin see only 1% of nominal performance. Nominal for me being what I clocked a big clients network doing in real-world traffic a few years back: ~20000 requests per second a few years back (Squid Project got approx 2x that in controlled lab tests).

Good to know there are strong performance implications, thanks. I don't understand these systems deeply enough to have anticipated this, so I appreciate the heads-up. Too many systems to learn, too quickly...

I got the nearly-equivalent functionality working for reverse proxying using nginx, but so far I've found no way to do it with forward proxying. Nginx doesn't do https forward proxying (no handling of CONNECT).

So Nginx can be used to attack networks from inside. Good no know we now have to watch out for that in viral payloads too.

"Can be used to attack" because of source spoofing, or something else?

If squid can't do what I'm looking for today, I would welcome pointers to other possible approaches.

Squid supports X-Forwarded-For fully - it was invented by Squid devs back in the day, and Squid is still the authoritative implementation for how it is supposed to work. As an old feature just about all other HTTP server and intermediary software have support for that too so you should have no issue pulling the data out at the receiving end, or in HTTP processing DPI software / firewalls etc. It is sent on all outgoing Squid messages unless you explicitly configure something else to happen with the forwarded_for directive.
 <http://www.squid-cache.org/Doc/config/forwarded_for/>

I'll ask the team managing the next-hop device to evaluate that possibility; it looks to me from the docs like it might work. Thanks for the suggestion.

David

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Matus UHLAR - fantomas
In reply to this post by Amos Jeffries
>On 13/06/17 13:48, David Kewley wrote:
>>I want my clients to explicitly address squid as a proxy (not use
>>tproxy), but have squid spoof the source addresses in the forwarded
>>connection, so that further hops know the original source address
>>from the IPv4 headers.
>>
>>I could find no indication that anyone else has done this, and when
>>I tried various things, I could not get it working.
>>
>>Is this possible today? If not, is it worth considering as a future
>>feature? Or am I overlooking a reason that this cannot work even in
>>theory?

On 13.06.17 16:50, Amos Jeffries wrote:
>It is not possible.
>
>No, it is a terrible idea.
>
>It is prohibited by the OS kernel as part of the anti-malware
>protections, in this case to prevent the local machine being used to
>attack its surrounding network nodes. And by Squid to make it harder
>to use Squid as viral payload and damage the brand reputation.

For me to fully understand (I was curious about this some time ago), it is
allowed to fake clients' IPs when intercepting their connections, but not
when connections are done to proxy server directly?

What's the difference that makes it more terrible than spoofing IPs of
intercepted connections?

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Eagles may soar, but weasels don't get sucked into jet engines.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Amos Jeffries
Administrator
On 13/06/17 19:34, Matus UHLAR - fantomas wrote:

>> On 13/06/17 13:48, David Kewley wrote:
>>> I want my clients to explicitly address squid as a proxy (not use
>>> tproxy), but have squid spoof the source addresses in the forwarded
>>> connection, so that further hops know the original source address
>>> from the IPv4 headers.
>>>
>>> I could find no indication that anyone else has done this, and when
>>> I tried various things, I could not get it working.
>>>
>>> Is this possible today? If not, is it worth considering as a future
>>> feature? Or am I overlooking a reason that this cannot work even in
>>> theory?
>
> On 13.06.17 16:50, Amos Jeffries wrote:
>> It is not possible.
>>
>> No, it is a terrible idea.
>>
>> It is prohibited by the OS kernel as part of the anti-malware
>> protections, in this case to prevent the local machine being used to
>> attack its surrounding network nodes. And by Squid to make it harder
>> to use Squid as viral payload and damage the brand reputation.
>
> For me to fully understand (I was curious about this some time ago),
> it is
> allowed to fake clients' IPs when intercepting their connections, but not
> when connections are done to proxy server directly?

Yes.

> What's the difference that makes it more terrible than spoofing IPs of
> intercepted connections?

If you take a close look at the packets you should see the incoming ones
as (client-IP:server-IP:server-port) and outgoing from Squid has
identical (client-IP:server-IP:server-port). Only the src-port differs.
  - As far as the rest of the network is concerned Squid is acting as if
it were a TCP router. The kernel is also configured as a router in order
to do TPROXY, so the whole environment except for the proxy and a few
rules in the networking stack is setup for routing.


By comparison, without TPROXY the incoming packets have
(client-IP:client-port:squid-IP:squid-port) and the server connection
packets would have (client-IP:random-port:server-IP:server-port). The
machine is setup as a server host, not a router. Which is where Ingress
and Egress Filtering both stomp on it hard for using other machines IPs.


PS. if you spoof (with or without TPROXY) and have not implemented
BCP-38 ingress filtering (AND its equivalent egress filtering) on your
network then you are part of the DoS attack system, guaranteed, whether
you know it or not. Very likely you will _not_ be aware of it because
all the information you can log or gather from TCP is spoofed (duh!) and
appears to be your own innocent clients doing things that clients tend
to do (NTP lookups? DNS lookups? sending email? using HTTP? sure no harm
there ... unless it wasn't them).

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Amos Jeffries
Administrator
In reply to this post by David Kewley
On 13/06/17 18:14, David Kewley wrote:

> Thanks for your reply, Amos.
>
> On Mon, Jun 12, 2017 at 9:50 PM, Amos Jeffries <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 13/06/17 13:48, David Kewley wrote:
>
>         I want my clients to explicitly address squid as a proxy (not
>         use tproxy), but have squid spoof the source addresses in the
>         forwarded connection, so that further hops know the original
>         source address from the IPv4 headers.
>
>         I could find no indication that anyone else has done this, and
>         when I tried various things, I could not get it working.
>
>         Is this possible today? If not, is it worth considering as a
>         future feature? Or am I overlooking a reason that this cannot
>         work even in theory?
>
>
>     It is not possible.
>
>     No, it is a terrible idea.
>
>     It is prohibited by the OS kernel as part of the anti-malware
>     protections, in this case to prevent the local machine being used
>     to attack its surrounding network nodes. And by Squid to make it
>     harder to use Squid as viral payload and damage the brand reputation.
>
>
> What exactly is the "it" that you're saying is prohibited by the OS
> kernel? Source spoofing alone, or something else?

The "it" that was the subject of your question "it" - spoofing.

>
>     Also, HTTP contains multiplexing and persistent connections. So
>     there is no particular relation between one incoming/client
>     connection and the outgoing/server connection(s) the traffic from
>     that client goes out on. Added to that, a client request may
>     generate multiple outgoing requests of various types, or Squid may
>     itself generate traffic for its own needs without any client
>     interaction.
>
>     So doing this just degrades the proxy performance. And not in a
>     small way - intercepted traffic pinning everything as this would
>     need comes out about 10% nominal (90% reduction), and at the
>     extreme end proxies with NTLM going through to an origin see only
>     1% of nominal performance. Nominal for me being what I clocked a
>     big clients network doing in real-world traffic a few years back:
>     ~20000 requests per second a few years back (Squid Project got
>     approx 2x that in controlled lab tests).
>
>
> Good to know there are strong performance implications, thanks. I
> don't understand these systems deeply enough to have anticipated this,
> so I appreciate the heads-up. Too many systems to learn, too quickly...
>
>         I got the nearly-equivalent functionality working for reverse
>         proxying using nginx, but so far I've found no way to do it
>         with forward proxying. Nginx doesn't do https forward proxying
>         (no handling of CONNECT).
>
>
>     So Nginx can be used to attack networks from inside. Good no know
>     we now have to watch out for that in viral payloads too.
>
>
> "Can be used to attack" because of source spoofing, or something else?
>

Directly because of source spoofing, or equivalent if you got it sending
from any non-local IP address. "local" meaning an IP address explicitly
assigned for use by the machine the proxy runs on.

This might be of help if you are not already aware of the risks and
issues involved with spoofing and handling of non-local IPs;
<http://www.bcp38.info/>


>         If squid can't do what I'm looking for today, I would welcome
>         pointers to other possible approaches.
>
>
>     Squid supports X-Forwarded-For fully - it was invented by Squid
>     devs back in the day, and Squid is still the authoritative
>     implementation for how it is supposed to work. As an old feature
>     just about all other HTTP server and intermediary software have
>     support for that too so you should have no issue pulling the data
>     out at the receiving end, or in HTTP processing DPI software /
>     firewalls etc. It is sent on all outgoing Squid messages unless
>     you explicitly configure something else to happen with the
>     forwarded_for directive.
>      <http://www.squid-cache.org/Doc/config/forwarded_for/
>     <http://www.squid-cache.org/Doc/config/forwarded_for/>>
>
>
> I'll ask the team managing the next-hop device to evaluate that
> possibility; it looks to me from the docs like it might work. Thanks
> for the suggestion.
>
> David

That would be best if it works.

I came up with a bodgy workaround using NAT after sending the earlier
mail. So if there is no other way than delivering the client-IP on the
packets there is still something that might be done. But, that would
still run up against HTTP multiplexing and also add all sorts of NAT
related issues as well. So only a last resort really.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Negotiate Kerberos Auth - BH Invalid request

CrossfireAUT

Hello list,


I asked about a problem with NTLM-Authentication before. (BH SPNEGO request invalid prefix; thats the error of the helper protocol "helper-protocol=squid-2.5-ntlmssp" I used with NTLM, while basic works fine)

A user told me I should use negotiate_kerberos_auth instead of ntlm_auth.

Now here's my new problem:


root@x-x-testproxy01:/etc/squid# /usr/lib/squid/negotiate_kerberos_auth -d -s HTTP/[hidden email]
negotiate_kerberos_auth.cc(487): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Starting version 3.0.4sq
negotiate_kerberos_auth.cc(546): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Setting keytab to FILE:/etc/squid/HTTP.keytab
negotiate_kerberos_auth.cc(570): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Changed keytab to MEMORY:negotiate_kerberos_auth_5305
testuser xxxxxxx
negotiate_kerberos_auth.cc(610): pid=5305 :2017/06/13 13:29:47| negotiate_kerberos_auth: DEBUG: Got 'testuser xxxxxx' from squid (length: 18).
negotiate_kerberos_auth.cc(647): pid=5305 :2017/06/13 13:29:47| negotiate_kerberos_auth: ERROR: Invalid request [testuser xxxxxxx]
BH Invalid request
So my configuration has mistakes, but I can't find them. I don't really know where to search, or what works for sure. I tried many tutorials on krb5 and samba. Every form of testing I tried works fine except indeed using the required kerberos authentication of my squid-proxy.


Tests that come to my mind:

kinit a user

Warning: Your password will expire in 36 days on Don 20 Jul 2017 13:23:54 CEST



klist

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [hidden email]

Valid starting       Expires              Service principal
2017-06-13 13:38:37  2017-06-13 23:38:37  krbtgt/[hidden email]
    renew until 2017-06-14 13:38:34


klist -k on my HTTP.keytab

Keytab name: FILE:/etc/squid/HTTP.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL

basic-auth using ntlm

root@x-x-testproxy01:/etc/squid# /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic --username=testuser --password=xxxxxxxx
testuser xxxxxxxxxx
OK
[hidden email] xxxxxxxx
OK

wbinfo -u
administrator
testuser
...
wbinfo -g
allowed rodc password replication group
enterprise read-only domain controllers
...

wbinfo --krb5auth=testuser%xxxxxxx
plaintext kerberos password authentication for [testuser%xxxxxxx] succeeded (requesting cctype: FILE)

wbinfo -t
checking the trust secret for domain X-XXX via RPC calls succeeded

wbinfo --authenticate=testuser%xxxxxxxx
plaintext password authentication succeeded
challenge/response password authentication succeeded

/usr/lib/squid/negotiate_kerberos_auth_test x-x-testproxy01.x-xxx.local
Token: YIIFOgYGKwYBBQUCoIIFLjCCBSqgJzAlBgkqhkiG9xIBAgIGBSsFAQUCBgkqhkiC9xIBAgIGBisGAQUCBaKCBP0EggT5YIIE9QYJKoZIhvcSAQICAQBuggTkMIIE4KADAgEFoQMCAQ6iBwMFAAAAAACjggP2YYID8jCCA+6gAwIBBaENGwtYLU5FVC5MT0NBTKIuMCygAwIBA6ElMCMbBEhUVFAbG3gtbC10ZXN0cHJveHkwMS54LW5ldC5sb2NhbKOCA6YwggOioAMCARKhAwIBAaKCA5QEggOQIMtincRDtWjh44pew3twk26Gm9rTC7CbkobNrzaRq/weljVl5TSbMQTFIVRQXVe4CQBWJ/Gcg472cgLA3mjOH8Z30zxQFP8fsK46wAtTEzJhonzXLImhaPtXvCVz94xaCVG7cBlNJCUmZQHsQMxFsGJZfKCkDvztiNplXEEwRgT7S6f8HQNm62xPAyz9aK8Wqfz9suW5cSBk8wdRAQNleKP1Xe/2LqZ4jfDpodPdcy9A8vh1dKu4tmbz+EJ/bKvWA+/twuXiOhhGq4W39TlOu/3zD87pXAh65ka1QsepkCWgUMHImDw8nUr8Zvi4j4vI9WhyMyLFYBya8BvAX9kLg73zl80g82bQVIAb8QU3gS2Akhpd7r1flfUbfRDUQfuS/bsaHspZoP+2c8+Bxy38OML4Gg29y6fJvRNfDaCnqmTQdiPtyqELVUS+4x7r260mM1wQKzD2Jb5pcz4wMHUK0sdw8xmMARGxB7XdyGSbo759GD6tOaTAKkNwccno6i9wyOoLqfhVRjE/K9FLCvCnzEFI07q1/dFz1Ce/ZzroW3nOwNQe3V6qqBELwuTvHgxIdGq4HEPeLqAUkVWxneXemRNbLKiOs/BIe3qkXxizgAkFLqRO5az2pVOD7/KBevxYZKeAgIuDsbIYG/u3Ic+KtCDaaM/to2b41SB8ZOFKJau3BuMPOvZ4ipiMbv0N/Svk4Tg61BIhN3CJYKA3ep/3p3wSfJlkcYeRVkDpasQnmFnjxV2YS7Q8nvmf9LIz+KIYBIT8X9yRPuV/E2lSELZlxJ8CySLFLUKgtMj97GPMlacc+UN3lJjyoExUKHpMZtUmaKrw7ueT3wnLMWgx0BBPkiAebUAedKj9u9sEscFylmI/+PdCCraMNbkOckCsggYXfJm6LxFZJhnDvw1+Z87xsJFDs5fasF6j8REiG8aHTmKHgt2M9TmIRNo/PsYrZGvuVQhkk2fuyFxwwyfs7ysNEkOmBWlFlTEjddjT9YShHfV8Fo8+M2UYY5nYiUIQq5BBfZ679ntivs7F80lKMOqhc7SOY63VwRJOwoq35+bnsIB08b9cttySiOcFsZb6uTnYvHzUFVSUha4nxrg3zW3fL9KVu4XY+lgCRZrBZMxioy6vbAOJBmpqXOJvel2gBbGN6PEd2ReeX43l1gcn+Bd3mQykmUIEzMMuRpSHda9233aWHbwEZ9H9rOdJdhgX4U+upIHQMIHNoAMCARKigcUEgcJu7kcC1zuJdhOQk+YA0Hw2G9kyg46tpaNIgj1CEgkD/KE28kaKivCTZTnHfrNOpIOJaaiYw4RMwJsbgZdRU+fz/jUwxvXdUSGon6JrU7S2XZ9CjXRXfdpXc4HjP0QW6Cql1SE95MpkcbMRH8FQvGNryBzsqIkELnvXceTGCmwlN3n60nqkoR5/41p2PtSz4hFMOVdT6AkPlNC5VTCtCZUj7YbrYVYImPnG3aAfQxXEHRy19/v0mL2845jZFA7Xw96s1A==



Sorry for posting so many output...
I already read many documentations, but no one really tests in small steps, they just assume that it works for everyone out of the box...

Does anyone have a clue what could be my mistake?

Thanks in advance.




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Negotiate Kerberos Auth - BH Invalid request

L.P.H. van Belle
First, it very handy to know your os and samba and squid versions used.
 
Second,
Squid/radius etc anything that uses NTLMv1 with samba stopped working after 4.5.0
I think your main problem can be explained by this extract from the release notes for 4.5.0:
 

NTLMv1 authentication disabled by default

-----------------------------------------

 

In order to improve security we have changed the default value for the "ntlm auth" option from "yes" to "no". 
This may have impact on very old clients which doesn't support NTLMv2 yet.

 

The primary user of NTLMv1 is MSCHAPv2 for VPNs and 802.1x.

 

By default, Samba will only allow NTLMv2 via NTLMSSP now,
as we have the following default "lanman auth = no", "ntlm auth = no" and "raw NTLMv2 auth = no".

 

 

Greetz,

 

Louis

 

 

 


Van: squid-users [mailto:[hidden email]] Namens Kevin M???hlparzer
Verzonden: dinsdag 13 juni 2017 14:00
Aan: [hidden email]
Onderwerp: [squid-users] Negotiate Kerberos Auth - BH Invalid request

Hello list,


I asked about a problem with NTLM-Authentication before. (BH SPNEGO request invalid prefix; thats the error of the helper protocol "helper-protocol=squid-2.5-ntlmssp" I used with NTLM, while basic works fine)

A user told me I should use negotiate_kerberos_auth instead of ntlm_auth.

Now here's my new problem:


root@x-x-testproxy01:/etc/squid# /usr/lib/squid/negotiate_kerberos_auth -d -s HTTP/[hidden email]
negotiate_kerberos_auth.cc(487): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Starting version 3.0.4sq
negotiate_kerberos_auth.cc(546): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Setting keytab to FILE:/etc/squid/HTTP.keytab
negotiate_kerberos_auth.cc(570): pid=5305 :2017/06/13 13:29:41| negotiate_kerberos_auth: INFO: Changed keytab to MEMORY:negotiate_kerberos_auth_5305
testuser xxxxxxx
negotiate_kerberos_auth.cc(610): pid=5305 :2017/06/13 13:29:47| negotiate_kerberos_auth: DEBUG: Got 'testuser xxxxxx' from squid (length: 18).
negotiate_kerberos_auth.cc(647): pid=5305 :2017/06/13 13:29:47| negotiate_kerberos_auth: ERROR: Invalid request [testuser xxxxxxx]
BH Invalid request
So my configuration has mistakes, but I can't find them. I don't really know where to search, or what works for sure. I tried many tutorials on krb5 and samba. Every form of testing I tried works fine except indeed using the required kerberos authentication of my squid-proxy.


Tests that come to my mind:

kinit a user

Warning: Your password will expire in 36 days on Don 20 Jul 2017 13:23:54 CEST



klist

Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [hidden email]

Valid starting       Expires              Service principal
2017-06-13 13:38:37  2017-06-13 23:38:37  krbtgt/[hidden email]
    renew until 2017-06-14 13:38:34


klist -k on my HTTP.keytab

Keytab name: FILE:/etc/squid/HTTP.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 host/[hidden email]
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL
   1 X-X-TESTPROXY01$@X-XXX.LOCAL

basic-auth using ntlm

root@x-x-testproxy01:/etc/squid# /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic --username=testuser --password=xxxxxxxx
testuser xxxxxxxxxx
OK
[hidden email] xxxxxxxx
OK

wbinfo -u
administrator
testuser
...
wbinfo -g
allowed rodc password replication group
enterprise read-only domain controllers
...

wbinfo --krb5auth=testuser%xxxxxxx
plaintext kerberos password authentication for [testuser%xxxxxxx] succeeded (requesting cctype: FILE)

wbinfo -t
checking the trust secret for domain X-XXX via RPC calls succeeded

wbinfo --authenticate=testuser%xxxxxxxx
plaintext password authentication succeeded
challenge/response password authentication succeeded

/usr/lib/squid/negotiate_kerberos_auth_test x-x-testproxy01.x-xxx.local
Token: YIIFOgYGKwYBBQUCoIIFLjCCBSqgJzAlBgkqhkiG9xIBAgIGBSsFAQUCBgkqhkiC9xIBAgIGBisGAQUCBaKCBP0EggT5YIIE9QYJKoZIhvcSAQICAQBuggTkMIIE4KADAgEFoQMCAQ6iBwMFAAAAAACjggP2YYID8jCCA+6gAwIBBaENGwtYLU5FVC5MT0NBTKIuMCygAwIBA6ElMCMbBEhUVFAbG3gtbC10ZXN0cHJveHkwMS54LW5ldC5sb2NhbKOCA6YwggOioAMCARKhAwIBAaKCA5QEggOQIMtincRDtWjh44pew3twk26Gm9rTC7CbkobNrzaRq/weljVl5TSbMQTFIVRQXVe4CQBWJ/Gcg472cgLA3mjOH8Z30zxQFP8fsK46wAtTEzJhonzXLImhaPtXvCVz94xaCVG7cBlNJCUmZQHsQMxFsGJZfKCkDvztiNplXEEwRgT7S6f8HQNm62xPAyz9aK8Wqfz9suW5cSBk8wdRAQNleKP1Xe/2LqZ4jfDpodPdcy9A8vh1dKu4tmbz+EJ/bKvWA+/twuXiOhhGq4W39TlOu/3zD87pXAh65ka1QsepkCWgUMHImDw8nUr8Zvi4j4vI9WhyMyLFYBya8BvAX9kLg73zl80g82bQVIAb8QU3gS2Akhpd7r1flfUbfRDUQfuS/bsaHspZoP+2c8+Bxy38OML4Gg29y6fJvRNfDaCnqmTQdiPtyqELVUS+4x7r260mM1wQKzD2Jb5pcz4wMHUK0sdw8xmMARGxB7XdyGSbo759GD6tOaTAKkNwccno6i9wyOoLqfhVRjE/K9FLCvCnzEFI07q1/dFz1Ce/ZzroW3nOwNQe3V6qqBELwuTvHgxIdGq4HEPeLqAUkVWxneXemRNbLKiOs/BIe3qkXxizgAkFLqRO5az2pVOD7/KBevxYZKeAgIuDsbIYG/u3Ic+KtCDaaM/to2b41SB8ZOFKJau3BuMPOvZ4ipiMbv0N/Svk4Tg61BIhN3CJYKA3ep/3p3wSfJlkcYeRVkDpasQnmFnjxV2YS7Q8nvmf9LIz+KIYBIT8X9yRPuV/E2lSELZlxJ8CySLFLUKgtMj97GPMlacc+UN3lJjyoExUKHpMZtUmaKrw7ueT3wnLMWgx0BBPkiAebUAedKj9u9sEscFylmI/+PdCCraMNbkOckCsggYXfJm6LxFZJhnDvw1+Z87xsJFDs5fasF6j8REiG8aHTmKHgt2M9TmIRNo/PsYrZGvuVQhkk2fuyFxwwyfs7ysNEkOmBWlFlTEjddjT9YShHfV8Fo8+M2UYY5nYiUIQq5BBfZ679ntivs7F80lKMOqhc7SOY63VwRJOwoq35+bnsIB08b9cttySiOcFsZb6uTnYvHzUFVSUha4nxrg3zW3fL9KVu4XY+lgCRZrBZMxioy6vbAOJBmpqXOJvel2gBbGN6PEd2ReeX43l1gcn+Bd3mQykmUIEzMMuRpSHda9233aWHbwEZ9H9rOdJdhgX4U+upIHQMIHNoAMCARKigcUEgcJu7kcC1zuJdhOQk+YA0Hw2G9kyg46tpaNIgj1CEgkD/KE28kaKivCTZTnHfrNOpIOJaaiYw4RMwJsbgZdRU+fz/jUwxvXdUSGon6JrU7S2XZ9CjXRXfdpXc4HjP0QW6Cql1SE95MpkcbMRH8FQvGNryBzsqIkELnvXceTGCmwlN3n60nqkoR5/41p2PtSz4hFMOVdT6AkPlNC5VTCtCZUj7YbrYVYImPnG3aAfQxXEHRy19/v0mL2845jZFA7Xw96s1A==



Sorry for posting so many output...
I already read many documentations, but no one really tests in small steps, they just assume that it works for everyone out of the box...

Does anyone have a clue what could be my mistake?

Thanks in advance.




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

David Kewley
In reply to this post by Amos Jeffries
On Tue, Jun 13, 2017 at 3:15 AM, Amos Jeffries <[hidden email]> wrote:
On 13/06/17 18:14, David Kewley wrote:
This might be of help if you are not already aware of the risks and issues involved with spoofing and handling of non-local IPs; <http://www.bcp38.info/>

I was aware of at least most of those issues, though I'm not an expert on them. So the reference is useful, thanks. 

Our squid server will only accept connections from its internal IP spaces, and I only wanted it to spoof the actual client source IPs to make downstream decision making easier based on the IP headers (but X-Forwarded-For might be sufficient, as you pointed out). Also the actual egress point to the internet is a NAT device that always uses its external IP as the source IP. So I see zero risk of us leaking foreign spoofed source IP addresses.

I will take a look at our ingress filtering to make sure we're rejecting external inbound packets claiming to be from our internal network.

Do you see anything obvious I'm missing around what I should do for BCP38?

I'm taking seriously your warning about a significant performance impact. I'll be curious to see if similar issues impact our nginx reverse proxies that spoof the original source IP in the proxied connection. Makes sense it might.

    Squid supports X-Forwarded-For fully - it was invented by Squid
    devs back in the day, and Squid is still the authoritative
    implementation for how it is supposed to work. As an old feature
    just about all other HTTP server and intermediary software have
    support for that too so you should have no issue pulling the data
    out at the receiving end, or in HTTP processing DPI software /
    firewalls etc. It is sent on all outgoing Squid messages unless
    you explicitly configure something else to happen with the
    forwarded_for directive.
     <http://www.squid-cache.org/Doc/config/forwarded_for/
    <http://www.squid-cache.org/Doc/config/forwarded_for/>>


I'll ask the team managing the next-hop device to evaluate that possibility; it looks to me from the docs like it might work. Thanks for the suggestion.

That would be best if it works.

I came up with a bodgy workaround using NAT after sending the earlier mail. So if there is no other way than delivering the client-IP on the packets there is still something that might be done. But, that would still run up against HTTP multiplexing and also add all sorts of NAT related issues as well. So only a last resort really.

Thanks! I'll come back to you if I think that might be useful. For now I'll proceed with using squid in a more normal fashion, and work with the owners of our next-hop-device about using X-Forwarded-For for any decision making.

I will proceed assuming that squid will never support the sort of spoofing I was hoping for (since it would probably simplify things greatly for us), even though I believe in our design that spoofing would have been safe.

David

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Alex Rousskov
On 06/13/2017 02:41 PM, David Kewley wrote:

> I will proceed assuming that squid will never support the sort of
> spoofing I was hoping for (since it would probably simplify things
> greatly for us), even though I believe in our design that spoofing would
> have been safe.

If you have a legitimate use case, Squid may address it. You just need
to convince developers that your use case does not violate basic
internet principles (more than the existing code does) and is generally
useful (i.e., many folks may find the new feature useful).

In such discussions, claims of RFC or BCP violations are often made.
Sometimes, those claims are correct. Sometimes, they are smoke and
mirrors. Sometimes, Squid already violates those documents. The onus of
distinguishing these cases while defending your use case is on you.

If you believe that your feature does not violate an RFC that is being
thrown against it, then you have to convince others that it does not.
You may request that others cite specific MUST-level requirements that
the feature would violate and then build a logical argument proving that
those MUSTs will not be violated or that those MUSTs are already
violated by other Squid features.


Please do not misinterpret the above as veiled support for the feature
you are requesting. I am just clarifying the rules of the game because
your current assumptions about feature request triage may not match the
reality. I do not know whether your feature violates any important RFCs
(more than other features do) or is generally useful.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

David Kewley
That's very helpful guidance, Alex. Thank you.

It's probably not in scope currently for me to take on championing such an effort, but I'll keep it in mind as an option for the future.

David

On Tue, Jun 13, 2017 at 2:43 PM, Alex Rousskov <[hidden email]> wrote:
On 06/13/2017 02:41 PM, David Kewley wrote:

> I will proceed assuming that squid will never support the sort of
> spoofing I was hoping for (since it would probably simplify things
> greatly for us), even though I believe in our design that spoofing would
> have been safe.

If you have a legitimate use case, Squid may address it. You just need
to convince developers that your use case does not violate basic
internet principles (more than the existing code does) and is generally
useful (i.e., many folks may find the new feature useful).

In such discussions, claims of RFC or BCP violations are often made.
Sometimes, those claims are correct. Sometimes, they are smoke and
mirrors. Sometimes, Squid already violates those documents. The onus of
distinguishing these cases while defending your use case is on you.

If you believe that your feature does not violate an RFC that is being
thrown against it, then you have to convince others that it does not.
You may request that others cite specific MUST-level requirements that
the feature would violate and then build a logical argument proving that
those MUSTs will not be violated or that those MUSTs are already
violated by other Squid features.


Please do not misinterpret the above as veiled support for the feature
you are requesting. I am just clarifying the rules of the game because
your current assumptions about feature request triage may not match the
reality. I do not know whether your feature violates any important RFCs
(more than other features do) or is generally useful.


HTH,

Alex.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Eliezer Croitoru
In reply to this post by David Kewley
Hey,

This is a library I wrote that uses tproxy:
https://github.com/elico/go-linux-tproxy

It’s doable using some enthusiasm but technically you cannot spoof just any IP since you need to be able to receive back this traffic.
You cannot really "cheap nationally" the BGP protocol but only for specific small areas which are all under your "domain" and management.

All The Bests,
Eliezer

----
http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


From: squid-users [mailto:[hidden email]] On Behalf Of David Kewley
Sent: Tuesday, June 13, 2017 4:48 AM
To: [hidden email]
Subject: [squid-users] source spoofing without tproxy?

I want my clients to explicitly address squid as a proxy (not use tproxy), but have squid spoof the source addresses in the forwarded connection, so that further hops know the original source address from the IPv4 headers.

I could find no indication that anyone else has done this, and when I tried various things, I could not get it working.

Is this possible today? If not, is it worth considering as a future feature? Or am I overlooking a reason that this cannot work even in theory?

I got the nearly-equivalent functionality working for reverse proxying using nginx, but so far I've found no way to do it with forward proxying. Nginx doesn't do https forward proxying (no handling of CONNECT).

If squid can't do what I'm looking for today, I would welcome pointers to other possible approaches.

Thanks,
David

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Eliezer Croitoru
Rephrase the "cheap nationally" into "cheat inernationally".

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Eliezer Croitoru
Sent: Wednesday, June 14, 2017 11:09 AM
To: 'David Kewley' <[hidden email]>; [hidden email]
Subject: Re: [squid-users] source spoofing without tproxy?

Hey,

This is a library I wrote that uses tproxy:
https://github.com/elico/go-linux-tproxy

It’s doable using some enthusiasm but technically you cannot spoof just any IP since you need to be able to receive back this traffic.
You cannot really "cheap nationally" the BGP protocol but only for specific small areas which are all under your "domain" and management.

All The Bests,
Eliezer

----
http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]


From: squid-users [mailto:[hidden email]] On Behalf Of David Kewley
Sent: Tuesday, June 13, 2017 4:48 AM
To: [hidden email]
Subject: [squid-users] source spoofing without tproxy?

I want my clients to explicitly address squid as a proxy (not use tproxy), but have squid spoof the source addresses in the forwarded connection, so that further hops know the original source address from the IPv4 headers.

I could find no indication that anyone else has done this, and when I tried various things, I could not get it working.

Is this possible today? If not, is it worth considering as a future feature? Or am I overlooking a reason that this cannot work even in theory?

I got the nearly-equivalent functionality working for reverse proxying using nginx, but so far I've found no way to do it with forward proxying. Nginx doesn't do https forward proxying (no handling of CONNECT).

If squid can't do what I'm looking for today, I would welcome pointers to other possible approaches.

Thanks,
David

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: source spoofing without tproxy?

Yuri Voinov
Nice shoot, Eliezer :-D


14.06.2017 19:28, Eliezer Croitoru пишет:

> Rephrase the "cheap nationally" into "cheat inernationally".
>
> ----
> Eliezer Croitoru
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: [hidden email]
>
>
> -----Original Message-----
> From: squid-users [mailto:[hidden email]] On Behalf Of Eliezer Croitoru
> Sent: Wednesday, June 14, 2017 11:09 AM
> To: 'David Kewley' <[hidden email]>; [hidden email]
> Subject: Re: [squid-users] source spoofing without tproxy?
>
> Hey,
>
> This is a library I wrote that uses tproxy:
> https://github.com/elico/go-linux-tproxy
>
> It’s doable using some enthusiasm but technically you cannot spoof just any IP since you need to be able to receive back this traffic.
> You cannot really "cheap nationally" the BGP protocol but only for specific small areas which are all under your "domain" and management.
>
> All The Bests,
> Eliezer
>
> ----
> http://ngtech.co.il/lmgtfy/
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: [hidden email]
>
>
> From: squid-users [mailto:[hidden email]] On Behalf Of David Kewley
> Sent: Tuesday, June 13, 2017 4:48 AM
> To: [hidden email]
> Subject: [squid-users] source spoofing without tproxy?
>
> I want my clients to explicitly address squid as a proxy (not use tproxy), but have squid spoof the source addresses in the forwarded connection, so that further hops know the original source address from the IPv4 headers.
>
> I could find no indication that anyone else has done this, and when I tried various things, I could not get it working.
>
> Is this possible today? If not, is it worth considering as a future feature? Or am I overlooking a reason that this cannot work even in theory?
>
> I got the nearly-equivalent functionality working for reverse proxying using nginx, but so far I've found no way to do it with forward proxying. Nginx doesn't do https forward proxying (no handling of CONNECT).
>
> If squid can't do what I'm looking for today, I would welcome pointers to other possible approaches.
>
> Thanks,
> David
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

signature.asc (484 bytes) Download Attachment
Loading...