Quantcast

Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Amos Jeffries
Administrator
On 20/01/2017 9:40 p.m., Alexander wrote:
> Hello, I have a question regarding a native FTP relay (squid's version is
> 3.5.23).

Have you tried NAT intercept for the FTP port?
TPROXY has some low-level things including socket mapping that might not
go so well with how FTP uses multiple connections.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Amos Jeffries
Administrator
On 23/01/2017 2:49 a.m., Alexander wrote:
> As far as I remember, I have tried both options, REDIRECT and TPROXY, but
> TPROXY is the preferred one for us. I will try one more time on Monday.
> However, I suppose that something else prevents squid from working properly.
> Maybe on of sysctls, like net.ipv4.ip_nonlocal_bind, will do the trick.

Maybe.

I expect that REDIRECT will be the required way for FTP at present,
since TPROXY has requirements that there is a client socket state to
associate with the non-local binding. Essentially that sockets are
opened directionally in sequence client->proxy->server - whereas FTP
data connections are opened in the opposite sequencing order:
server->squid first, then squid->client.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
In reply to this post by Amos Jeffries
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alex Rousskov
In reply to this post by Alexander
On 01/23/2017 03:11 AM, Alexander wrote:

> 3. Squid opens a local port and sends it back to client via the "Entering
> passive mode" reply. Seems to be ok, but a client sees a real server's IP
> address, not a squid's one. So when a client tries to connect to a server,
> it gets ECONNREFUSED because no-one is listening on a requested port.


This Squid behavior is intentional:

>     // In interception setups, we combine remote server address with a
>     // local port number and hope that traffic will be redirected to us.
...
>     mb.appendf("227 Entering Passive Mode (%s,%i,%i).\r\n",


> So when a client tries to connect to a server,

... your networking rules should redirect that connection to Squid in
order to avoid the problem you are describing:


> it gets ECONNREFUSED because no-one is listening on a requested port.

Please note that I am _not_ claiming that the intentional Squid behavior
is correct in all cases. I only know that we made Squid do what it does
now to fix a (most likely real) problem:

> revno: 12742.1.11
> branch nick: ftp-gw
> timestamp: Wed 2013-08-21 09:39:09 -0600
> message:
>   Fixed address handling for PASV responses in interception cases.


HTH,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alex Rousskov
In reply to this post by Alexander
On 01/23/2017 11:11 AM, Alexander wrote:
> Actually, a PASV-handling logic looks a bit strange to me. In
> Ftp::Server::handlePasvReply() there is a comment:
>
> "In interception setups, we combine remote server address with a local port
> number and hope that traffic will be redirected to us."
>
> How is it supposed to work? I do
> not have any idea on how a traffic could be redirected to squid (redirecting
> everything from A to B is not an option).

You should only redirect FTP traffic, of course. Sorry, I do not know
how you can identify FTP data traffic in your environment, but I am sure
there are tools that can do that in some environments (e.g., by
monitoring FTP 227 responses on the already redirected connections).
There are also some ideas for future work below in case nobody can
suggest anything better.


> Also, why squid needs to intercept a data connection?

For the same set of reasons Squid needs to intercept everything else --
traffic logging, blocking, and adaptation. If you want Squid to proxy a
"message", Squid expects to proxy the entire "message". In FTP, a single
"message" (from high-level point of view) is often split among two or
more connections (from TCP point of view).

Needless to say, your specific needs may differ from that general
principle. It is possible that Squid needs a knob to handle your use
case differently. However, I am pretty sure that somebody does want
Squid to do what it does know so we should not change Squid behavior to
satisfy your use case.


> If I hardcode one of squid's IP in handlePasvReply(), everything works fine.
> However I am not sure if it is a correct way because a client opens a data
> connection not to an FTP server...

I agree that mixing intercepted [control] and direct [data] connections
is a bad design in general, even if it works in your use case. In many
cases, Squid IP address is not even reachable from the client!
Hopefully, you can find a better way to handle this.

What if you can restrict the set of ports that Squid uses to accept
passive FTP data connections? That way, you can redirect only those data
connections that match those ports. This is not an ideal solution, and
Squid does not support that directly right now, but it might work in
principle.

Another option is to modify Squid to report the expected data connection
IP:ports to some helper so that you can write a script that dynamically
modifies your network redirection rules.

Others may know a better way to handle this (short of deploying an
FTP-aware L7 networking gear).


Cheers,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alex Rousskov
On 01/23/2017 12:18 PM, Alexander wrote:
> 2017-01-23 21:41 GMT+03:00 Alex Rousskov:
>     It is possible that Squid needs a knob to handle your use
>     case differently. However, I am pretty sure that somebody does want
>     Squid to do what it does know so we should not change Squid behavior to
>     satisfy your use case.

Clarification: ... should not satisfy your use case alone (i.e., at the
expense of the other known use case). It is perfectly fine to change
Squid to satisfy more than one legitimate use case, of course.


> I understand that, however the first and foremost reason I asked the
> question was that my use case pretends to be pretty typical :)

That may be true, but please keep in mind that:

a) The "whole message or nothing" principle is fairly fundamental to
Squid and correctly accommodating exceptions to that principle may be
difficult, on many levels.

b) Native FTP relaying is a very recent feature so any "lots of Squid
users need FTP relay to do X" argument can be paired with "the vast and
increasing majority of Squid users do need FTP relay at all, so FTP code
should not inconvenience the other code much" argument followed by the
same "whole message or nothing" principle discussed earlier.

Neither (a) nor (b) means that your use case should not be supported,
one way or the other. I am just cautioning against a rushed judgment to
change Squid without thinking of other users and long-term
effects/consequences.


Cheers,

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Antony Stone
On Thursday 26 January 2017 at 17:41:21, Alexander wrote:

> It seems that I have solved the issue by using nf_conntrack_ftp and
> redirecting "NEW,RELATED" traffic to squid:

Excellent news.

> ftp_port 2121 intercept
>
> modprobe nf_conntrack_ftp ports=2121
>
> iptables -t nat -A PREROUTING -p tcp --dport 21 -j REDIRECT --to-port 2121
> iptables -t nat -A PREROUTING -p tcp -m state --state NEW,RELATED -j
> REDIRECT

Just out of interest, how are you getting the FTP traffic to the Squid box in
the first place?

I assume you're not routing all Internet-bound traffic via this machine
(otherwise that second REDIRECT rule would cause problems for SSH, SMTP, IMAP,
etc), so how are you identifying the FTP traffic to get it from your router to
the Squid box?


Antony.

--
Police have found a cartoonist dead in his house.  They say that details are
currently sketchy.

                                                   Please reply to the list;
                                                         please *don't* CC me.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Alexander
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Native FTP relay: connection closes (?) after 'cannot assign requested address' error

Matus UHLAR - fantomas
In reply to this post by Alexander
On 26.01.17 08:41, Alexander wrote:

>It seems that I have solved the issue by using nf_conntrack_ftp and
>redirecting "NEW,RELATED" traffic to squid:
>
>ftp_port 2121 intercept
>
>modprobe nf_conntrack_ftp ports=2121
>
>iptables -t nat -A PREROUTING -p tcp --dport 21 -j REDIRECT --to-port 2121
>iptables -t nat -A PREROUTING -p tcp -m state --state NEW,RELATED -j
>REDIRECT

just note that connections may be related to different connections than
FTP...

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...