Quantcast

FTP relay with active client is broken?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

FTP relay with active client is broken?

Alex-3
  Hello.

  Recently I gave FTP relay a try and it seems that it doesn't work out of the box :(
  I've seen a topic regarding passive mode (when squid puts real server's IP into 'Entering passive mode' message), however, I've solved this by writing a kernel module with custom netfilter hooks (the module intercepts squid's reply, gets IP and port and marks corresponding incoming connection, so it's possible to write a REDIRECT rule).
  I thought that active mode will cause less problems, but it seems that what squid tries to do is illegal. As far as I understand, in active mode squid tries to connect to a client and spoofs source IP address. But it simply does not work: even if bind() succeeds after setting 'ip_nonlocal_bind' sysctl to 1, the connect() call fails with EINVAL. According to https://lkml.org/lkml/2001/6/7/17, such kernel's behaviour is legit and squid tries to do something nasty.

  Here's the excerpt from squid's log (3.5.24 on CentOS 6.5 with 4.x kernel):

017/02/07 15:24:12.262| 5,3| ConnOpener.cc(289) createFd: local=172.17.10.30 remote=172.17.11.31:56676 flags=9 will timeout in 60
2017/02/07 15:24:12.262| 5,9| comm.cc(602) comm_connect_addr: connecting socket FD 16 to 172.17.11.31:56676 (want family: 2)
2017/02/07 15:24:12.262| 5,5| comm.cc(644) comm_connect_addr: sock=16, addrinfo( flags=4, family=2, socktype=1, protocol=6, &addr=0x1bffc00, addrlen=16 )
2017/02/07 15:24:12.262| 5,9| comm.cc(645) comm_connect_addr: connect FD 16: (-1) (22) Invalid argument
2017/02/07 15:24:12.262| 14,9| comm.cc(646) comm_connect_addr: connecting to: 172.17.11.31:56676
2017/02/07 15:24:12.262| 5,7| ConnOpener.cc(357) doConnect: local=172.17.10.30 remote=172.17.11.31:56676 flags=9: failure #1 <= 0: (22) Invalid argument
2017/02/07 15:24:12.262| 5,5| ConnOpener.cc(365) doConnect: local=172.17.10.30 remote=172.17.11.31:56676 flags=9: * - ERR tried too many times already.
2017/02/07 15:24:12.262| 17,3| AsyncCall.cc(93) ScheduleCall: ConnOpener.cc(137) will call Ftp::Server::connectedForData(local=172.17.10.30 remote=172.17.11.31:56676 flags=9, errno=22, flag=-8, data=0x17d6188) [call95]

  Any thoughts?
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FTP relay with active client is broken?

Alex-3
Anyone? Is it a bug or something should be tuned? I've specified 'ftp_port 2121 intercept' and made squid intercept outgoing FTP traffic according to the following rules:

iptables -t nat -A OUTPUT -p tcp -m owner --gid-owner squid -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 21 -j REDIRECT --to-port 2121


07.02.2017, 16:23, "Alex" <[hidden email]>:

>   Hello.
>
>   Recently I gave FTP relay a try and it seems that it doesn't work out of the box :(
>   I've seen a topic regarding passive mode (when squid puts real server's IP into 'Entering passive mode' message), however, I've solved this by writing a kernel module with custom netfilter hooks (the module intercepts squid's reply, gets IP and port and marks corresponding incoming connection, so it's possible to write a REDIRECT rule).
>   I thought that active mode will cause less problems, but it seems that what squid tries to do is illegal. As far as I understand, in active mode squid tries to connect to a client and spoofs source IP address. But it simply does not work: even if bind() succeeds after setting 'ip_nonlocal_bind' sysctl to 1, the connect() call fails with EINVAL. According to https://lkml.org/lkml/2001/6/7/17, such kernel's behaviour is legit and squid tries to do something nasty.
>
>   Here's the excerpt from squid's log (3.5.24 on CentOS 6.5 with 4.x kernel):
>
> 017/02/07 15:24:12.262| 5,3| ConnOpener.cc(289) createFd: local=172.17.10.30 remote=172.17.11.31:56676 flags=9 will timeout in 60
> 2017/02/07 15:24:12.262| 5,9| comm.cc(602) comm_connect_addr: connecting socket FD 16 to 172.17.11.31:56676 (want family: 2)
> 2017/02/07 15:24:12.262| 5,5| comm.cc(644) comm_connect_addr: sock=16, addrinfo( flags=4, family=2, socktype=1, protocol=6, &addr=0x1bffc00, addrlen=16 )
> 2017/02/07 15:24:12.262| 5,9| comm.cc(645) comm_connect_addr: connect FD 16: (-1) (22) Invalid argument
> 2017/02/07 15:24:12.262| 14,9| comm.cc(646) comm_connect_addr: connecting to: 172.17.11.31:56676
> 2017/02/07 15:24:12.262| 5,7| ConnOpener.cc(357) doConnect: local=172.17.10.30 remote=172.17.11.31:56676 flags=9: failure #1 <= 0: (22) Invalid argument
> 2017/02/07 15:24:12.262| 5,5| ConnOpener.cc(365) doConnect: local=172.17.10.30 remote=172.17.11.31:56676 flags=9: * - ERR tried too many times already.
> 2017/02/07 15:24:12.262| 17,3| AsyncCall.cc(93) ScheduleCall: ConnOpener.cc(137) will call Ftp::Server::connectedForData(local=172.17.10.30 remote=172.17.11.31:56676 flags=9, errno=22, flag=-8, data=0x17d6188) [call95]
>
>   Any thoughts?
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FTP relay with active client is broken?

Alex Rousskov
On 02/08/2017 06:10 AM, Alex wrote:

> I've specified
> 'ftp_port 2121 intercept' and made squid intercept outgoing FTP
> traffic according to the following rules:

> iptables -t nat -A OUTPUT -p tcp -m owner --gid-owner squid -j ACCEPT
> iptables -t nat -A OUTPUT -p tcp --dport 21 -j REDIRECT --to-port 2121

> 07.02.2017, 16:23, "Alex" <[hidden email]>:

>> I thought that active mode will cause less problems, but it seems
>> that what squid tries to do is illegal. As far as I understand, in
>> active mode squid tries to connect to a client and spoofs source IP
>> address.

Since spoofing client IP addresses is common for many working Squid
interception setups doing HTTP, it has to be technically possible (i.e.,
"legal" in your terminology). Unfortunately, I do not know enough
low-level details to guide you further. Most likely, the FTP-specific
Squid code facilitating IP spoofing is buggy or you are doing something
wrong (or both).

FWIW, IIRC, FTP interception code has worked for many folks.

Let's hope that somebody with a working FTP interception setup speaks up.

Alex.

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FTP relay with active client is broken?

Alex-3
  Well, actually it looks like a bug in squid to me. Some points:

1. When client connects with active FTP mode, squid opens a local socket and tries to assign server's IP address to it. So far so good, however this doesn't work (tested in both 'intercept' and 'tproxy' modes):

   2017/02/13 14:58:51.234| 50,3| comm.cc(347) comm_openex: comm_openex: Attempt open socket for: 172.17.10.30
   2017/02/13 14:58:51.234| 50,3| comm.cc(388) comm_openex: comm_openex: Opened socket local=172.17.10.30 remote=[::] FD 17 flags=1 : family=2, type=1, protocol=6
   2017/02/13 14:58:51.234| 5,5| comm.cc(420) comm_init_opened: local=172.17.10.30 remote=[::] FD 17 flags=1 is a new socket
   2017/02/13 14:58:51.234| 51,3| fd.cc(198) fd_open: fd_open() FD 17
   2017/02/13 14:58:51.234| commBind: Cannot bind socket FD 17 to 172.17.10.30: (99) Cannot assign requested address

2. Ok, we've got the errno. Let's look in comm_apply_flags(). There's something like this in the middle:

    /* MUST be done before binding or face OS Error: "(99) Cannot assign requested address"... */
    if ((flags & COMM_TRANSPARENT)) {
        comm_set_transparent(new_socket);
    }

    This means that socket is not IP_TRANSPARENT and can not be bound to arbitrary IP address ('net.ipv4.ip_nonlocal_bind' can fix bind(), but connect() will obviously fail with EINVAL).

3. Let's take a look on Ftp::Server::createDataConnection():

    Comm::ConnectionPointer conn = new Comm::Connection();
    conn->flags |= COMM_DOBIND;

    // Use local IP address of the control connection as the source address
    // of the active data connection, or some clients will refuse to accept.
    conn->setAddrs(clientConnection->local, cltAddr);

    Fine, looks reasonable. However, connection has only COMM_DOBIND flag, COMM_TRANSPARENT is missing. AFAIU, we'll never be able to bind and connect a socket...

4. Finally, in restoreCapabilities():

    if (Ip::Interceptor.TransparentActive() ||
                Ip::Qos::TheConfig.isHitNfmarkActive() ||
                Ip::Qos::TheConfig.isAclNfmarkActive() ||
                Ip::Qos::TheConfig.isAclTosActive()) {
            cap_list[ncaps] = CAP_NET_ADMIN;
            ++ncaps;
    }

    Hmm... Looks like that spoofing will not work for any mode other than 'tproxy', because there's no InterceptActive() check. But anyway, without COMM_TRANSPARENT it's useless.

  To sum up, I see some possible mistakes that may cause bugs in FTP relaying. I realise that probably I do something wrong also, however due to lack of documentation for FTP relay there's no simple way to check this :)


08.02.2017, 19:11, "Alex Rousskov" <[hidden email]>:

> On 02/08/2017 06:10 AM, Alex wrote:
>
>>  I've specified
>>  'ftp_port 2121 intercept' and made squid intercept outgoing FTP
>>  traffic according to the following rules:
>
>>  iptables -t nat -A OUTPUT -p tcp -m owner --gid-owner squid -j ACCEPT
>>  iptables -t nat -A OUTPUT -p tcp --dport 21 -j REDIRECT --to-port 2121
>
>>  07.02.2017, 16:23, "Alex" <[hidden email]>:
>
>>>  I thought that active mode will cause less problems, but it seems
>>>  that what squid tries to do is illegal. As far as I understand, in
>>>  active mode squid tries to connect to a client and spoofs source IP
>>>  address.
>
> Since spoofing client IP addresses is common for many working Squid
> interception setups doing HTTP, it has to be technically possible (i.e.,
> "legal" in your terminology). Unfortunately, I do not know enough
> low-level details to guide you further. Most likely, the FTP-specific
> Squid code facilitating IP spoofing is buggy or you are doing something
> wrong (or both).
>
> FWIW, IIRC, FTP interception code has worked for many folks.
>
> Let's hope that somebody with a working FTP interception setup speaks up.
>
> Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FTP relay with active client is broken?

Amos Jeffries
Administrator
On 14/02/2017 1:18 a.m., Alex wrote:

>   Well, actually it looks like a bug in squid to me. Some points:
>
> 1. When client connects with active FTP mode, squid opens a local socket and tries to assign server's IP address to it. So far so good, however this doesn't work (tested in both 'intercept' and 'tproxy' modes):
>
>    2017/02/13 14:58:51.234| 50,3| comm.cc(347) comm_openex: comm_openex: Attempt open socket for: 172.17.10.30
>    2017/02/13 14:58:51.234| 50,3| comm.cc(388) comm_openex: comm_openex: Opened socket local=172.17.10.30 remote=[::] FD 17 flags=1 : family=2, type=1, protocol=6
>    2017/02/13 14:58:51.234| 5,5| comm.cc(420) comm_init_opened: local=172.17.10.30 remote=[::] FD 17 flags=1 is a new socket
>    2017/02/13 14:58:51.234| 51,3| fd.cc(198) fd_open: fd_open() FD 17
>    2017/02/13 14:58:51.234| commBind: Cannot bind socket FD 17 to 172.17.10.30: (99) Cannot assign requested address
>
> 2. Ok, we've got the errno. Let's look in comm_apply_flags(). There's something like this in the middle:
>
>     /* MUST be done before binding or face OS Error: "(99) Cannot assign requested address"... */
>     if ((flags & COMM_TRANSPARENT)) {
>         comm_set_transparent(new_socket);
>     }
>
>     This means that socket is not IP_TRANSPARENT and can not be bound to arbitrary IP address ('net.ipv4.ip_nonlocal_bind' can fix bind(), but connect() will obviously fail with EINVAL).
>
> 3. Let's take a look on Ftp::Server::createDataConnection():
>
>     Comm::ConnectionPointer conn = new Comm::Connection();
>     conn->flags |= COMM_DOBIND;
>
>     // Use local IP address of the control connection as the source address
>     // of the active data connection, or some clients will refuse to accept.
>     conn->setAddrs(clientConnection->local, cltAddr);
>
>     Fine, looks reasonable. However, connection has only COMM_DOBIND flag, COMM_TRANSPARENT is missing. AFAIU, we'll never be able to bind and connect a socket...
>

Good catch. Thank you.

So AFAICS,
* NAT intercept needs to listen on the local IP of the Squid->server
control connection.
* TPROXY needs to listen on the local IP of the client->Squid control
connection plus the TRANSPARENT flag.

Any chance of a patch?

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FTP relay with active client is broken?

Alex-3
Well, I can try to make a patch for this... Two questions:
1. I should send it to squid-dev, do I?
2. Source code for which version should I use: 4.0 or 3.5?

14.02.2017, 04:59, "Amos Jeffries" <[hidden email]>:

> On 14/02/2017 1:18 a.m., Alex wrote:
>>    Well, actually it looks like a bug in squid to me. Some points:
>>
>>  1. When client connects with active FTP mode, squid opens a local socket and tries to assign server's IP address to it. So far so good, however this doesn't work (tested in both 'intercept' and 'tproxy' modes):
>>
>>     2017/02/13 14:58:51.234| 50,3| comm.cc(347) comm_openex: comm_openex: Attempt open socket for: 172.17.10.30
>>     2017/02/13 14:58:51.234| 50,3| comm.cc(388) comm_openex: comm_openex: Opened socket local=172.17.10.30 remote=[::] FD 17 flags=1 : family=2, type=1, protocol=6
>>     2017/02/13 14:58:51.234| 5,5| comm.cc(420) comm_init_opened: local=172.17.10.30 remote=[::] FD 17 flags=1 is a new socket
>>     2017/02/13 14:58:51.234| 51,3| fd.cc(198) fd_open: fd_open() FD 17
>>     2017/02/13 14:58:51.234| commBind: Cannot bind socket FD 17 to 172.17.10.30: (99) Cannot assign requested address
>>
>>  2. Ok, we've got the errno. Let's look in comm_apply_flags(). There's something like this in the middle:
>>
>>      /* MUST be done before binding or face OS Error: "(99) Cannot assign requested address"... */
>>      if ((flags & COMM_TRANSPARENT)) {
>>          comm_set_transparent(new_socket);
>>      }
>>
>>      This means that socket is not IP_TRANSPARENT and can not be bound to arbitrary IP address ('net.ipv4.ip_nonlocal_bind' can fix bind(), but connect() will obviously fail with EINVAL).
>>
>>  3. Let's take a look on Ftp::Server::createDataConnection():
>>
>>      Comm::ConnectionPointer conn = new Comm::Connection();
>>      conn->flags |= COMM_DOBIND;
>>
>>      // Use local IP address of the control connection as the source address
>>      // of the active data connection, or some clients will refuse to accept.
>>      conn->setAddrs(clientConnection->local, cltAddr);
>>
>>      Fine, looks reasonable. However, connection has only COMM_DOBIND flag, COMM_TRANSPARENT is missing. AFAIU, we'll never be able to bind and connect a socket...
>
> Good catch. Thank you.
>
> So AFAICS,
> * NAT intercept needs to listen on the local IP of the Squid->server
> control connection.
> * TPROXY needs to listen on the local IP of the client->Squid control
> connection plus the TRANSPARENT flag.
>
> Any chance of a patch?
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...