Reverse proxy and TUNNEL to same cache peer

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Reverse proxy and TUNNEL to same cache peer

Hariharan Sethuraman
Hi,

We have our company proxy and this is how the topology is expected to look like for the deployment:

Client -------------------squid-host.com---------------------------company-proxy------------Internet

Now I need to allow reverse proxy(3128) for some request from the client and tunnel (3129) as well.

Configuration:
http_port 3128 accel allow-direct
http_port 3129
never_direct allow all
always_direct deny all
...
cache_peer company-proxy parent 80       0  no-query no-digest login=PASS originserver
url_rewrite_access allow all
url_rewrite_program  /usr/bin/python ./rewriter_program.py

Usecases:

1) Reverse proxy: Now I can successfully get the response for the query like curl -X GET http://squid-host.com:3128/microsoftapi/api/something. Basically I rewrite URL to https://microsft.com/api/something and through company-proxy I get the response successfully from e.g., microsoft.com.

2) Tunnel: It fails when the client do a query like curl -x http://squid-host.com:3129 -X GET https://googlecloudapis.com/api/something
< HTTP/1.1 503 Service Unavailable
< Server: squid/3.5.20
< Mime-Version: 1.0
< Date: Tue, 07 Aug 2018 12:36:07 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3879
< X-Squid-Error: ERR_CANNOT_FORWARD 0
< Vary: Accept-Language
< Content-Language: en
<
* The requested URL returned error: 503
* CONNECT phase completed!
* Connection #0 to host squidhostname.com left intact
Now, if I remove the origin server, the TUNNEL goes through and getting the response but the reverse proxy fails.

Could you let me know how I can handle both tunneling and reverse proxy through same cache peer?

Thanks,
Hari


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Amos Jeffries
Administrator
On 08/08/18 01:04, Hariharan Sethuraman wrote:

> Hi,
>
> We have our company proxy and this is how the topology is expected to
> look like for the deployment:
>
> Client
> -------------------squid-host.com---------------------------company-proxy------------Internet
>
> Now I need to allow reverse proxy(3128) for some request from the client
> and tunnel (3129) as well.

This reads like you don't understand what either of those terms mean.

Your proxies port 3129 is setup for forward-proxy traffic.

"tunnel" is and action that can be done by clients. Also the term used
to describe the *payload* of a CONNECT message. The action a
forward-proxy performs when receiving a CONNECT is usually to setup a
tunnel CONNECT-ion for non-HTTP use.

Using port 3128 for reverse-proxy traffic is a very bad idea. That port
is a well-known port and also reserved for explicit / forward-proxy traffic.


>
> Configuration:
> http_port 3128 accel allow-direct
> http_port 3129
> never_direct allow all
> always_direct deny all
> ...
> cache_peer company-proxy parent 80       0  no-query no-digest
> login=PASS originserver
> url_rewrite_access allow all
> url_rewrite_program  /usr/bin/python ./rewriter_program.py
>
> Usecases:
>
> 1) Reverse proxy: Now I can successfully get the response for the query
> like curl -X GET http://squid-host.com:3128/microsoftapi/api/something.
> Basically I rewrite URL to https://microsft.com/api/something and
> through company-proxy I get the response successfully from e.g.,
> microsoft.com <http://microsoft.com>.

Re-writing message URLs across the insecure / secure boundary is a very
bad idea. All Cookies and other things in both message headers and
payload content which are tied to either the TLS or the Origin state
will be broken.

The client thinks it is talking to an insecure server, and the server
thinks it is connected to a secure client. They also have very different
ideas about the Origin scope for stateful security things - especially
when re-writing across domains like this. That means they each believe
different range of actions are valid, and that impacts on what they
expect the other agents behaviour to be in reaction to any given message.


>
> 2) Tunnel: It fails when the client do a query like curl -x
> http://squid-host.com:3129 -X GET https://googlecloudapis.com/api/something
> < HTTP/1.1 503 Service Unavailable
> < Server: squid/3.5.20
> < Mime-Version: 1.0
> < Date: Tue, 07 Aug 2018 12:36:07 GMT
> < Content-Type: text/html;charset=utf-8
> < Content-Length: 3879
> < X-Squid-Error: ERR_CANNOT_FORWARD 0
> < Vary: Accept-Language
> < Content-Language: en
> <
> * The requested URL returned error: 503
> * CONNECT phase completed!
> * Connection #0 to host squidhostname.com <http://squidhostname.com>
> left intact
> Now, if I remove the origin server, the TUNNEL goes through and getting
> the response but the reverse proxy fails.
>

Please add the --verbose option to your curl test commands to see what
is actually being sent to the proxy. Then test what your re-writer is
causing to happen on those received URI.

Note that all the re-writer does is change the URI. It does not change
the method or other request details. If it produces an invalid or
unreachable URI there is nothing Squid can do but present the client
with an error.



> Could you let me know how I can handle both tunneling and reverse proxy
> through same cache peer?

Both are already supported and work without any fancy setup. I think you
are hitting something else which should become clearer when you see what
curl is actually doing.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Hariharan Sethuraman
Thanks Amos: yes agree that I should have told forward proxy.

When I remove the originserver option from cache_peer, the forward proxy is working so which means the rewriter is not precluding from happening. Does that give any clue to us? 

Moreover the reverse proxy is in next hop to the client and not in internet. Time being, we are ok to have insecure channel between client and squid. Do you have any sample config that that uses a parent proxy to do both forward/reverse proxy? Or do you see my config is good enough for this requirement.

On Tue, Aug 7, 2018 at 9:07 PM, Amos Jeffries <[hidden email]> wrote:
On 08/08/18 01:04, Hariharan Sethuraman wrote:
> Hi,
>
> We have our company proxy and this is how the topology is expected to
> look like for the deployment:
>
> Client
> -------------------squid-host.com---------------------------company-proxy------------Internet
>
> Now I need to allow reverse proxy(3128) for some request from the client
> and tunnel (3129) as well.

This reads like you don't understand what either of those terms mean.

Your proxies port 3129 is setup for forward-proxy traffic.

"tunnel" is and action that can be done by clients. Also the term used
to describe the *payload* of a CONNECT message. The action a
forward-proxy performs when receiving a CONNECT is usually to setup a
tunnel CONNECT-ion for non-HTTP use.

Using port 3128 for reverse-proxy traffic is a very bad idea. That port
is a well-known port and also reserved for explicit / forward-proxy traffic.


>
> Configuration:
> http_port 3128 accel allow-direct
> http_port 3129
> never_direct allow all
> always_direct deny all
> ...
> cache_peer company-proxy parent 80       0  no-query no-digest
> login=PASS originserver
> url_rewrite_access allow all
> url_rewrite_program  /usr/bin/python ./rewriter_program.py
>
> Usecases:
>
> 1) Reverse proxy: Now I can successfully get the response for the query
> like curl -X GET http://squid-host.com:3128/microsoftapi/api/something.
> Basically I rewrite URL to https://microsft.com/api/something and
> through company-proxy I get the response successfully from e.g.,
> microsoft.com <http://microsoft.com>.

Re-writing message URLs across the insecure / secure boundary is a very
bad idea. All Cookies and other things in both message headers and
payload content which are tied to either the TLS or the Origin state
will be broken.

The client thinks it is talking to an insecure server, and the server
thinks it is connected to a secure client. They also have very different
ideas about the Origin scope for stateful security things - especially
when re-writing across domains like this. That means they each believe
different range of actions are valid, and that impacts on what they
expect the other agents behaviour to be in reaction to any given message.


>
> 2) Tunnel: It fails when the client do a query like curl -x
> http://squid-host.com:3129 -X GET https://googlecloudapis.com/api/something
> < HTTP/1.1 503 Service Unavailable
> < Server: squid/3.5.20
> < Mime-Version: 1.0
> < Date: Tue, 07 Aug 2018 12:36:07 GMT
> < Content-Type: text/html;charset=utf-8
> < Content-Length: 3879
> < X-Squid-Error: ERR_CANNOT_FORWARD 0
> < Vary: Accept-Language
> < Content-Language: en
> <
> * The requested URL returned error: 503
> * CONNECT phase completed!
> * Connection #0 to host squidhostname.com <http://squidhostname.com>
> left intact
> Now, if I remove the origin server, the TUNNEL goes through and getting
> the response but the reverse proxy fails.
>

Please add the --verbose option to your curl test commands to see what
is actually being sent to the proxy. Then test what your re-writer is
causing to happen on those received URI.

Note that all the re-writer does is change the URI. It does not change
the method or other request details. If it produces an invalid or
unreachable URI there is nothing Squid can do but present the client
with an error.



> Could you let me know how I can handle both tunneling and reverse proxy
> through same cache peer?

Both are already supported and work without any fancy setup. I think you
are hitting something else which should become clearer when you see what
curl is actually doing.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Amos Jeffries
Administrator
On 08/08/18 04:01, Hariharan Sethuraman wrote:
> Thanks Amos: yes agree that I should have told forward proxy.
>
> When I remove the originserver option from cache_peer, the forward proxy
> is working so which means the rewriter is not precluding from happening.
> Does that give any clue to us? 
>

Ah, cant believe I missed that. If the parent proxy is your access to
the Internet then is *not* a reverse-proxy. It cannot be and receive
proxy<->proxy traffic.

Any attempt to change the scheme is erased because the scheme is not
part of origin-form message syntax.


> Moreover the reverse proxy is in next hop to the client and not in
> internet. Time being, we are ok to have insecure channel between client
> and squid. Do you have any sample config that that uses a parent proxy
> to do both forward/reverse proxy? Or do you see my config is good enough
> for this requirement.
>

The traffic types have different syntax. It is possible to have a parent
proxy which receives both, but that has to be different ports and
different cache_peer links between them.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Hariharan Sethuraman
Yes correct, the parent Proxy is a forward, but the squid will have to do both from client aspect.

Can I run two instances of squid - forward and reverse separately considering my configuration is good enough?

On Tue, 7 Aug 2018, 22:00 Amos Jeffries, <[hidden email]> wrote:
On 08/08/18 04:01, Hariharan Sethuraman wrote:
> Thanks Amos: yes agree that I should have told forward proxy.
>
> When I remove the originserver option from cache_peer, the forward proxy
> is working so which means the rewriter is not precluding from happening.
> Does that give any clue to us? 
>

Ah, cant believe I missed that. If the parent proxy is your access to
the Internet then is *not* a reverse-proxy. It cannot be and receive
proxy<->proxy traffic.

Any attempt to change the scheme is erased because the scheme is not
part of origin-form message syntax.


> Moreover the reverse proxy is in next hop to the client and not in
> internet. Time being, we are ok to have insecure channel between client
> and squid. Do you have any sample config that that uses a parent proxy
> to do both forward/reverse proxy? Or do you see my config is good enough
> for this requirement.
>

The traffic types have different syntax. It is possible to have a parent
proxy which receives both, but that has to be different ports and
different cache_peer links between them.

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Hariharan Sethuraman
> The traffic types have different syntax. It is possible to have a parent
> proxy which receives both, but that has to be different ports and
> different cache_peer links between them.

As I said in same cache_peer (without changing the parent proxy port), both forward (removed originserver option) and reverse (with origin server option) works.
For now, I will go with two squid instances one instance for forward and other for reverse till I get an answer.

On Wed, Aug 8, 2018 at 6:07 AM, Hariharan Sethuraman <[hidden email]> wrote:
Yes correct, the parent Proxy is a forward, but the squid will have to do both from client aspect.

Can I run two instances of squid - forward and reverse separately considering my configuration is good enough?

On Tue, 7 Aug 2018, 22:00 Amos Jeffries, <[hidden email]> wrote:
On 08/08/18 04:01, Hariharan Sethuraman wrote:
> Thanks Amos: yes agree that I should have told forward proxy.
>
> When I remove the originserver option from cache_peer, the forward proxy
> is working so which means the rewriter is not precluding from happening.
> Does that give any clue to us? 
>

Ah, cant believe I missed that. If the parent proxy is your access to
the Internet then is *not* a reverse-proxy. It cannot be and receive
proxy<->proxy traffic.

Any attempt to change the scheme is erased because the scheme is not
part of origin-form message syntax.


> Moreover the reverse proxy is in next hop to the client and not in
> internet. Time being, we are ok to have insecure channel between client
> and squid. Do you have any sample config that that uses a parent proxy
> to do both forward/reverse proxy? Or do you see my config is good enough
> for this requirement.
>

The traffic types have different syntax. It is possible to have a parent
proxy which receives both, but that has to be different ports and
different cache_peer links between them.

Amos


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Hariharan Sethuraman
I think giving name helped to fwd/reverse to same parent proxy port:
cache_peer parent-proxy.domain.com parent 80       0  no-query no-digest login=PASS originserver name=reverseproxy
cache_peer parent-proxy.domain.com parent 80       0  no-query no-digest login=PASS name=forwardproxy


On Wed, Aug 8, 2018 at 9:55 AM, Hariharan Sethuraman <[hidden email]> wrote:
> The traffic types have different syntax. It is possible to have a parent
> proxy which receives both, but that has to be different ports and
> different cache_peer links between them.

As I said in same cache_peer (without changing the parent proxy port), both forward (removed originserver option) and reverse (with origin server option) works.
For now, I will go with two squid instances one instance for forward and other for reverse till I get an answer.

On Wed, Aug 8, 2018 at 6:07 AM, Hariharan Sethuraman <[hidden email]> wrote:
Yes correct, the parent Proxy is a forward, but the squid will have to do both from client aspect.

Can I run two instances of squid - forward and reverse separately considering my configuration is good enough?

On Tue, 7 Aug 2018, 22:00 Amos Jeffries, <[hidden email]> wrote:
On 08/08/18 04:01, Hariharan Sethuraman wrote:
> Thanks Amos: yes agree that I should have told forward proxy.
>
> When I remove the originserver option from cache_peer, the forward proxy
> is working so which means the rewriter is not precluding from happening.
> Does that give any clue to us? 
>

Ah, cant believe I missed that. If the parent proxy is your access to
the Internet then is *not* a reverse-proxy. It cannot be and receive
proxy<->proxy traffic.

Any attempt to change the scheme is erased because the scheme is not
part of origin-form message syntax.


> Moreover the reverse proxy is in next hop to the client and not in
> internet. Time being, we are ok to have insecure channel between client
> and squid. Do you have any sample config that that uses a parent proxy
> to do both forward/reverse proxy? Or do you see my config is good enough
> for this requirement.
>

The traffic types have different syntax. It is possible to have a parent
proxy which receives both, but that has to be different ports and
different cache_peer links between them.

Amos



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Reverse proxy and TUNNEL to same cache peer

Amos Jeffries
Administrator
On 08/08/18 16:49, Hariharan Sethuraman wrote:
> I think giving name helped to fwd/reverse to same parent proxy port:
> cache_peer parent-proxy.domain.com <http://parent-proxy.domain.com>
> parent 80       0  no-query no-digest login=PASS originserver
> name=reverseproxy
> cache_peer parent-proxy.domain.com <http://parent-proxy.domain.com>
> parent 80       0  no-query no-digest login=PASS name=forwardproxy
>

The problem is what that parent proxy is doing with the requests it
receives. The cache_peer with "originserver" option sends origin-form
URLs, the other sends absolute-form URLs.

By delivering them to port 80 you can expect it to automatically
re-write the URLs back to the http:// scheme. Which fixes some safety
issues broken by your re-writer - by undoing what your re-writer is
trying to do.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users