While using icap_service squid working when ip is used and failing when domain name is provided

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

While using icap_service squid working when ip is used and failing when domain name is provided

prudhvisagar
Hi, 
Thanks for checking my message. 
 Please check the below configuration, we are running squid 3.5 version. 

This service is running on aws its a ui application trying to connect to virus scanner to scan the uploaded file and send the request to downstream application if the file is valid. 

We implemented squid before the virus scanner 
 
https_port 8443 accel defaultsite=imageuploadqa.com no-vhost cert=/qa/certificate/imageupload.cer key=/qa/certificate/private/imageupload.pem
cache_peer imageuploadroute53downstreamappkication.com. parent 443 0 proxy-only name=imageuploadAccel ssl sslflags=DONT_VERIFY_PEER
acl imageupload dstdomain imageuploadqa.com
http_access allow imageupload
cache_peer_access imageuploadAccel allow imageupload
cache_peer_access imageuploadAccel deny all
icap_enable on
icap_service service_avi_req reqmod_precache icap://domainnameofvirusscanner:1344/SYMCScanReqEx-AV bypass=off (not working, but working when we are trying to use the IP)
adaptation_access service_avi_req allow all
icap_log /var/log/squid/icap.log icap_squid


it also working when "cache_peer_access imageuploadAccel deny all" Line is removed

Please let me know if am missing any configuration 

Thanks. 


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: While using icap_service squid working when ip is used and failing when domain name is provided

Amos Jeffries
Administrator
On 13/08/19 3:55 am, Prudhvisagar Bellamkonda wrote:

> Hi, 
> Thanks for checking my message. 
>  Please check the below configuration, we are running squid 3.5 version. 
>
> This service is running on aws its a ui application trying to connect to
> virus scanner to scan the uploaded file and send the request to
> downstream application if the file is valid. 
>
> We implemented squid before the virus scanner 
>  
> https_port 8443 accel defaultsite=imageuploadqa.com no-vhost

Since this is a reverse-proxy it really should be listening on port 443
unless you have a good reason not to.

Do all these backend systems accept URLs of the form:
  https://imageuploadqa.com:8443/...

FYI: One of the major benefits of reverse-proxy is that they can protect
against garbage traffic for bogus domains etc aimed at your domain. The
no-vhost style config disables that protection completely.
 No matter what URL anyone sends to this proxy it will automatically
force re-write with that scheme://domain:port/ string before any
internal services and even Squids own ACLs get to see the traffic.


> cert=/qa/certificate/imageupload.cer
> key=/qa/certificate/private/imageupload.pem
> cache_peer imageuploadroute53downstreamappkication.com. parent 443 0 proxy-only
> name=imageuploadAccel ssl sslflags=DONT_VERIFY_PEER

Please remove that DONT_VERIFY_PEER. It is highly dangerous and actually
not useful.

Just add the sslcafile= option with a PEM file containing the CA(s)
which issued that peers X.509 certificate.


> acl imageupload dstdomain imageuploadqa.com
> http_access allow imageupload
> cache_peer_access imageuploadAccel allow imageupload
> cache_peer_access imageuploadAccel deny all
> icap_enable on
> icap_service service_avi_req reqmod_precache
> icap://domainnameofvirusscanner:1344/SYMCScanReqEx-AV bypass=off (not
> working, but working when we are trying to use the IP)

That is a very strong hint that the problem is DNS related.

Check both A and AAAA are resolving without a timeout or SERVFAIL
result. That the IP(s) produced are all able to be connected to by the
proxy machine OR connection attempts get a quick non-routable ICMP error
back.


> adaptation_access service_avi_req allow all
> icap_log /var/log/squid/icap.log icap_squid
>
>
> it also working when "cache_peer_access imageuploadAccel deny all" Line
> is removed

Very Odd. All that line is doing is making it clear to you what the
behaviour is for that peer.

>
> Please let me know if am missing any configuration 
>

Please explain "not working" in more detail - what do you see happening
exactly?

Is it;
 * failing to connect?
 * - does the domain name resolve properly when looked up by your Squid?
 * failing to send the ICAP request?
 * failing to get a response?
 * failing to deliver the response it gets?
 * is any of those a timeout or an explicit error seen by Squid?
 * is Squid producing any error message explaining the problem?
 * are there any hints in cache.log?


Lots of details please.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: While using icap_service squid working when ip is used and failing when domain name is provided

prudhvisagar
*Thanks a lot for replying back*

*"From the cache logs i see that its failing:" *

2019/08/13 01:08:02.809| 14,3| Address.cc(389) lookupHostIP: Given Non-IP
'domainnameofvirusscanner': Name or service not known
Address.cc(389) lookupHostIP: Given Non-IP 'domainnameofvirusscanner': Name
or service not known
essential ICAP service is down after an options fetch failure:
icap://domainnameofvirusscanner:1344/SYMCScanReq-AV [down,!opt]

*"At a diffrent location i can also see that internal dns was able to
resolve the addresses. "
"this domain name is a alb in aws, I am able to connect to the instance IP
under the ALB through squid"*

2019/08/13 01:08:02.852| 78,6| dns_internal.cc(1053) idnsCallback: Merging
DNS results domainnameofvirusscanner A has 3 RR, AAAA has 1 RR
2019/08/13 01:08:02.852| 45,9| cbdata.cc(321) cbdataInternalFree:
0x55a43f8b7218
2019/08/13 01:08:02.852| 45,9| cbdata.cc(338) cbdataInternalFree: Freeing
0x55a43f8b7218
2019/08/13 01:08:02.852| 78,6| dns_internal.cc(1086) idnsCallback: Sending 4
(OK) DNS results to caller.
2019/08/13 01:08:02.852| 45,9| cbdata.cc(492) cbdataReferenceValid:
0x55a43f85aaf8
2019/08/13 01:08:02.852| 45,9| cbdata.cc(426) cbdataInternalUnlock:
0x55a43f85aaf8=0
2019/08/13 01:08:02.852| 45,9| cbdata.cc(321) cbdataInternalFree:
0x55a43f85aaf8
2019/08/13 01:08:02.852| 45,9| cbdata.cc(338) cbdataInternalFree: Freeing
0x55a43f85aaf8
2019/08/13 01:08:02.852| 14,3| ipcache.cc(362) ipcacheParse: 4 answers for
'domainnameofvirusscanner'
2019/08/13 01:08:02.852| 14,3| ipcache.cc(420) ipcacheParse:
domainnameofvirusscanner #0 10.55.10.2
2019/08/13 01:08:02.852| 14,3| ipcache.cc(420) ipcacheParse:
domainnameofvirusscanner #1 10.55.10.3

----------

Squid error messages:

One more error i found was
2019/08/13 01:08:02.812| 20,3| store.cc(499) setReleaseFlag:
StoreEntry::setReleaseFlag: '[null_store_key]'
----------
2019/08/13 01:08:02.810| 93,5| AsyncJob.cc(123) callStart:
Adaptation::Icap::ServiceRep status in:[down,!opt,fetch]
2019/08/13 01:08:02.810| 45,9| cbdata.cc(492) cbdataReferenceValid:
0x5534xa43f8b9324ddb8
2019/08/13 01:08:02.810| 45,9| cbdata.cc(492) cbdataReferenceValid:
0x5534xa43f8b9324ddb8
2019/08/13 01:08:02.810| 45,9| cbdata.cc(426) cbdataInternalUnlock:
0x5534xa43f8b9324ddb8=0
2019/08/13 01:08:02.810| 45,9| cbdata.cc(449) cbdataInternalUnlock: Freeing
0x5534xa43f8b9324ddb8
2019/08/13 01:08:02.810| 93,3| ServiceRep.cc(534) noteAdaptationAnswer:
failed to fetch options [down,!opt]
2019/08/13 01:08:02.810| 93,8| ServiceRep.cc(448) changeOptions: changes
options from 0 to 0 [down,!opt]
2019/08/13 01:08:02.810| essential ICAP service is down after an options
fetch failure: icap://domainnameofvirusscanner:1344/SYMCScanReq-AV
[down,!opt]
----------

HTTP/1.1 500 Internal Server Error
Server: squid/3.5.20
Mime-Version: 1.0
Date: Tue, 13 Aug 2019 01:08:02 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3271
X-Squid-Error: ERR_ICAP_FAILURE 0
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from 5e1c4b7853860819
X-Cache-Lookup: NONE from 5e1c4b7854860819:8443
Via: 1.1 5ec4b7886089 (squid/3.5.20)
Connection: keep-alive

----------
2019/08/13 01:08:02.810| 93,5| Xaction.cc(92) disableRepeats:
Adaptation::Icap::ModXact still cannot be repeated because ICAP service is
unusable [G/R job8]
2019/08/13 01:08:02.810| 93,3| ../../../src/base/AsyncJobCalls.h(177) dial:
Adaptation::Icap::ModXact::noteServiceReady threw exception: ICAP service is
unusable


----------



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: While using icap_service squid working when ip is used and failing when domain name is provided

prudhvisagar
it also looks like DNS is able to resolve the domainname for the
virusscanner, but looks like by the time its resolved the ICAP_SERVICE
command failed.

if my understanding is right, Is there any configuration to do the dns
resolving first and before executing the below command
icap_service service_avi_req reqmod_precache
icap://domainnameofvirusscanner:1344/SYMCScanReqEx-AV bypass=off

Also we are using 3.5, let us know if we have to upgrade ?

Thanks a lot.





--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users