a specific host generates a 503 ...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

a specific host generates a 503 ...

Walter H.
Hello,

can someone test the following URL

http://db.local.clamav.net/daily-26102.cdiff

e.g.   wget http://db.local.clamav.net/daily-26102.cdiff

I have an older squid (v3.1) there this works,
but with the newer ones (v3.4 and v3.5) this doesn't;

is there an explanation why?

the log shows this:

client-ip - - [10/Mar/2021:06:43:50 +0100] "GET
http://db.local.clamav.net/daily-26102.cdiff HTTP/1.0" 503 8645 "-"
"Wget/1.12 (linux-gnu)" TCP_MISS:HIER_DIRECT

the suspicious thing: when using a browser: this works with any squid,
but this doesn't help because the clamav signature updates are loaded
by the freshclam which shows the  same failure as e.g. wget

client-ip - - [09/Mar/2021:06:00:03 +0100] "GET
http://db.local.clamav.net/daily-26102.cdiff HTTP/1.0" 503 8642 "-"
"ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)"
TCP_MISS:HIER_DIRECT

I noticed this two days after the nightly freshclam (signature update)
failure,
and changed the freshclam config to use the squid v3.1;

Thanks,
Walter

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: a specific host generates a 503 ...

Eliezer Croitoru-3
Hey Walter,

It's sitting behind:  DDoS protection by Cloudflare
So it makes sense that you would not be able to download it using wget.
The only option probably is using a web browser.
I would suggest contacting clamav.net web/system admins to verify what are the options.

All The Bests,
Eliezer

----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: [hidden email]
Zoom: Coming soon


-----Original Message-----
From: squid-users <[hidden email]> On Behalf Of Walter H.
Sent: Wednesday, March 10, 2021 7:55 AM
To: Squid Users <[hidden email]>
Subject: [squid-users] a specific host generates a 503 ...

Hello,

can someone test the following URL

http://db.local.clamav.net/daily-26102.cdiff

e.g.   wget http://db.local.clamav.net/daily-26102.cdiff

I have an older squid (v3.1) there this works,
but with the newer ones (v3.4 and v3.5) this doesn't;

is there an explanation why?

the log shows this:

client-ip - - [10/Mar/2021:06:43:50 +0100] "GET
http://db.local.clamav.net/daily-26102.cdiff HTTP/1.0" 503 8645 "-"
"Wget/1.12 (linux-gnu)" TCP_MISS:HIER_DIRECT

the suspicious thing: when using a browser: this works with any squid,
but this doesn't help because the clamav signature updates are loaded
by the freshclam which shows the  same failure as e.g. wget

client-ip - - [09/Mar/2021:06:00:03 +0100] "GET
http://db.local.clamav.net/daily-26102.cdiff HTTP/1.0" 503 8642 "-"
"ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)"
TCP_MISS:HIER_DIRECT

I noticed this two days after the nightly freshclam (signature update)
failure,
and changed the freshclam config to use the squid v3.1;

Thanks,
Walter

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: a specific host generates a 503 ...

Amos Jeffries
Administrator
On 12/03/21 1:14 am, Eliezer Croitoru wrote:
> Hey Walter,
>
> It's sitting behind:  DDoS protection by Cloudflare
> So it makes sense that you would not be able to download it using wget.
> The only option probably is using a web browser.
> I would suggest contacting clamav.net web/system admins to verify what are the options.
>

FWIW, the tools I use seem to fetch it fine when adding the header
"User-Agent: ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)".

Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: a specific host generates a 503 ...

Walter H.
On 11.03.2021 15:33, Amos Jeffries wrote:

> On 12/03/21 1:14 am, Eliezer Croitoru wrote:
>> Hey Walter,
>>
>> It's sitting behind:  DDoS protection by Cloudflare
>> So it makes sense that you would not be able to download it using wget.
>> The only option probably is using a web browser.
>> I would suggest contacting clamav.net web/system admins to verify
>> what are the options.
>>
>
> FWIW, the tools I use seem to fetch it fine when adding the header
> "User-Agent: ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)".
the freshclam updater does adding this User-Agent, and fails on the
newer squids, only the older ones succeeeds;

and wget succeeds using the older squid, too?
(without adding a User-Agent)

why is that?

Thanks,
Walter



_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: a specific host generates a 503 ...

Matus UHLAR - fantomas
>>On 12/03/21 1:14 am, Eliezer Croitoru wrote:
>>>It's sitting behind:  DDoS protection by Cloudflare
>>>So it makes sense that you would not be able to download it using wget.
>>>The only option probably is using a web browser.
>>>I would suggest contacting clamav.net web/system admins to verify
>>>what are the options.

>On 11.03.2021 15:33, Amos Jeffries wrote:
>>FWIW, the tools I use seem to fetch it fine when adding the header
>>"User-Agent: ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU:
>>x86_64)".

On 13.03.21 13:03, Walter H. wrote:
>the freshclam updater does adding this User-Agent, and fails on the
>newer squids, only the older ones succeeeds;
>
>and wget succeeds using the older squid, too?
>(without adding a User-Agent)
>
>why is that?

due to huge abuse from web fetchers like wget, clamav has recently blocked
fetching virus databases by non-freshclam clients and freshclam older than
0.100:
https://lists.clamav.net/pipermail/clamav-users/2021-March/010578.html


I can't really tell why or how it worked with older squid releases (maybe
clearing the user-agent?), but you should definitely download ClamAV updates
using recent freshclam.

Please, upgrade to recent ClamAV first, or set up cvdupdate:
https://lists.clamav.net/pipermail/clamav-users/2021-March/010613.html

If that doesn't work, contact the clamav - changes are, you are blacklisted
from fetching updates.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: a specific host generates a 503 ...

Walter H.
On 15.03.2021 10:14, Matus UHLAR - fantomas wrote:

>>> On 12/03/21 1:14 am, Eliezer Croitoru wrote:
>>>> It's sitting behind:  DDoS protection by Cloudflare
>>>> So it makes sense that you would not be able to download it using
>>>> wget.
>>>> The only option probably is using a web browser.
>>>> I would suggest contacting clamav.net web/system admins to verify
>>>> what are the options.
>
>> On 11.03.2021 15:33, Amos Jeffries wrote:
>>> FWIW, the tools I use seem to fetch it fine when adding the header
>>> "User-Agent: ClamAV/0.103.1 (OS: linux-gnu, ARCH: x86_64, CPU:
>>> x86_64)".
>
>
> due to huge abuse from web fetchers like wget, clamav has recently
> blocked
> fetching virus databases by non-freshclam clients and freshclam older
> than
> 0.100:
> https://lists.clamav.net/pipermail/clamav-users/2021-March/010578.html
>
I found out, my older squid was the only squid, not clearing the User-Agent;

thanks for the infos;

now it works again with the originally used squid;

Thanks,
Walter




_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

smime.p7s (4K) Download Attachment