force full download and limit bandwith

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

force full download and limit bandwith

joseph
if the client   exit download  squid will force full speed on obj dose not
stay  at that rate

acl limit_ext_bandwith url_regex \.esd \.exe
delay_pools 1
delay_class 1 1
delay_parameters 1 128000/128000
delay_access 1 allow limit_ext_bandwith
delay_access 1 deny all

range_offset_limit -1 limit_ext_bandwith
quick_abort_min -1
quick_abort_max 0 KB
quick_abort_pct 100

this work fine  it keep downloading 128KB   until the client exit squid will
continue at full speed of the main bandwidth
is it bug  or im missing something ??
so i need to force cache full obj  at that specific rate  even if the client
exit
tks



-----
**************************
***** Crash to the future  ****
**************************
--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
************************** ***** Crash to the future **** **************************
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Amos Jeffries
Administrator
On 15/01/18 03:23, joseph wrote:

> if the client   exit download  squid will force full speed on obj dose not
> stay  at that rate
>
> acl limit_ext_bandwith url_regex \.esd \.exe
> delay_pools 1
> delay_class 1 1
> delay_parameters 1 128000/128000
> delay_access 1 allow limit_ext_bandwith
> delay_access 1 deny all
>
> range_offset_limit -1 limit_ext_bandwith
> quick_abort_min -1
> quick_abort_max 0 KB
> quick_abort_pct 100
>
> this work fine  it keep downloading 128KB   until the client exit squid will
> continue at full speed of the main bandwidth
> is it bug  or im missing something ??

No bug, that is how delay pools are designed. Squid always fetches as
fast as it can so as to finish with the more expensive network resource
(port numbers) faster and free them up for other traffic sooner.


> so i need to force cache full obj  at that specific rate  even if the client
> exit

Use QoS controls in your operating system for that. Delay pools in Squid
were designed long before QoS existed. Modern TOS/QoS capabilities are
much better than Squid can do.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

joseph
tks another question related to this

lets say client start downloading obj   and it reach 50% at the speed 128k
wen the client  leave or stop downloading the download continue at full
speed right from that 50% up to the end or it re start from the beginning ??
its important to understand those mixed combination delay_pools and
range_offset_limit   if they work as i tough or no tks



-----
**************************
***** Crash to the future  ****
**************************
--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
************************** ***** Crash to the future **** **************************
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Amos Jeffries
Administrator
On 18/01/18 00:41, joseph wrote:
> tks another question related to this
>
> lets say client start downloading obj   and it reach 50% at the speed 128k
> wen the client  leave or stop downloading the download continue at full
> speed right from that 50% up to the end or it re start from the beginning ??
> its important to understand those mixed combination delay_pools and
> range_offset_limit   if they work as i tough or no tks
>

Each I/O read() Squid performs in receiving a response is assessed as to
what buffer free space and pool speeds are affecting it. If a client
disappears midway the pool(s) that client caused to be applied no longer
are. So the transaction may speed up if the server and network can do
more speed.

"range_offset_limit N" only affects the initial starting point for a
transaction. If a client wants to start reading a range somewhere in the
first N bytes Squid will request the full file in order to cache it for
future requests. Otherwise only the range the client wants will be
requested - and cannot (yet) be cached.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

joseph
Amos Jeffries wrote
> "range_offset_limit N" only affects the initial starting point for a
> transaction. If a client wants to start reading a range somewhere in the
> first N bytes Squid will request the full file in order to cache it for
> future requests. Otherwise only the range the client wants will be
> requested - and cannot (yet) be cached.

meaning it will continue  dose not restart exelent that way file dose not
downloaded twice once wen active delay
and range_offset_limit none  
meaning one transaction happen



-----
**************************
***** Crash to the future  ****
**************************
--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
************************** ***** Crash to the future **** **************************
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Amos Jeffries
Administrator
On 18/01/18 01:47, joseph wrote:

> Amos Jeffries wrote
>> "range_offset_limit N" only affects the initial starting point for a
>> transaction. If a client wants to start reading a range somewhere in the
>> first N bytes Squid will request the full file in order to cache it for
>> future requests. Otherwise only the range the client wants will be
>> requested - and cannot (yet) be cached.
>
> meaning it will continue  dose not restart exelent that way file dose not
> downloaded twice once wen active delay
> and range_offset_limit none
> meaning one transaction happen
>

Note that none of this relates to parallel transactions for the same
object URL. The older your Squid is the more likely that it will be
doing multiple parallel fetches, especially as the object size goes up.

That can only be avoided with some changes in the latest Squid and the
collapsed_forwarding feature.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

joseph
using latest squid 5



-----
**************************
***** Crash to the future  ****
**************************
--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
************************** ***** Crash to the future **** **************************
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Alex Rousskov
In reply to this post by joseph
On 01/17/2018 05:47 AM, joseph wrote:
> Amos Jeffries wrote
>> "range_offset_limit N" only affects the initial starting point for a
>> transaction. If a client wants to start reading a range somewhere in the
>> first N bytes Squid will request the full file in order to cache it for
>> future requests. Otherwise only the range the client wants will be
>> requested - and cannot (yet) be cached.

> meaning it will continue  dose not restart

You have misinterpreted Amos's response: Whether Squid continues after
the client disappears depends on quick_abort_* settings, not
range_offset_limit settings.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Heiler Bemerguy

Everytime I enable this:

acl fullDLext urlpath_regex -i \.(exe|ms[iup]|cab|bin|zip|mar|pdf|appx(bundle)?|esd|lzma2)\??$
range_offset_limit -1 fullDLext
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 95

Our link is fully utilized for a whole day if at least 2 users tries to get the same file with range (206) requests (Windows Update does this)
Rockstore with SMP and 3.5.27

The file will be simultaneously downloaded from the beginning by each of workers that got the request, and in the end it won't be cached, giving a SWAPFAIL_MISS message.. This easily TOPS our link for a whole day (maybe till the users turn their computers off..?)

It's a known bug, as I filled a bug report years ago.. it was never fixed...


-- 
Atenciosamente / Best Regards,

Heiler Bemerguy
Network Manager - CINBESA
55 91 98151-4894/3184-1751


Em 17/01/2018 12:36, Alex Rousskov escreveu:
On 01/17/2018 05:47 AM, joseph wrote:
Amos Jeffries wrote
"range_offset_limit N" only affects the initial starting point for a 
transaction. If a client wants to start reading a range somewhere in the 
first N bytes Squid will request the full file in order to cache it for 
future requests. Otherwise only the range the client wants will be 
requested - and cannot (yet) be cached.

      
meaning it will continue  dose not restart 
You have misinterpreted Amos's response: Whether Squid continues after
the client disappears depends on quick_abort_* settings, not
range_offset_limit settings.


HTH,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: force full download and limit bandwith

Alex Rousskov
On 01/17/2018 09:15 AM, Heiler Bemerguy wrote:

> Everytime I enable this:
>
>     range_offset_limit -1 fullDLext
>     quick_abort_min 0 KB
>     quick_abort_max 0 KB
>     quick_abort_pct 95
>
> Our link is fully utilized for a whole day if at least 2 users tries to
> get the same file with range (206) requests (Windows Update does this)
> Rockstore with SMP and 3.5.27
>
> The file will be simultaneously downloaded from the beginning by each of
> workers that got the request, and in the end it won't be cached, giving
> a SWAPFAIL_MISS message.. This easily TOPS our link for a whole day
> (maybe till the users turn their computers off..?)
>
> It's a known bug, as I filled a bug report years ago.. it was never
> fixed...

To avoid misundertanding, there are at least three overlapping issues
here. Only one of them is a bug:

* The fact that Squid may fetch more data than is necessary when
configured to prefetch is not a bug. Both range_offset_limit and
quick_abort_* are essentially prefetching options -- they tell Squid to
fetch more than what the asking client needs.

* The fact that multiple workers may attempt concurrent downloads of the
same object is not necessarily a bug. Squid behavior in such cases is
controlled by collapsed_forwarding (and various HTTP rules).

* The fact that the concurrently fetched cachable object may never be
cached is a bug. It may be fixed as a side effect of the pending pull
request #46 changes, but I am not sure.

Alex.


> Em 17/01/2018 12:36, Alex Rousskov escreveu:
>> On 01/17/2018 05:47 AM, joseph wrote:
>>> Amos Jeffries wrote
>>>> "range_offset_limit N" only affects the initial starting point for a
>>>> transaction. If a client wants to start reading a range somewhere in the
>>>> first N bytes Squid will request the full file in order to cache it for
>>>> future requests. Otherwise only the range the client wants will be
>>>> requested - and cannot (yet) be cached.
>>> meaning it will continue  dose not restart
>> You have misinterpreted Amos's response: Whether Squid continues after
>> the client disappears depends on quick_abort_* settings, not
>> range_offset_limit settings.
>>
>>
>> HTH,
>>
>> Alex.
>> _______________________________________________
>> squid-users mailing list
>> [hidden email]
>> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users