Squid 3.5.27 not caching at all

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Squid 3.5.27 not caching at all

Mohamed Ali Ahmed
Hello everyone,
I have set up Squid 3.5.27 on ubuntu 18.04 from the packages. I have made the minimal change but when i check the access.log i get tcp_miss 200 most of the websites even visiting the same website over and over again.

I have searched online and i did not come across any which solves my problem. 

this is squid.conf file:

root@squid:/etc/squid# sed 's/#.*//;s/^\s*$//;/^$/d' /etc/squid/squid.conf
 acl NOC src 154.73.26.5
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost NOC
http_access allow all
http_port 3128
cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 3600       90%     43200
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320
root@squid:/etc/squid#

here's access.log file:

1564825166.134    426 154.73.26.5 TCP_MISS/200 10644 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.layeranimation.min.js - HIER_DIRECT/192.185.31.191 application/javascript
1564825166.142    436 154.73.26.5 TCP_MISS/200 8613 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.slideanims.min.js - HIER_DIRECT/192.185.31.191 application/javascript
1564825166.150    440 154.73.26.5 TCP_MISS/200 8771 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.navigation.min.js - HIER_DIRECT/192.185.31.191 application/javascript
1564825166.156    450 154.73.26.5 TCP_MISS/200 2892 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.actions.min.js - HIER_DIRECT/192.185.31.191 application/javascript
1564825166.804    454 154.73.26.5 TCP_MISS/200 7970 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/fonts/revicons/revicons.woff? - HIER_DIRECT/192.185.31.191 font/woff
1564825166.821    437 154.73.26.5 TCP_MISS/200 2982 GET http://somaliren.org/wp-content/plugins/revslider/public/assets/assets/loader.gif - HIER_DIRECT/192.185.31.191 image/gif


Here's cach.log file:

root@squid:/etc/squid# tail /var/log/squid/cache.log
2019/08/03 12:05:00 kid1|         0 Objects expired.
2019/08/03 12:05:00 kid1|         0 Objects cancelled.
2019/08/03 12:05:00 kid1|         0 Duplicate URLs purged.
2019/08/03 12:05:00 kid1|         0 Swapfile clashes avoided.
2019/08/03 12:05:00 kid1|   Took 0.10 seconds (  0.00 objects/sec).
2019/08/03 12:05:00 kid1| Beginning Validation Procedure
2019/08/03 12:05:00 kid1|   Completed Validation Procedure
2019/08/03 12:05:00 kid1|   Validated 0 Entries
2019/08/03 12:05:00 kid1|   store_swap_size = 0.00 KB
2019/08/03 12:05:01 kid1| storeLateRelease: released 0 objects

Any help would be appreciated. 

thanks,
Mohamed

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid 3.5.27 not caching at all

Amos Jeffries
Administrator
On 3/08/19 9:45 pm, Mohamed Ali Ahmed wrote:
> Hello everyone,
> I have set up Squid 3.5.27 on ubuntu 18.04 from the packages. I have
> made the minimal change but when i check the access.log i get tcp_miss
> 200 most of the websites even visiting the same website over and over again.
>

You have not provided enough information for us to know whether this is
a problem or testing mistake.

What is this unstated "minimal change" you made that made caching
suddenly stop working?

What exactly does "visiting the same website over and over again" mean?
what exact actions did you do?
  - Pressing refresh, Shift+reload, or CTL+refresh in a browser will
*force* a MISS - yet comes under the description you gave.
  - some tools send headers to forbid cached contents being delivered by
default. So cannot be used for this kind of test without special options.


The more specific details you can give, the better anyone help can be.


> I have searched online and i did not come across any which solves my
> problem. 
>
> this is squid.conf file:
>
> root@squid:/etc/squid# sed 's/#.*//;s/^\s*$//;/^$/d' /etc/squid/squid.conf
>  acl NOC src 154.73.26.5
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 21
> acl Safe_ports port 443
> acl Safe_ports port 70
> acl Safe_ports port 210
> acl Safe_ports port 1025-65535
> acl Safe_ports port 280
> acl Safe_ports port 488
> acl Safe_ports port 591
> acl Safe_ports port 777
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost NOC
> http_access allow all

Please do not do that "allow all".

For one thing it makes your NOC ACL pointless, *everybody* is allowed to
use your proxy for whatever they want to do.

If you did it to debug a caching problem, be aware that http_access has
nothing to do with cache.


Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid 3.5.27 not caching at all

Mohamed Ali Ahmed
In reply to this post by Mohamed Ali Ahmed
You have not provided enough information for us to know whether this is
a problem or testing mistake.

What is this unstated "minimal change" you made that made caching
suddenly stop working? 
 The only change i have made is adding NOC ACL and uncommenting the cache_dir ufs.

What exactly does "visiting the same website over and over again" mean?
what exact actions did you do?
I expected when i visit a website for the time it will miss and after retrieving it that squid will cache it internally. if i visit the same website again, squid to read it from the local cache. i was visiting by adding a new tap using chrome or visiting from another browser like MS edge or IE. 

Please do not do that "allow all".

For one thing it makes your NOC ACL pointless, *everybody* is allowed to
use your proxy for whatever they want to do.

If you did it to debug a caching problem, be aware that http_access has
nothing to do with cache. 
Thank you amos, I have changed to  http_access deny all   instead of http_access allow all
 Please ask me anything else i should share with so that you can help me. thank you. 
 

 - Pressing refresh, Shift+reload, or CTL+refresh in a browser will
*force* a MISS - yet comes under the description you gave.
  - some tools send headers to forbid cached contents being delivered by
default. So cannot be used for this kind of test without special options.  
So what do you recommend i do in order to check if squid is caching or not?
 


On Sat, Aug 3, 2019 at 3:02 PM <[hidden email]> wrote:
Send squid-users mailing list submissions to
        [hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.squid-cache.org/listinfo/squid-users
or, via email, send a message with subject or body 'help' to
        [hidden email]

You can reach the person managing the list at
        [hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of squid-users digest..."


Today's Topics:

   1. Re: sending certificate chain from squid reverse proxy
      (Martin Hoffmann)
   2. Squid 3.5.27 not caching at all (Mohamed Ali Ahmed)
   3. Re: Squid 3.5.27 not caching at all (Amos Jeffries)


----------------------------------------------------------------------

Message: 1
Date: Fri, 2 Aug 2019 13:08:50 -0500 (CDT)
From: Martin Hoffmann <[hidden email]>
To: [hidden email]
Subject: Re: [squid-users] sending certificate chain from squid
        reverse proxy
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=us-ascii

Any ETA on this?

This would really be a nice feature, since on debian/ubuntu squid comes with
GnuTLS support, while OpenSSL Support means recompiling from source. So for
me, it seems, intermediate certificates is the last thing missing to be able
to use squid/GnuTLS instead of squid/OpenSSL.



--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html


------------------------------

Message: 2
Date: Sat, 3 Aug 2019 12:45:14 +0300
From: Mohamed Ali Ahmed <[hidden email]>
To: [hidden email]
Subject: [squid-users] Squid 3.5.27 not caching at all
Message-ID:
        <CAK4=zavgzFG=yMv=+SCid3FB5VnNfXE5xYs5361r8PeE-c=[hidden email]>
Content-Type: text/plain; charset="utf-8"

Hello everyone,
I have set up Squid 3.5.27 on ubuntu 18.04 from the packages. I have made
the minimal change but when i check the access.log i get tcp_miss 200 most
of the websites even visiting the same website over and over again.

I have searched online and i did not come across any which solves my
problem.

this is squid.conf file:

root@squid:/etc/squid# sed 's/#.*//;s/^\s*$//;/^$/d' /etc/squid/squid.conf
 acl NOC src 154.73.26.5
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost NOC
http_access allow all
http_port 3128
cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 3600       90%     43200
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320
root@squid:/etc/squid#

here's access.log file:

1564825166.134    426 154.73.26.5 TCP_MISS/200 10644 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.layeranimation.min.js
- HIER_DIRECT/192.185.31.191 application/javascript
1564825166.142    436 154.73.26.5 TCP_MISS/200 8613 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.slideanims.min.js
- HIER_DIRECT/192.185.31.191 application/javascript
1564825166.150    440 154.73.26.5 TCP_MISS/200 8771 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.navigation.min.js
- HIER_DIRECT/192.185.31.191 application/javascript
1564825166.156    450 154.73.26.5 TCP_MISS/200 2892 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/js/extensions/revolution.extension.actions.min.js
- HIER_DIRECT/192.185.31.191 application/javascript
1564825166.804    454 154.73.26.5 TCP_MISS/200 7970 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/fonts/revicons/revicons.woff?
- HIER_DIRECT/192.185.31.191 font/woff
1564825166.821    437 154.73.26.5 TCP_MISS/200 2982 GET
http://somaliren.org/wp-content/plugins/revslider/public/assets/assets/loader.gif
- HIER_DIRECT/192.185.31.191 image/gif


Here's cach.log file:

root@squid:/etc/squid# tail /var/log/squid/cache.log
2019/08/03 12:05:00 kid1|         0 Objects expired.
2019/08/03 12:05:00 kid1|         0 Objects cancelled.
2019/08/03 12:05:00 kid1|         0 Duplicate URLs purged.
2019/08/03 12:05:00 kid1|         0 Swapfile clashes avoided.
2019/08/03 12:05:00 kid1|   Took 0.10 seconds (  0.00 objects/sec).
2019/08/03 12:05:00 kid1| Beginning Validation Procedure
2019/08/03 12:05:00 kid1|   Completed Validation Procedure
2019/08/03 12:05:00 kid1|   Validated 0 Entries
2019/08/03 12:05:00 kid1|   store_swap_size = 0.00 KB
2019/08/03 12:05:01 kid1| storeLateRelease: released 0 objects

Any help would be appreciated.

thanks,
Mohamed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20190803/28c6e860/attachment-0001.html>

------------------------------

Message: 3
Date: Sat, 3 Aug 2019 22:36:28 +1200
From: Amos Jeffries <[hidden email]>
To: [hidden email]
Subject: Re: [squid-users] Squid 3.5.27 not caching at all
Message-ID: <[hidden email]>
Content-Type: text/plain; charset=utf-8

On 3/08/19 9:45 pm, Mohamed Ali Ahmed wrote:
> Hello everyone,
> I have set up Squid 3.5.27 on ubuntu 18.04 from the packages. I have
> made the minimal change but when i check the access.log i get tcp_miss
> 200 most of the websites even visiting the same website over and over again.
>

You have not provided enough information for us to know whether this is
a problem or testing mistake.

What is this unstated "minimal change" you made that made caching
suddenly stop working?

What exactly does "visiting the same website over and over again" mean?
what exact actions did you do?
  - Pressing refresh, Shift+reload, or CTL+refresh in a browser will
*force* a MISS - yet comes under the description you gave.
  - some tools send headers to forbid cached contents being delivered by
default. So cannot be used for this kind of test without special options.


The more specific details you can give, the better anyone help can be.


> I have searched online and i did not come across any which solves my
> problem. 
>
> this is squid.conf file:
>
> root@squid:/etc/squid# sed 's/#.*//;s/^\s*$//;/^$/d' /etc/squid/squid.conf
>  acl NOC src 154.73.26.5
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 21
> acl Safe_ports port 443
> acl Safe_ports port 70
> acl Safe_ports port 210
> acl Safe_ports port 1025-65535
> acl Safe_ports port 280
> acl Safe_ports port 488
> acl Safe_ports port 591
> acl Safe_ports port 777
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost NOC
> http_access allow all

Please do not do that "allow all".

For one thing it makes your NOC ACL pointless, *everybody* is allowed to
use your proxy for whatever they want to do.

If you did it to debug a caching problem, be aware that http_access has
nothing to do with cache.


Amos


------------------------------

Subject: Digest Footer

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users


------------------------------

End of squid-users Digest, Vol 60, Issue 4
******************************************


--


Mohamed Ali Ahmed (Bile)
Head of Systems
Somali Research & Education Network (SomaliREN)
+252 615 567671
Skype: bilesuper

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Squid 3.5.27 not caching at all

Amos Jeffries
Administrator
On 4/08/19 1:18 am, Mohamed Ali Ahmed wrote:
>     You have not provided enough information for us to know whether this is
>     a problem or testing mistake.
>
>     What is this unstated "minimal change" you made that made caching
>     suddenly stop working? 
>
>  The only change i have made is adding NOC ACL and uncommenting the
> cache_dir ufs.
>

Okay.

>     What exactly does "visiting the same website over and over again" mean?
>     what exact actions did you do?
>
> I expected when i visit a website for the time it will miss and after
> retrieving it that squid will cache it internally. if i visit the same
> website again, squid to read it from the local cache. i was visiting by
> adding a new tap using chrome or visiting from another browser like MS
> edge or IE. 
>

Ah. This website uses "Vary: Accept-Encoding" and each web browser
supports different types of encoding, or has different preference order
for the ones they do share.

So the first request from each browser should be expected to be a MISS.
Only the second use by that particular browser can be expected to HIT or
REFRESH. I'm not sure if Chrome new-tab does the same, it should not but
maybe.
You can add "debug_options 11,2" to get a cache.log trace of the headers
from each test and see if they should have been a MISS, HIT or REFRESH.



>     Please do not do that "allow all".
>
>     For one thing it makes your NOC ACL pointless, *everybody* is allowed to
>     use your proxy for whatever they want to do.
>
>     If you did it to debug a caching problem, be aware that http_access has
>     nothing to do with cache. 
>
> Thank you amos, I have changed to  http_access deny all   instead of
> http_access allow all
>  Please ask me anything else i should share with so that you can help
> me. thank you. 
>

The 11,2 debug trace log would help. You may be able to find the problem
yourself from that.


>
>      - Pressing refresh, Shift+reload, or CTL+refresh in a browser will
>     *force* a MISS - yet comes under the description you gave.
>       - some tools send headers to forbid cached contents being delivered by
>     default. So cannot be used for this kind of test without special
>     options.  
>
> So what do you recommend i do in order to check if squid is caching or not?
>  

If you need to use a browser, then clicking on the address bar and
pressing enter re-fetches that URL without any cache affecting requirements.

Or curl, wget or squidclient tools can be used to test a single URL fetch.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users