Backing up squid cache and restoring it

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Backing up squid cache and restoring it

Arne-Tobias Rak

Hi,

my goal is to restore a previous cache state in squid 3.x running on Ubuntu 16.04.

So far I have tried to create a copy of the /var/spool/squid and /var/log/squid folders.
When restoring the cache, I first shutdown squid using
sudo squid -k shutdown
sudo service squid stop -k
and then restore the previously copied folder contents. I then start squid again using
sudo service squid start.

Unfortunately, this does not restore the previous cache contents, as the spool/squid/swap.state
file is modified during squid startup.

Thanks in advance!
Arne


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Backing up squid cache and restoring it

Alex Rousskov
On 1/17/19 6:47 AM, Arne-Tobias Rak wrote:

> my goal is to restore a previous cache state in squid 3.x running on
> Ubuntu 16.04.
>
> So far I have tried to create a copy of the /var/spool/squid and
> /var/log/squid folders.
> When restoring the cache, I first shutdown squid using
> /sudo squid -k shutdown//
> //sudo service squid stop -k
> /and then restore the previously copied folder contents. I then start
> squid again using
> /sudo service squid start./

> Unfortunately, this does not restore the previous cache contents, as the
> spool/squid/swap.state file is modified during squid startup.

Modification of swap.state upon startup is not incompatible with cache
contents preservation.

In a clean shutdown context, the swap.state* files are essentially an
optimization. You may preserve/restore them if you want to speed up
building of the restored cache index OR you can delete them (and Squid
will slowly build a new cache index from scratch). Just do not leave the
newer swap.state* files when trying to restore the old cache.

If you preserve/restore the old swap.state* files, you may need to
preserve their timestamps as well.


If you need further help, please share cache.log entries related to
cache_dir loading and indicate why you think the old cache contents is
not preserved. Sharing your cache_dir configuration may also help.


Cheers,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Backing up squid cache and restoring it

Eliezer Croitoru
Just to make sure:
This swap.state behavior is only for UFS/AUFS cache_dir or also to rock?

Thanks,
Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: [hidden email]



-----Original Message-----
From: squid-users [mailto:[hidden email]] On Behalf Of Alex Rousskov
Sent: Thursday, January 17, 2019 19:23
To: [hidden email]
Subject: Re: [squid-users] Backing up squid cache and restoring it

On 1/17/19 6:47 AM, Arne-Tobias Rak wrote:

> my goal is to restore a previous cache state in squid 3.x running on
> Ubuntu 16.04.
>
> So far I have tried to create a copy of the /var/spool/squid and
> /var/log/squid folders.
> When restoring the cache, I first shutdown squid using
> /sudo squid -k shutdown//
> //sudo service squid stop -k
> /and then restore the previously copied folder contents. I then start
> squid again using
> /sudo service squid start./

> Unfortunately, this does not restore the previous cache contents, as the
> spool/squid/swap.state file is modified during squid startup.

Modification of swap.state upon startup is not incompatible with cache
contents preservation.

In a clean shutdown context, the swap.state* files are essentially an
optimization. You may preserve/restore them if you want to speed up
building of the restored cache index OR you can delete them (and Squid
will slowly build a new cache index from scratch). Just do not leave the
newer swap.state* files when trying to restore the old cache.

If you preserve/restore the old swap.state* files, you may need to
preserve their timestamps as well.


If you need further help, please share cache.log entries related to
cache_dir loading and indicate why you think the old cache contents is
not preserved. Sharing your cache_dir configuration may also help.


Cheers,

Alex.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Backing up squid cache and restoring it

Alex Rousskov
On 1/20/19 3:04 PM, Eliezer Croitoru wrote:

> This swap.state behavior is only for UFS/AUFS cache_dir or also to rock?

Rock cache_dirs do not use swap.state.

Alex.


> -----Original Message-----
> From: squid-users [mailto:[hidden email]] On Behalf Of Alex Rousskov
> Sent: Thursday, January 17, 2019 19:23
> To: [hidden email]
> Subject: Re: [squid-users] Backing up squid cache and restoring it
>
> On 1/17/19 6:47 AM, Arne-Tobias Rak wrote:
>
>> my goal is to restore a previous cache state in squid 3.x running on
>> Ubuntu 16.04.
>>
>> So far I have tried to create a copy of the /var/spool/squid and
>> /var/log/squid folders.
>> When restoring the cache, I first shutdown squid using
>> /sudo squid -k shutdown//
>> //sudo service squid stop -k
>> /and then restore the previously copied folder contents. I then start
>> squid again using
>> /sudo service squid start./
>
>> Unfortunately, this does not restore the previous cache contents, as the
>> spool/squid/swap.state file is modified during squid startup.
>
> Modification of swap.state upon startup is not incompatible with cache
> contents preservation.
>
> In a clean shutdown context, the swap.state* files are essentially an
> optimization. You may preserve/restore them if you want to speed up
> building of the restored cache index OR you can delete them (and Squid
> will slowly build a new cache index from scratch). Just do not leave the
> newer swap.state* files when trying to restore the old cache.
>
> If you preserve/restore the old swap.state* files, you may need to
> preserve their timestamps as well.
>
>
> If you need further help, please share cache.log entries related to
> cache_dir loading and indicate why you think the old cache contents is
> not preserved. Sharing your cache_dir configuration may also help.
>
>
> Cheers,
>
> Alex.
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
>

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Backing up squid cache and restoring it

Arne-Tobias Rak
In reply to this post by Alex Rousskov
Thank you for your help. I eventually managed to get it working. The
problem was related to stopping the squid service using the -k argument.

Closing squid using

sudo service squid stop

allows me to restore previous cache contents without any issues.


Am 17.01.2019 um 18:22 schrieb Alex Rousskov:

> On 1/17/19 6:47 AM, Arne-Tobias Rak wrote:
>
>> my goal is to restore a previous cache state in squid 3.x running on
>> Ubuntu 16.04.
>>
>> So far I have tried to create a copy of the /var/spool/squid and
>> /var/log/squid folders.
>> When restoring the cache, I first shutdown squid using
>> /sudo squid -k shutdown//
>> //sudo service squid stop -k
>> /and then restore the previously copied folder contents. I then start
>> squid again using
>> /sudo service squid start./
>> Unfortunately, this does not restore the previous cache contents, as the
>> spool/squid/swap.state file is modified during squid startup.
> Modification of swap.state upon startup is not incompatible with cache
> contents preservation.
>
> In a clean shutdown context, the swap.state* files are essentially an
> optimization. You may preserve/restore them if you want to speed up
> building of the restored cache index OR you can delete them (and Squid
> will slowly build a new cache index from scratch). Just do not leave the
> newer swap.state* files when trying to restore the old cache.
>
> If you preserve/restore the old swap.state* files, you may need to
> preserve their timestamps as well.
>
>
> If you need further help, please share cache.log entries related to
> cache_dir loading and indicate why you think the old cache contents is
> not preserved. Sharing your cache_dir configuration may also help.
>
>
> Cheers,
>
> Alex.
> _______________________________________________
> squid-users mailing list
> [hidden email]
> http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|

Re: Backing up squid cache and restoring it

Amos Jeffries
Administrator
On 24/01/19 10:50 pm, Arne-Tobias Rak wrote:
> Thank you for your help. I eventually managed to get it working. The
> problem was related to stopping the squid service using the -k argument.
>
> Closing squid using
>
> sudo service squid stop
>
> allows me to restore previous cache contents without any issues.
>

Hmm, that does not make sense to me.

Squid-3.x are not compatible with systemd. They only way to use Squid-3
with a .service setup is for that to be linked up to use the "-k
shutdown" commands internally. Even then systemd's default SIGKILL
behaviour and confusion over what PID it is supposed to be monitoring
can result in problems anyway.

Please switch to Squid-4 if you want to use systemd controls.

Amos
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users