Squid Redirection Scripting

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Squid Redirection Scripting

Willsz.net

Hi, folks

Hi, I used Perl Scripting for 1 year couple ago. But this script doesn't
work with new version Squid.

root:~# uname -smrm
FreeBSD 9.3-STABLE i386

root:~# squid -v
Squid Cache: Version 3.5.24
Service Name: squid
configure options:  '--prefix=/usr/local'
'--includedir=/usr/local/include' '--bindir=/usr/local/sbin'
'--libexecdir=/usr/local/libexec/squid'
'--sysconfdir=/usr/local/etc/squid' '--with-default-user=squid'
'--localstatedir=/var/cache/squid' '--libdir=/usr/local/lib'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--with-swapdir=/var/cache/squid' '--without-gnutls'
'--enable-build-info' '--enable-loadable-modules'
'--enable-removal-policies=lru,heap' '--disable-epoll'
'--disable-linux-netfilter' '--disable-linux-tproxy'
'--disable-translation' '--disable-arch-native'
'--mandir=/usr/local/man' '--infodir=/usr/local/info' '--disable-wccp'
'--disable-wccpv2' '--enable-ipfw-transparent' '--enable-ssl-crtd'
'--with-openssl' '--with-large-files' '--disable-htcp' '--disable-eui'
'--enable-cachemgr-hostname=ip.proxy-cache.willsz.net'
'--disable-auth-negotiate' --enable-ltdl-convenience

Here's my script:

#!/usr/local/bin/perl

$|=1;

while (<>) {
chomp;
@X = split;
$url = $X[0];

     if ( $url =~ m{ ^https?://(?:
         ad[0-9]?        |
         ad[szvx]?[0-9]? |
         ads[0-9]?       |
         adsbox          |
         adserver        |
         adservices?     |
         adserving       |
         adv             |
         advertising     |
         banners?        |
         adserver)\.(.*) }x ) {
             print "http://somedomain.com/null.png\n";
     }

     else {
         print "$url\n";
     }
}

Anyone guide me to make it work?

Thank you.
_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Squid Redirection Scripting

Amos Jeffries
Administrator
On 10/04/2017 8:26 p.m., Willsz.net wrote:
>
> Hi, folks
>
> Hi, I used Perl Scripting for 1 year couple ago. But this script doesn't
> work with new version Squid.

Please explain "doesn't work".
Does it make all your traffic divert to cat pictures now?
  something else?


What was your previous version of Squid.
 2.7?  3.5.23?  something else?


>
> root:~# uname -smrm
> FreeBSD 9.3-STABLE i386
>
> root:~# squid -v
> Squid Cache: Version 3.5.24
> Service Name: squid
...
>  '--with-openssl'

NP: Since this is a self-build with OpenSSL I recommend upgrade to
3.5.25. For other reasons that your current problem, but it may solve both.


>
> Here's my script:
>
> #!/usr/local/bin/perl
>
> $|=1;
>
> while (<>) {
> chomp;
> @X = split;
> $url = $X[0];
>
>     if ( $url =~ m{ ^https?://(?:
>         ad[0-9]?        |
>         ad[szvx]?[0-9]? |
>         ads[0-9]?       |
>         adsbox          |
>         adserver        |
>         adservices?     |
>         adserving       |
>         adv             |
>         advertising     |
>         banners?        |
>         adserver)\.(.*) }x ) {

Does that regex actually work?

1) I've never known URLs to contain long series of whitespaces like that.

2) ad[szvx]?[0-9]? matches the same things as ad[0-9]? and ads[0-9]? and
adv.



>             print "http://somedomain.com/null.png\n";
>     }
>
>     else {
>         print "$url\n";
>     }
> }
>
> Anyone guide me to make it work?
>

From the overall structure and output I am guessing that this is a Squid
URL re-writer or Store-ID helper.


IMO you should replace this simple helper with ACLs. Like so:

squid.conf:
 acl adverts_domain dtsdom_regex "/etc/squid/adverts_domain.regex"
 deny_info 302:http://somedomain.com/null.png adverts_domain
 http_access deny adverts_domain

/etc/squid/adverts_domain.regex:
   ad[szvx]?[0-9]?\.
   adsbox\.
   adserv(er|ing|ices?)\.
   adv(ertising)?\.
   banners?\.


Amos

_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Squid Redirection Scripting

Willsz.net
On 11/04/2017 8:48, Amos Jeffries wrote:

> On 10/04/2017 8:26 p.m., Willsz.net wrote:
>>
>> Hi, folks
>>
>> Hi, I used Perl Scripting for 1 year couple ago. But this script doesn't
>> work with new version Squid.
>
> Please explain "doesn't work".
> Does it make all your traffic divert to cat pictures now?
>   something else?

Thank Amos,

Let me explain more detail, something like this:

root:~# cat /usr/local/bin/squid-rdr
#!/usr/local/bin/perl

$|=1;

while (<>) {
chomp;
@X = split;
$url = $X[0];

     if ( $url =~ m{ ^https?://(?:
         ad[0-9]?        |
         ad[szvx]?[0-9]? |
         ads[0-9]?       |
         adsbox          |
         adserver        |
         adservices?     |
         adserving       |
         adv             |
         advertising     |
         banners?        |
         adserver)\.(.*) }x ) {
             print "http://somedomain.com/null.png\n";
     }

     else {
         print "$url\n";
     }
}

root:~# chown squid:squid /usr/local/bin/squid-rdr
root:~# chmod 755 /usr/local/bin/squid-rdr

Afterward, I add in /usr/local/etc/squid/squid.conf:

url_rewrite_program /usr/local/bin/squid-rdr
url_rewrite_children 32 startup=0 idle=1 concurrency=0
url_rewrite_bypass on

root:~# /usr/local/etc/rc.d/squid.sh reload
Reload proxy system...

Trying to browse some URL, so I get this error message and I failed to
retrieve web-page.

root:~# tail /var/log/squid/cache.log
2017/04/11 12:24:38 kid1| helperOpenServers: Starting 1/32 'squid-rdr'
processes
2017/04/11 12:24:38 kid1| assertion failed: ../src/base/Lock.h:30:
"count_ == 0"


> What was your previous version of Squid.
>  2.7?  3.5.23?  something else?

Sorry, I forgot exactly version, maybe squid-3.5.13/squid-3.5.14. Which
I remember at that time, I was hit by DDoS. So I change to latest version.

>> Here's my script:
>>
>> #!/usr/local/bin/perl
>>
>> $|=1;
>>
>> while (<>) {
>> chomp;
>> @X = split;
>> $url = $X[0];
>>
>>     if ( $url =~ m{ ^https?://(?:
>>         ad[0-9]?        |
>>         ad[szvx]?[0-9]? |
>>         ads[0-9]?       |
>>         adsbox          |
>>         adserver        |
>>         adservices?     |
>>         adserving       |
>>         adv             |
>>         advertising     |
>>         banners?        |
>>         adserver)\.(.*) }x ) {
>
> Does that regex actually work?
>
> 1) I've never known URLs to contain long series of whitespaces like that.
>
> 2) ad[szvx]?[0-9]? matches the same things as ad[0-9]? and ads[0-9]? and
> adv.

Yes, that script work in previous version of Squid.

>
>>             print "http://somedomain.com/null.png\n";
>>     }
>>
>>     else {
>>         print "$url\n";
>>     }
>> }
>>
>> Anyone guide me to make it work?
>>
>
> From the overall structure and output I am guessing that this is a Squid
> URL re-writer or Store-ID helper.
>
>
> IMO you should replace this simple helper with ACLs. Like so:
>
> squid.conf:
>  acl adverts_domain dtsdom_regex "/etc/squid/adverts_domain.regex"
>  deny_info 302:http://somedomain.com/null.png adverts_domain
>  http_access deny adverts_domain
>
> /etc/squid/adverts_domain.regex:
>    ad[szvx]?[0-9]?\.
>    adsbox\.
>    adserv(er|ing|ices?)\.
>    adv(ertising)?\.
>    banners?\.

OK, let me try first. Thank you Amos.


_______________________________________________
squid-users mailing list
[hidden email]
http://lists.squid-cache.org/listinfo/squid-users
Loading...