Zen Cart Logo
Forums / General Questions / Easiest Way To Block Spambots Crawling?

Easiest Way To Block Spambots Crawling?

Views: 2,437

Results 1 to 8 of 8
6 Jan 2017, 5:01 PM
#1
csgodeimos avatar

csgodeimos

Zen Follower

Join Date:
May 2016
Posts:
191
Plugin Contributions:
0

Easiest Way To Block Spambots Crawling?

Hey guys,

Currently getting bombarded by Yannik, b a i d u and Poneytelecom.eu. I noticed an increased search for a particular manufacturer too. It seems to have been spammed. I tried blocking them through robots.txt, they ignore the robots.txt. I tried blocking them through .htaccess with ```
RewriteEngine On
RewriteCond %blahblahblah
RewriteRule .* - [F]

But they just come back with a new IP within seconds.

Any help would be appreciated,

Cheers,

Ash.
6 Jan 2017, 5:34 PM
#2
rixstix avatar

rixstix

Totally Zenned

Join Date:
Aug 2009
Location:
North Idaho, USA
Posts:
2,015
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?


https://www.zen-cart.com/downloads.php?do=file&id=1854

Zencart Block Bad Bots plugin.
We have nearly 100 variants of the file in use. Doesn't depend upon IP address.

6 Jan 2017, 5:53 PM
#3
csgodeimos avatar

csgodeimos

Zen Follower

Join Date:
May 2016
Posts:
191
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

RixStix:


https://www.zen-cart.com/downloads.php?do=file&id=1854

Zencart Block Bad Bots plugin.
We have nearly 100 variants of the file in use. Doesn't depend upon IP address.

I'll give this a try and see if it works, just added them in.
How is this different to doing it via the .htaccess file though?

6 Jan 2017, 6:04 PM
#4
csgodeimos avatar

csgodeimos

Zen Follower

Join Date:
May 2016
Posts:
191
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

RixStix:

x

This doesn't seem to be working for "rev.poneytelecom.eu"
Which seems to be the one spamming the sites search with random queries.

6 Jan 2017, 6:41 PM
#5
rixstix avatar

rixstix

Totally Zenned

Join Date:
Aug 2009
Location:
North Idaho, USA
Posts:
2,015
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

CSGODeimos:

How is this different to doing it via the .htaccess file though?

It blocks by matching a portion of the User-Agent string that you use/insert into the file.
We create an additional file for each User-Agent that we choose to block.
I never use the entire User-Agent, only a portion which I happen to consider unique to that User-Agent.

6 Jan 2017, 7:19 PM
#6
csgodeimos avatar

csgodeimos

Zen Follower

Join Date:
May 2016
Posts:
191
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

RixStix:

It blocks by matching a portion of the User-Agent string that you use/insert into the file.
We create an additional file for each User-Agent that we choose to block.
I never use the entire User-Agent, only a portion which I happen to consider unique to that User-Agent.

Okay, which would be Poneytelecom but that's still not working sadly. They're now spamming both websites with search queries.

6 Jan 2017, 7:38 PM
#7
rixstix avatar

rixstix

Totally Zenned

Join Date:
Aug 2009
Location:
North Idaho, USA
Posts:
2,015
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

Is that part of the User-Agent string or part of the URL?

Suggestions would be
Did you drop the file(s) in the proper zencart folder?
Did you properly edit the 'example' file and/or create you own copy of it?

You will still see them in the cpanel with a 410 error but not in your zencart whosonline.
You didn't say where you are seeing them.

Not a coder here, it would take DrByte (who wrote the code snippet) to say why it is not functioning in v154

6 Jan 2017, 7:53 PM
#8
csgodeimos avatar

csgodeimos

Zen Follower

Join Date:
May 2016
Posts:
191
Plugin Contributions:
0

Re: Easiest Way To Block Spambots Crawling?

RixStix:

Is that part of the User-Agent string or part of the URL?

Suggestions would be
Did you drop the file(s) in the proper zencart folder?
Did you properly edit the 'example' file and/or create you own copy of it?

You will still see them in the cpanel with a 410 error but not in your zencart whosonline.
You didn't say where you are seeing them.

Not a coder here, it would take DrByte (who wrote the code snippet) to say why it is not functioning in v154

I copied the example file and edited it. Managed to get it to work I think. Did another Google search and the url + ip came up with a forum where people were talking about something called ahref, I copied the example file, edited it appropriately and I haven't had any hits + queries since. I'm watching my Whosonline to see if they come back.