blocking spiders

[RixStix]

See post #13 by DrByte
in this thread http://www.zen-cart.com/showthread.p...283#post817283

We've used it to block quite a few spiders that we dont want to see. Some have gotten really aggressive and don't obey the rules.

We also tried the IP block within Cpanel but all that does is give a 302 redirect.

[stitchnkitty]

I am assuming this is what you are referring: Is it best to use this as is or should I modify it for the host:yandex.ru ???? and can someone specify the code modification?

Also how has this worked for you? I understand this may effect the rankings?



Create a new file: /includes/extra_configures/block_yandex_spider.php ... containing ONLY the following code:
Code:

<?php
if (stristr(strtolower($_SERVER['HTTP_USER_AGENT']), 'yandex')) {
@header("HTTP/1.1 410 Gone");
exit();
}

[RixStix]

No modifications are needed.

You asked about blocking the yandex spider. DrByte's code and instructions are specifically for blocking the Yandex spider. We have created additional files where we replaced the text yandex with the names of other unwanted spiders who show signs of not behaving well.

We haven't noticed any change in ranking.