Hi,
yes we did had to add .htaccess because we were pestered with hundreds of hits from nonbehaving robots:
Code:
ExpiresActive On
# Format: A where n is a number of seconds. Expires cache n seconds after access
# A2629740
ExpiresByType application/javascript A2629740
ExpiresByType application/x-javascript A2629740
ExpiresByType text/javascript A604800
ExpiresByType text/css A604800
ExpiresByType image/gif A900
ExpiresByType image/x-icon A900
ExpiresByType image/icon A900
ExpiresByType image/jpg A900
ExpiresByType image/jpeg A900
ExpiresByType image/png A900
SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
Deny from 110.0.0.0/8
Deny from 111.0.0.0/8
Deny from 112.0.0.0/5
Deny from 120.0.0.0/6
Deny from 124.0.0.0/8
Deny from 125.0.0.0/8
Deny from 147.0.0.0/8
Deny from 46.4.62.121
Deny from 46.229.168.139
Deny from 88.99.250.124
Deny from 46.229.168.163
Deny from 46.229.168.149
Deny from 144.76.68.88
Deny from 88.99.250.124
Deny from 46.229.168.0/8
Deny from 46.4.62.0/8
Deny from 46.4.62.0/8
Deny from 144.76.68.0/8
Deny from 46.229.168.0/8
Deny from 144.76.68.0/8
Deny from 46.229.168.0/8
Deny from 46.229.168.154
Deny from 46.229.168.0/8
Deny from 46.4.62.0/8
Deny from 46.4.62.121
Deny from 46.229.168.144
Deny from 46.229.168.145
Deny from 144.76.68.88
</Limit>
Bookmarks