Forums / General Questions / Site loads extremely slow - heavy bot traffic

Site loads extremely slow - heavy bot traffic

Views: 2,176

Results 1 to 5 of 5
15 Apr 2014, 3:15 AM
#1
bfox2718 avatar

bfox2718

New Zenner

Join Date:
May 2010
Posts:
1
Plugin Contributions:
0

Site loads extremely slow - heavy bot traffic

In the past few days, we have started to notice our site loads EXTREMELY Slow... pages parse in 2-3 minutes if they dont time out all together.

My hosting provider told me that my site is being attached by bots, and they want to refer me to the development team to get it protected.

I find it hard to believe that we cannot more simply protect ourselves. This is what my hosting company offered up. Any suggestions that are more clear?

After further research, we've identified that your site store.foxmar.com is encountering massive traffic. Just today there's 1,326,195 requests. Going through it, it seems like different bots are contributing to this. Particularly b a i d u. I would recommend setting up robots.txt to disallow certain bots from accessing specific pages on your site.

Site is http: / / s h o p . f o x m a r .com/FoxShoppe/

In particular, the Spring Portraits Section.

Thank you in advance!

15 Apr 2014, 1:06 PM
#2
stevesh avatar

stevesh

Black Belt

Join Date:
Feb 2005
Location:
Lansing, Michigan USA
Posts:
19,793
Plugin Contributions:
2

Re: Site loads extremely slow - heavy bot traffic

Loads OK here for me this morning.

15 Apr 2014, 1:46 PM
#3
rixstix avatar

rixstix

Totally Zenned

Join Date:
Aug 2009
Location:
North Idaho, USA
Posts:
2,015
Plugin Contributions:
0

Re: Site loads extremely slow - heavy bot traffic

We use THIS METHOD posted by DrByte to block bots and spiders that do not "Play Nice" and follow the rules.

Heed his words of Use at your own risk. We have several dozen being blocked for not playing nice.

Rick
RixStix (dot) com
aka: ChainWeavers (dot) com

15 Apr 2014, 2:06 PM
#4
rodg avatar

rodg

Deceased

Join Date:
Jan 2007
Location:
Australia
Posts:
6,263
Plugin Contributions:
4

Re: Site loads extremely slow - heavy bot traffic

RixStix:

We use THIS METHOD posted by DrByte to block bots and spiders that do not "Play Nice" and follow the rules.

Heed his words of Use at your own risk. We have several dozen being blocked for not playing nice.

If not already done, you should also add Yandex as one of those spiders that don't follow the rules. This one is notorious for hitting sites hard.

b a i d u is actually quite tame in comparison.

Cheers
RodG

NOTE: We are sorry that Rod is no longer with us.
We are grateful for all his contributions to the Zen Cart community.

Ozpost - The Ultimate Shipping module for Australian Merchants. Click these links for its Homepage, or Download from Zen-Cart.com.

16 Apr 2014, 11:42 PM
#5
picandnix avatar

picandnix

Totally Zenned

Join Date:
Dec 2010
Location:
UK
Posts:
1,780
Plugin Contributions:
2

Re: Site loads extremely slow - heavy bot traffic

RixStix:

We use THIS METHOD posted by DrByte to block bots and spiders that do not "Play Nice" and follow the rules.

Heed his words of Use at your own risk. We have several dozen being blocked for not playing nice.

Many a time I've had to search for and use the post linked above by Rixstix, so I've packaged it and it's now available here http://www.zen-cart.com/downloads.php?do=file&id=1854