Forums / General Questions / HIGH CPU Usage with FB's Bots

HIGH CPU Usage with FB's Bots

Views: 1,246

Results 1 to 3 of 3
18 Oct 2012, 9:35 AM
#1
lilly06 avatar

lilly06

New Zenner

Join Date:
Dec 2010
Location:
monaco
Posts:
40
Plugin Contributions:
0

HIGH CPU Usage with FB's Bots

Hello

I have huge issue since a few weeks now, i hav emy store down with error 500 like 5 times a days.. i looked at my log error, and i see each time it happens when FB's bot is scrowling the site... i got in touch with my host, and they slowed down the scrolling with 30 second per request in my HTACCESS.

But again i got this issue... here is the error i got:
[client 69.171.237.5] SoftException in Application.cpp:574: Could not execute script "/home/XXXX/public_html/store/index.php"
[Thu Oct 18 04:16:39 2012] [error] [client 69.171.237.5] File does not exist: /home/XXXX/public_html/500.shtml

Also i noticed i hav ethis line in this folder:
/store/includes/templates/MYTEMPLATE/common/html_header.php

<head prefix="og: <http://ogp.me/ns#> fb: <http://ogp.me/ns/fb#> MY STORE: http://ogp.me/ns/fb/MYSTORE#">

I don't know what to do to solve this issue :( i spent many hours ecven upgraded the store, cuz i thought i had a malware, so i did a filecompare etc.. and again this issue, till i figure out it's from FB's BOt :(

any help is much appreciated!

19 Oct 2012, 3:41 AM
#2
ajeh avatar

ajeh

Oba-san

Join Date:
Sep 2003
Location:
Ohio
Posts:
62,757
Plugin Contributions:
1

Re: HIGH CPU Usage with FB's Bots

You could look at adding Crawl-Delay to your robots.txt file for Faceboot bots, something like:

User-agent: facebookexternalhit/1.1 
Crawl-delay: 5

User-agent: facebookexternalhit/1.0 
Crawl-delay: 5

User-agent: facebookexternalhit/* 
Crawl-delay: 5

Linda McGrath
If you have to think ... you haven't been zenned ...

**Did YOU buy the Zen Cart Team a cup of coffee and a donut today? Just click here to support the Zen Cart Team!!**

Are you using the latest? Perhaps you've a problem that's fixed in the latest version: [Upgrade today!]
Officially PayPal-Certified! Just click here

Try our Zen Cart Recommended Services - Hosting, Payment and more ...
Signup for our Announcements Forums to stay up to date on important changes and updates!

23 Oct 2012, 5:44 PM
#3
lilly06 avatar

lilly06

New Zenner

Join Date:
Dec 2010
Location:
monaco
Posts:
40
Plugin Contributions:
0

Re: HIGH CPU Usage with FB's Bots

thank you!! going to try this or i will have to block Fb's bots :(