HIGH CPU Usage with FB's Bots

**lilly06** · 18 Oct 2012, 10:35 AM

Hello

I have huge issue since a few weeks now, i hav emy store down with error 500 like 5 times a days.. i looked at my log error, and i see each time it happens when FB's bot is scrowling the site... i got in touch with my host, and they slowed down the scrolling with 30 second per request in my HTACCESS.

But again i got this issue... here is the error i got:
[client 69.171.237.5] SoftException in Application.cpp:574: Could not execute script "/home/XXXX/public_html/store/index.php"
[Thu Oct 18 04:16:39 2012] [error] [client 69.171.237.5] File does not exist: /home/XXXX/public_html/500.shtml

Also i noticed i hav ethis line in this folder:
/store/includes/templates/MYTEMPLATE/common/html_header.php
<head prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb# MY STORE: http://ogp.me/ns/fb/MYSTORE#">

I don't know what to do to solve this issue

i spent many hours ecven upgraded the store, cuz i thought i had a malware, so i did a filecompare etc.. and again this issue, till i figure out it's from FB's BOt

any help is much appreciated!

**Ajeh** · 19 Oct 2012, 04:41 AM

You could look at adding Crawl-Delay to your robots.txt file for Faceboot bots, something like:

Code:

User-agent: facebookexternalhit/1.1 
Crawl-delay: 5

User-agent: facebookexternalhit/1.0 
Crawl-delay: 5

User-agent: facebookexternalhit/* 
Crawl-delay: 5

**lilly06** · 23 Oct 2012, 06:44 PM

thank you!! going to try this or i will have to block Fb's bots

Thread: HIGH CPU Usage with FB's Bots

Thread Tools

Search Thread

Display

HIGH CPU Usage with FB's Bots

Re: HIGH CPU Usage with FB's Bots

Re: HIGH CPU Usage with FB's Bots

Similar Threads

v151 Surpass Hosting tells me our site's CPU usage is too high

High cpu usage and error

How to lower MySQL CPU usage on GoDaddy with v1.3.8?

Zencart and High CPU Usage?

Posting Permissions