Googlebot ALWAYS on my site

[makenoiz]

Evertime I check whos online I see

1.3.9h



User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Sometimes there are 3 or 4 instances of the bot on my site. Right now there are two:

Host: crawl-66-249-73-118.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Host: crawl-66-249-71-24.googlebot.com
User Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

In google Webmaster tools I have set crawl rate to the minimum but the bot is still always always always on the site. I need to slow down the bot somehow.

Any suggestions?
Thank you

[joejoejoe]

That's a good thing, don't know why you would want to stop it. But if you want to tell google to back off then create a google webmaster's account, claim ownership of your store, and you can set the crawl rate that way.

EDIT: Well if I wasn't stuffing my face full of bbq ribs I would have read the part where you already have a webmaster account.... If you've set the crawl rate there then google usually follows it pretty closely it just might take some time for it to catch up.

[makenoiz]

HI yes, I mentioned in my post that I have set the crawl rate to minimum in Webmaster tools. The problem is, I see spikes in CPU with several instances of the bot on my site. I like googlebot but doesnt it seem odd that its always there.... maybe thats normal?? Do others see several instances of the bot on thier site at any give moment?

Thank you

[joejoejoe]

Yes I've had three or more at a time. One is just for images, the others just following internal links. It's just as server intensive as a guest clicking through your site, so there's nothing really to worry about.