Zen Cart Logo
Forums / General Questions / Question regarding Spiders accessing my site

Question regarding Spiders accessing my site

Locked

Views: 757

Results 1 to 4 of 4
This thread is locked. New replies are disabled.
1 Jun 2009, 2:57 AM
#1
toypallet avatar

toypallet

New Zenner

Join Date:
Mar 2007
Posts:
21
Plugin Contributions:
0

Question regarding Spiders accessing my site

Hi everyone, I am not sure if I am posting in the right category, but hopefully someone can help me. I have a question regarding spider crawls.

I have been noticing when I click "who's online" I see a lot of "YSpider" logs. Is this normal? I am finding out that this is throwing off my products viewed and probably visitor hit counter as well. Is there any way to prevent this?

Under the Store tab in the "Sessions" option my "prevent spider sessions" is turned on. I also changed the session name from (ZenID) to (SESID) in includes/init_includes (I forget what specific file it was).

I did all the above and still see "YSpiders" accessing my page through my "Who's online" tab. What else do I need to do to prevent this, I don't want interior pages of my site indexed, just the "www.mysitename.com indexed (added to search engines) for example.

Thank you for anyones help, everyone is so polite and helpful here :)

Ben

1 Jun 2009, 3:29 AM
#2
afo avatar

afo

Totally Zenned

Join Date:
Aug 2004
Location:
New York City
Posts:
6,800
Plugin Contributions:
0

Re: Question regarding Spiders accessing my site

You don't want to stop them from crawling your site and products. It's what allows people to find your site then searching the net.

1 Jun 2009, 3:40 AM
#3
get_em_fast avatar

get_em_fast

Totally Zenned

Join Date:
Dec 2006
Location:
Seligman, MO U.S.A.
Posts:
2,072
Plugin Contributions:
1

Re: Question regarding Spiders accessing my site

afo's right, as afo always is. You might wanna take a look at THIS Google page on bots, as well. You might get a better understanding.

Hope this helps.

1 Jun 2009, 3:58 AM
#4
toypallet avatar

toypallet

New Zenner

Join Date:
Mar 2007
Posts:
21
Plugin Contributions:
0

Re: Question regarding Spiders accessing my site

Thanks for your responses. I wasn't sure if this was normal or not. I noticed alot of pages were being viewed, but figured a lot of those numbers were from spiders and not actual people viewing them.