Zen Cart Logo
Forums / General Questions / Spider or not?

Spider or not?

Locked

Views: 1,783

Results 1 to 5 of 5
This thread is locked. New replies are disabled.
22 Feb 2009, 12:03 AM
#1
stride_r avatar

stride_r

Zen Follower

Join Date:
Aug 2008
Posts:
115
Plugin Contributions:
0

Spider or not?

Below are two recent entries in my Who's Online. The first declares as a guest and is being assigned a Session ID , while the latter is clearly a spider. However, the former's host name seems to imply (to me anyway) as though it is also a spider.

Anyone able to confirm my interpretation for me?
If not, then why would their "Host" contain "bot" and "search.msn"?

¥Guest 65.55.107.233
Session ID: op2dpnrkvt7efdfgffgdebnrrr8i51
Host: msnbot-65-55-107-233.search.msn.com
User Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

¥Spider 65.55.231.120
Session ID:
Host: 65.55.231.120
User Agent: msnbot-media/1.1(+http://search.msn.com/msnbot.htm)

22 Feb 2009, 4:29 AM
#2
mprough avatar

mprough

Totally Zenned

Join Date:
Nov 2007
Location:
Woodbine, Georgia, United States
Posts:
4,290
Plugin Contributions:
39

Re: Spider or not?

Those are genuinely MSN

Check them out http://network-tools.com/

~Melanie

22 Feb 2009, 1:41 PM
#3
stride_r avatar

stride_r

Zen Follower

Join Date:
Aug 2008
Posts:
115
Plugin Contributions:
0

Re: Spider or not?

mprough:

Those are genuinely MSN

Ummm...okay - MSN what? MSN Browser ...or MSN spider?

I guess I'm also not comfortably sure I'm reading the results of Network Tools lookup correctly. It clearly tells me that it's from Microsoft Corp network ....but how do I know whether it wasn't simply an employee visitng my site?

22 Feb 2009, 8:13 PM
#5
stride_r avatar

stride_r

Zen Follower

Join Date:
Aug 2008
Posts:
115
Plugin Contributions:
0

Re: Spider or not?

mprough:

msnbot = spider

Okay, thanks!
Now the question is: If the visitor question in IS a bot - then it should be recognized as such and not be assigned a Session ID.

However, my "spiders.txt" already contains an entry "msnbot" and this entity was not recognized. It appears that Zencart examines the User Agent data and not the Host data for its determination.

Since this entity is supposedly a Bot but its User Agent resembles a browser, then how does one get it to be recognized as a bot ??? There's nothing in the user agent line to designate it as a one.

Host: msnbot-65-55-107-233.search.msn.com
User Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

Any thoughts???