search. in the spider list

[joejoejoe]

Based on some posts in this thread:
http://www.zen-cart.com/forum/showth...=163007&page=5

I downloaded the new spiders.txt and made sure that search. wasn't in there.

So now the msnbot: search.msn.com is checking in as guest instead of indexer. That will mean indexed results with session ids in the url. (At least the # of msn searchers compared to google is very low.)

What am I supposed to do to ensure that msn can spider as a spider, but that some of my customers don't get the cookie/login errors?

1.3.9h, linux server. no addons anymore.

[DrByte]

What is the EXACT user-agent you're seeing on these msn spiders?

MSN traditionally sets its user-agent to "msnbot".
Zen Cart filters spiders based on user-agent. NOT ON IP ADDRESS OR HOST NAME like 22.33.44.55 or search.msn.com