robots.txt

[IDW]

I tryed searching for my site using Msn and it did not show up. google picks it up fine.
Can anyone explain to me what a [FONT=Courier New]robots.txt file is.[/FONT]
[FONT=Courier New][/FONT]
[FONT=Courier New]There site also said the HTTP compression and HTTP conditional GET are not enabled.[/FONT]
[FONT=Courier New][/FONT]
[FONT=Courier New][/FONT]

[IDW]

Ok I know what a robots.txt file is and where it goes , but is it nessasay. Is there not code in the Zen cart files that tell what robots what to do.
Or does MSN robots what to see a txt file.

I do not understand the HTTP compression and HTTP conditional GET

[DrByte]

Ok I know what a robots.txt file is and where it goes , but is it nessasay. Is there not code in the Zen cart files that tell what robots what to do.
Or does MSN robots what to see a txt file.

Zen Cart doesn't need or use a robots.txt file.
See the various threads in the "Similar Threads" section at the bottom of this page for more related discussion on the topic.

I do not understand the HTTP compression and HTTP conditional GET

Probably best to ask *them* what they mean by that, since they're the ones pointing out the so-called problem.

[Tech-E]

The robots.txt file is also called the robots exclusion file. Basically, you use it to tell spiders which pages and directories you do NOT want to see indexed. The exception to this is an optional directive that can be used to tell spiders where to find an XML sitemap.

A very basic robots.txt file would be:

 
User-agent: *
Disallow: /images/

This tells all spiders (* = wild card) to exclude the images directory from their index.

You can learn more about it at the following sites:

http://www.tech-evangelist.com/2008/10/18/robotstxt/

http://www.robotstxt.org/

The robots.txt file must be placed in the root directory.

There site also said the HTTP compression and HTTP conditional GET are not enabled.

Whose site said these were not enabled? There is no need to use HTTP compression or HTTP conditional GET. MSN does have problems indexing sites using compression techniques.

I would not worry too much about MSN. It typically drives less than 5% of all web traffic. Google and Yahoo are much more important.

[IDW]

Thanks guys. Thanks for the help.
I just thought it was odd that msn does not show it so I went to there webmasters site and registered the site and ran their
MSN Webmaster Center

HTTP compression and HTTP conditional GET test tool .

It said they were not enabled. I have no idea what that means.
You are probally right about MSN not mattering much

[Tech-E]

MSN = Microsoft. That should explain why the messages you see are confusing.