Forums / General Questions / Do i need Robots.txt ? yes or not?

Do i need Robots.txt ? yes or not?

Locked

Results 1 to 4 of 4

This thread is locked. New replies are disabled.

17 Jun 2009, 1:05 PM

jeffhardy

New Zenner

Join Date:: Aug 2008
Posts:: 72
Plugin Contributions:: 0

Do i need Robots.txt ? yes or not?

17 Jun 2009, 1:33 PM

superprg

Totally Zenned

Join Date:: Feb 2006
Location:: Chicago
Posts:: 1,349
Plugin Contributions:: 0

Re: Do i need Robots.txt ? yes or not?

No
unless you want the web engine to spare some pages/folders from appearing in the search results

17 Jun 2009, 2:10 PM

Kim

Obaa-san

Join Date:: Jun 2003
Location:: West Coast, North America
Posts:: 26,611
Plugin Contributions:: 0

Re: Do i need Robots.txt ? yes or not?

I would disagree - You probably want to have a one, just to prevent logging server errors when the spiders can't locate it. The robots.txt can be blank.

17 Jun 2009, 8:20 PM

jeffhardy

New Zenner

Join Date:: Aug 2008
Posts:: 72
Plugin Contributions:: 0

Re: Do i need Robots.txt ? yes or not?

can you please tell me which folders and files, i block search engine to dont index it?