Yes :smile:Quote:
And does that mean I am out of luck for my ez-pages and html pages?
Why would you want to include your ez-pages and html pages anyway? Googlebot will index them.
Printable View
Yes :smile:Quote:
And does that mean I am out of luck for my ez-pages and html pages?
Why would you want to include your ez-pages and html pages anyway? Googlebot will index them.
I guess it is my lack of understanding how the sitemap and google works. I was thinking that some of these pages would be changed more often and I would like to update to them as soon as possible.
So if one is regularly indexed, what is the real purpose to send sitemap of anything to google?:huh:
Hello! I am having trouble with this module. I have my zencart in the folder of /catalog/, and have my homepage redirected to /catalog/. The site is www.bridgecampus.com
Google web crawl gives the error of 'robots.txt unreachable'. When I check the robot.txt analysis, it shows following:
This site is not located at the top level for the domain. A robots.txt file is only valid when located in the highest-level directory and applies to all directories within the domain. The robots.txt file that applies to your site (if one exists) is located at http://bridgecampus.com/robots.txt.
I search the forum, but no one reported the same problem before. But I do think there should be many of us having zencart installed in a folder rather than the root. So I am really confused. What should I look for?
Anyone can help! Thanks so much!
Hi Stargre,
At the moment your shopping cart sits here: http://bridgecampus.com/catalog/, and from what I am reading from you below either you don't have a robots.txt file, or it is located somewhere within the /catalog/ folder, instead of your main directory.
Your robots.txt file should sit in the top level folder, which is one up from /catalog/
Does this make sense?
As the error (and quantum) says, you need your robots.txt file at the top level of your site - ie. above the /catalog folder.
Well, someone has reported the error message now ... :smile:
Hi! Thanks for your response. Actually, I don't quite understand. According to your explaination, it seems that there should be a robots.txt file which I can move to the web root and it will solve this problem. However, I could not find a robot.txt file by searching my web folder. Is it dynamically produced by php files?Quote:
Originally Posted by quantum
I used to think that Zencart requires the robots.txt to sit in my zencart folder, in my case, /catalog/ folder. And that's what this module is built for. Am I wrong? What I did is just uploaded the contribution to my catalog folder and that's it.
You can tell me what I should do in this case? Thanks!
Well, yeah, I guess I know I should have it in the top level. But the question is how. I was thinking of just find this file and move it to the web root. But I cannot even find it.Quote:
Originally Posted by Pixxi
hi,
I don't think that ZenCart comes with a robotos.txt file as standard, you need to create one (someone correct me if I am wrong on this)
All you really need to do to create one is past the following into a notepad file, and save it to your main directory as robots.txt. The following between the ~~~~ is a sample:
~~~~
User-agent: *
Disallow: catalog/images/
Disallow: catalog/docs/
Disallow: catalog/download/
Disallow: catalog/admin/
~~~~
This file helps control whether you block or let search engine robots index your site. In this example it is telling all search engine bots that they can index anything within the site except those 4 directories.
The robots.txt file is not a necessity for your site, but it will help with MSN search as they seem to better index websites that have it.
To learn more about what it does, visit: http://www.robotstxt.org/wc/robots.html
I can't find this contribution. Where can I download it? I saw the attachment to this thread but this can't be the whole deal? All links that I found in other threads are not working.
So, you mean just create such robot.txt file and put it in my root directory. And don't touch a thing in either my original store files or the googlesitemapadmin module? Thanks!
Quote:
Originally Posted by Pixxi
Quote:
Originally Posted by quantum