Google SitemapAdmin

[quantum]

Are you able to place a screen print up of the page in your Google Sitemaps that shows the error Google is giving you?

[stargre]

Are you able to place a screen print up of the page in your Google Sitemaps that shows the error Google is giving you?

Sure! Since I didn't have the store under root directory, I submitted both www.bridgecampus.com and www.bridgecampus.com/catalog/ Both screen prints are attached. Thanks for your help!

[rod44]

Sure! Since I didn't have the store under root directory, I submitted both www.bridgecampus.com and www.bridgecampus.com/catalog/ Both screen prints are attached. Thanks for your help!

Well, but looking at what you have posted, it appears that the last 'validation' was about 10 days ago, and correct me if I am wrong, you have changed the robots.txt file since. If you look at the last 2 pages dated the 18th it shows success. Seems that if they WOULD crawl your site again that it would probably work.

[stargre]

Well, but looking at what you have posted, it appears that the last 'validation' was about 10 days ago, and correct me if I am wrong, you have changed the robots.txt file since. If you look at the last 2 pages dated the 18th it shows success. Seems that if they WOULD crawl your site again that it would probably work.

That's true and that's also what I am confused about. It seems that they have seen the robots.txt change, but that didn't fix the crawl 'robots.txt unreachable' problem. So, modifying robots.txt might not be the correct resolution?

[rod44]

That's true and that's also what I am confused about. It seems that they have seen the robots.txt change, but that didn't fix the crawl 'robots.txt unreachable' problem. So, modifying robots.txt might not be the correct resolution?

No, this is why you need to use the Google SiteMaps and relax. The Googlebot crawls through robots.txt SLOW, and you just have to wait for your 'number to be called' and once the recrawl is completed the page validations should change. This is one of those things you will have to wait a bit to see if it worked. But just so you know, my robots.txt looks like:

# robots.txt for http://www.hatnpatch.com/

User-agent: *
Disallow: /cgi-bin/ # these will soon disappear
Disallow: /logs/ # these will soon disappear

and they are crawling my .html pages off of that. Patience my friend,

[rod44]

Maybe you should place a 'specific' allow and see if that will help. Try adding this:

User-Agent: Googlebot
Allow: /

Also, you still have not said if you have Google SiteMaps installed. Do you? If not, I would add it as it is the BEST way to get the attention of Google and get them to your site. Using the site map, as soon as you update your site you can just update the sitemap with the click of a button in your admin, then ping their server, and it's done. Might be better than getting too caughtup with the robots.txt? I am telling you, the sitemap mod is the best way to go.

One other thing, I use Firefox as a browser and your site comes up with the center page off to the right overlapping all the right boxes.

Hope that helps!

[stargre]

Maybe you should place a 'specific' allow and see if that will help. Try adding this:

User-Agent: Googlebot
Allow: /

Also, you still have not said if you have Google SiteMaps installed. Do you? If not, I would add it as it is the BEST way to get the attention of Google and get them to your site. Using the site map, as soon as you update your site you can just update the sitemap with the click of a button in your admin, then ping their server, and it's done. Might be better than getting too caughtup with the robots.txt? I am telling you, the sitemap mod is the best way to go.

One other thing, I use Firefox as a browser and your site comes up with the center page off to the right overlapping all the right boxes.

Hope that helps!

Sorry that I missed your question. But yeah, I do have googlesitemapadmin contribution installed. That's what you are talking about, right? I used to think that the google sitemapadmin was built to utilize google robots that sort of things. It seems I was wrong. They are totally different two routes of getting google notified? I don't see google sitemapadmin produce a lot of traffic for my site though.

[quantum]

I also just noticed in your last two screen prints that it shows that the robots.txt file was succeffully downloaded on November 18th.

[rod44]

Sorry that I missed your question. But yeah, I do have googlesitemapadmin contribution installed. That's what you are talking about, right? I used to think that the google sitemapadmin was built to utilize google robots that sort of things. It seems I was wrong. They are totally different two routes of getting google notified? I don't see google sitemapadmin produce a lot of traffic for my site though.

Robots.txt and sitemaps are independent of eachother, and google actually PREFERS the SiteMap route as it doesn't require them to cralw links. What you will see if you use the User Tracking Mod (if you don't have it, this is an INCREDIBLE TOOL to see how visitors use your site) is 'hits' from BOTS and GoogleBot will be identified. Here are a couple of questions (and answers) to make sure you have everything in place:

1. Are you signed up WITH Google under sitemaps? If so, have you added their 'validation' page?

2. Have you created the sitemap through your admin and verified it is actually in your /catalog/ directory? You need to VIEW it through a browser or download and read in notepad or something to make sure it is right.

3. In the sitemap admin once you have verified the sitemap is there, use the ping button to let them know you are there.

If I sound like I am being elementary, I am not. I just want to see this start working for you so you can see some success with your site.

Good Luck!

[stargre]

Robots.txt and sitemaps are independent of eachother, and google actually PREFERS the SiteMap route as it doesn't require them to cralw links. What you will see if you use the User Tracking Mod (if you don't have it, this is an INCREDIBLE TOOL to see how visitors use your site) is 'hits' from BOTS and GoogleBot will be identified. Here are a couple of questions (and answers) to make sure you have everything in place:

1. Are you signed up WITH Google under sitemaps? If so, have you added their 'validation' page?

2. Have you created the sitemap through your admin and verified it is actually in your /catalog/ directory? You need to VIEW it through a browser or download and read in notepad or something to make sure it is right.

3. In the sitemap admin once you have verified the sitemap is there, use the ping button to let them know you are there.

If I sound like I am being elementary, I am not. I just want to see this start working for you so you can see some success with your site.

Good Luck!

1) Yes, I did signup and verified the site.
2) That's right. All three .xml files are in /catalog/ directory. They can be seen in web browser.
3) I did pinged them several times.

This seems so strange. I wish myself some good luck as well. But, any new ideas? Thanks!