Zen Cart Logo
Forums / Search Engines / Google Crawl Errors

Google Crawl Errors

Views: 116

Results 1 to 12 of 12
27 Feb 2014, 23:18
#1
traytray avatar

traytray

Totally Zenned

Join Date:
May 2012
Posts:
566
Plugin Contributions:
0

Google Crawl Errors

I just checked my crawl errors on Google Webmaster tools and I have 96 pages with 404 errors. The pages with errors are listed like this:

http://www.designerperfumesnob.com/index.php?main_page=product_info&products_id=2201

How can I find out what this page is, so I can fix it, and I have 95 others and the only difference seems to be the number at the end.

I checked the Multiple Categories Link Manager hoping I might see one of those ID numbers, but I haven't been able to find any that match.

Thank you for your time.

28 Feb 2014, 03:03
#2
dbltoe avatar

dbltoe

Totally Zenned

Join Date:
Jan 2004
Location:
N of San Antonio TX
Posts:
9,760
Plugin Contributions:
9

Re: Google Crawl Errors

The file you need to work on is the one that requested the link(s). Since they are all 404 errors (not found), they well.... won't be found.
You need to find out what was sending the browser to the bad address. A link from an external site? Social Media? Perhaps a faulty sitemap?
The needs to be at the file erroneously giving a bad link. If you have cpanel with your host, go to the Logs area and find Awstats. The last item is Pages not found. This area should help you by letting you know the Referrer(s).

28 Feb 2014, 04:17
#3
gilby avatar

gilby

Totally Zenned

Join Date:
Aug 2005
Location:
Vic, Oz
Posts:
1,816
Plugin Contributions:
0

Re: Google Crawl Errors

traytray:

I just checked my crawl errors on Google Webmaster tools and I have 96 pages with 404 errors. The pages with errors are listed like this:

http://www.designerperfumesnob.com/index.php?main_page=product_info&products_id=2201

How can I find out what this page is, so I can fix it, and I have 95 others and the only difference seems to be the number at the end.

I checked the Multiple Categories Link Manager hoping I might see one of those ID numbers, but I haven't been able to find any that match.

Thank you for your time.This seems to be a conventional zencart link
Whereas your site is now using some url rewriter.

These might be leftovers from before you installed the url rewriter!
Is there a product with ID 2201?
If so is it disabled?

28 Feb 2014, 05:33
#4
traytray avatar

traytray

Totally Zenned

Join Date:
May 2012
Posts:
566
Plugin Contributions:
0

Re: Google Crawl Errors

gilby:

This seems to be a conventional zencart link
Whereas your site is now using some url rewriter.

These might be leftovers from before you installed the url rewriter!
Is there a product with ID 2201?
If so is it disabled?

I have looked in the Multiple Categories Link Manager and I cannot find a category with an ID of 2201.

Here is another example from error details:

http://www.designerperfumesnob.com/index.php?main_page=product_info&products_id=5351

Now I clicked on Linked From and I see this information:

http://www.designerperfumesnob.com/index.php?main_page=index&manufacturers_id=109

http://www.designerperfumesnob.com/dana

I don't see a common thread between these 4 URLs.

http://www.designerperfumesnob.com/?main_page=index&cPath=&manufacturers_id=128&sort=20a&alpha_filter_id=89

http://www.designerperfumesnob.com/

I am also looking at Cpanel and I really don't understand the information listed under REQUIRED BUT NOT FOUND URLS (http CODE 404)

It says URL (189) and I guess the list that follows has 189 entries. Then to the right of this list is the heading Error Hits and under that are numbers and then to the right of that are Referrers. Many of the referrers are just blank. Of the ones that are listed, if I enter them in a browser, the product page comes up.

Under the URL list are a lot of images, but if I enter the image number in my store, the page displays.

I'm not sure how prudent it is for me to post this information because it lists a lot of named files and their location, so I didn't do a screenshot.

I did have someone install CEON, but I hadn't checked the errors prior to that installation.

I hope this additional information can help.

28 Feb 2014, 05:54
#5
gilby avatar

gilby

Totally Zenned

Join Date:
Aug 2005
Location:
Vic, Oz
Posts:
1,816
Plugin Contributions:
0

Re: Google Crawl Errors

gilby:

Is there a product with ID 2201?
If so is it disabled?

traytray:

I have looked in the Multiple Categories Link Manager and I cannot find a category with an ID of 2201.
Is there a product (NOT a category) with ID 2201?
If so is it disabled?

Find it by going to admin->catalog->Categories/Products
and enter 2201 into the search box

28 Feb 2014, 06:32
#6
traytray avatar

traytray

Totally Zenned

Join Date:
May 2012
Posts:
566
Plugin Contributions:
0

Re: Google Crawl Errors

gilby:

Is there a product (NOT a category) with ID 2201?
If so is it disabled?

Find it by going to admin->catalog->Categories/Products
and enter 2201 into the search box

The product IDs do not seem to be related to a product. I entered some of these product ID numbers in the search box and I receive strange results. For example:

http://www.designerperfumesnob.com/index.php?main_page=product_info&products_id=5351 is listed as a 404 page. When I enter it in the search box, nothing comes up.

For this error, I see four Linked From URLs:

http://www.designerperfumesnob.com/index.php?main_page=index&manufacturers_id=109 (a search on this ID brings up this product: http://www.designerperfumesnob.com/womens-perfume/Gucci/Flora-By-Gucci/edp/2.5oz

http://www.designerperfumesnob.com/dana (this is like a landing page which shows the two products I carry from the manufacturer Dana)

http://www.designerperfumesnob.com/?main_page=index&cPath=&manufacturers_id=128&sort=20a&alpha_filter_id=89 (a search of this ID brings up 39 different products. I have no idea what the common thread would be) If I paste this URL into a browser, it brings up the manufacturer "Parfums Gres" and there are no products listed. However, the URL http://www.designerperfumesnob.com/Parfums-Gres DOES list the products I carry from that manufacturer.

http://www.designerperfumesnob.com/ (this is the 4th Linked From and it is just my main page.

28 Feb 2014, 06:39
#7
gilby avatar

gilby

Totally Zenned

Join Date:
Aug 2005
Location:
Vic, Oz
Posts:
1,816
Plugin Contributions:
0

Re: Google Crawl Errors

I don't use a url re-writer...

All the links shown are standard zen-cart links
When you installed the re-writer you should have redirected all the old links to the new re-written links using 301 redirects.

It seems that these were NOT redirected and are just hanging around until google drops them some time in the future.

28 Feb 2014, 06:48
#8
traytray avatar

traytray

Totally Zenned

Join Date:
May 2012
Posts:
566
Plugin Contributions:
0

Re: Google Crawl Errors

gilby:

I don't use a url re-writer...

All the links shown are standard zen-cart links
When you installed the re-writer you should have redirected all the old links to the new re-written links using 301 redirects.

It seems that these were NOT redirected and are just hanging around until google drops them some time in the future.

does this tell you anything:

Code:
RewriteEngine On

Redirect all users to access the site WITH the 'www.' prefix

RewriteCond %{HTTP_HOST} !^www. [NC]
RewriteCond %{HTTP_HOST} !.([a-z-]+.[a-z]{2,6})$ [NC]
RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

BEGIN CEON URI MAPPING REWRITE RULE

Don't rewrite any URIs ending with a file extension (ending with .[xxxxx])

RewriteCond %{REQUEST_URI} !.[a-z]{2,5}$ [NC]

Don't rewrite any URIs for some, popular specific file format extensions,

which are not covered by main file extension condition above

RewriteCond %{REQUEST_URI} !.(mp3|mp4|h264)$ [NC]

Don't rewrite any URIs for some specific file format extensions,

which are not covered by main file extension condition above

Uncomment the following line to apply this condition! (Remove the # at the start of the next line)

#RewriteCond %{REQUEST_URI} !.(3gp|3g2|h261|h263|mj2|mjp2|mp4v|mpg4|m1v|m2v|m4u|f4v|m4v|3dml)$ [NC]

Don't rewrite admin directory

RewriteCond %{REQUEST_URI} !^/admin [NC]

Don't rewrite editors directory

RewriteCond %{REQUEST_URI} !^/editors/ [NC]

Don't rewrite cgi-bin directory

RewriteCond %{REQUEST_URI} !^/cgi-bin/ [NC]

Handle all other URIs using Zen Cart (its index.php)

RewriteRule .* /index.php [QSA,L]

END CEON URI MAPPING REWRITE RULE

28 Feb 2014, 06:51
#9
gilby avatar

gilby

Totally Zenned

Join Date:
Aug 2005
Location:
Vic, Oz
Posts:
1,816
Plugin Contributions:
0

Re: Google Crawl Errors

traytray:

does this tell you anything:Not me...
As I don't use re-writer.

Maybe someone who does use one could comment here now

28 Feb 2014, 15:02
#10
lhungil avatar

lhungil

Totally Zenned

Join Date:
Feb 2012
Location:
mostly harmless
Posts:
1,818
Plugin Contributions:
4

Re: Google Crawl Errors

traytray:

The product IDs do not seem to be related to a product. ...
Based upon your "linked from", I'd guess you had a product (probably on sale) which was deleted or disabled (using the Zen Cart administrative interface or possibly because the store ran out of stock).

As gilby noted, one can check if there is a product with a specific ID by logging into the Zen Cart administrative URL and going to "catalog->Categories/Products". On the Categories/Products page you can type 2201 in the "search for" box on that page. Do not try searching from public facing "store" pages, search using the Zen Cart administrative interface.

If you do not have a product with ID 2201 (or product 2201 is disabled), a HTTP 404/Not Found will be shown for the product page. It does not matter if you are using the "stock" Zen Cart URLs or alternative URLs. In most cases these show up when: a product was deleted or disabled, a bad link exists in your sitemap, a bad link exists on social media, a bad link exists on a third party site, etc. Sometimes this also happens because someone was "browsing" your store and typing in "random" product IDs (or used an old bookmark to a product which was deleted / disabled).

If you do have a product with ID 2201 (and it is not disabled)... If my memory serves me right (I have not used that specific module for well over two years), CEON URI can interpret both stock and custom URIs (and will auto 301 from the stock to custom URLs)... So I would be surprised... And would say you would be best served posting some more specifics about what you are seeing in the CEON URI support thread...

Note 1: If the product ID does exist and is not disabled, make sure the copy of CEON URI is up to date... If it is, worst case, one could use a known workaround to add historic URLs w/o hand modifying the database... Just edit the product changing the URL first to the stock Zen Cart URL and then back to the custom URL...

**Note 2:**Your template is causing some 403 / Access Denied messages... May want to look into those as well...

01 Mar 2014, 04:36
#11
traytray avatar

traytray

Totally Zenned

Join Date:
May 2012
Posts:
566
Plugin Contributions:
0

Re: Google Crawl Errors

lhungil:

Based upon your "linked from", I'd guess you had a product (probably on sale) which was deleted or disabled (using the Zen Cart administrative interface or possibly because the store ran out of stock).

As gilby noted, one can check if there is a product with a specific ID by logging into the Zen Cart administrative URL and going to "catalog->Categories/Products". On the Categories/Products page you can type 2201 in the "search for" box on that page. Do not try searching from public facing "store" pages, search using the Zen Cart administrative interface.

If you do not have a product with ID 2201 (or product 2201 is disabled), a HTTP 404/Not Found will be shown for the product page. It does not matter if you are using the "stock" Zen Cart URLs or alternative URLs. In most cases these show up when: a product was deleted or disabled, a bad link exists in your sitemap, a bad link exists on social media, a bad link exists on a third party site, etc. Sometimes this also happens because someone was "browsing" your store and typing in "random" product IDs (or used an old bookmark to a product which was deleted / disabled).

If you do have a product with ID 2201 (and it is not disabled)... If my memory serves me right (I have not used that specific module for well over two years), CEON URI can interpret both stock and custom URIs (and will auto 301 from the stock to custom URLs)... So I would be surprised... And would say you would be best served posting some more specifics about what you are seeing in the CEON URI support thread...

Note 1: If the product ID does exist and is not disabled, make sure the copy of CEON URI is up to date... If it is, worst case, one could use a known workaround to add historic URLs w/o hand modifying the database... Just edit the product changing the URL first to the stock Zen Cart URL and then back to the custom URL...

**Note 2:**Your template is causing some 403 / Access Denied messages... May want to look into those as well...

Could you please be more specific about the 403 pages you have seen. I don't see anything coming up in Webmaster Tools about those kind of errors. Thank you.

02 Mar 2014, 00:28
#12
lhungil avatar

lhungil

Totally Zenned

Join Date:
Feb 2012
Location:
mostly harmless
Posts:
1,818
Plugin Contributions:
4

Re: Google Crawl Errors

traytray:

Could you please be more specific about the 403 pages you have seen. I don't see anything coming up in Webmaster Tools about those kind of errors. Thank you.
Using Webmaster tools only shows one what the search engines have seen / recorded. You can get a much better picture of crawl errors, requests from script kiddies, and other web requests / responses from the web server's access logs (if enabled). In addition to looking at the logs by hand (mainly when debugging or handling potential security intrusions), most hosting providers will have awstats, logaholic, webalizer, or a similar tool you can use from their control panel (to see filtered versions of the log - such as non "200 / OK" HTTP status codes).

Of course, I do not have access to those for your site (they are the best source of potential issues - although more work then just using webmaster tools)... I tend to keep FireBug open when checking websites... I find it a valuable tool (for example, the "net" tab lets you see the URI request/response details).