Zen Follower
- Join Date:
- Mar 2006
- Location:
- Rosebud, Victoria, Australia
- Posts:
- 308
- Plugin Contributions:
- 2
Indexed, though blocked by robots.txt
Google Search Console was warning me:
Coverage > Indexed, though blocked by robots.txt
Affected pages: 1.57K
The number of warnings has increased significantly recently. Google probably changed a policy.
The warnings are mostly for things like:
https://www.scubadoctor.com.au/diveshop/index.php?main_page=popup_image&pID=4896
https://www.scubadoctor.com.au/diveshop/index.php?main_page=popup_image_additional&pID=4612&pic=2&products_image_large_additional=images/ocean-design/ocean-design-moana-msf-pkg_03.jpg
https://www.scubadoctor.com.au/diveshop/index.php?main_page=product_reviews_write&products_id=5691
I can see why it might be useful to let it index the additional images. So I changed my robots.txt file.
Instead of just…
Disallow: /diveshop/index.php?main_page=popup_image*
I now have…
Allow: /diveshop/index.php?main_page=popup_image&pID=*
Allow: /diveshop/index.php?main_page=popup_image_additional*
Disallow: /diveshop/index.php?main_page=popup_image*
That worked.
Google is still insisting that it wants to index pages like
https://www.scubadoctor.com.au/diveshop/index.php?main_page=product_reviews_write&products_id=5692
even though they're blocked by robots.txt
I can't see any reason to allow product_reviews_write pages to be indexed. It's meaningless to index them. It's pointless given the bot and user will just see a login page.
On each product page there are two links in the reviews section like this.
The plan is to add rel="noindex" to those two links. Then maybe Google will get the message and stop complaining.
Zen Cart 1.5.5e, PHP 5.3.29 MySQL 5.5.42