HELP! Why is canonical links not being indexed?

DrByte and anyone else: I noticed that most of our website product pages are not showing up high on Google SEO like they once did.

For example, let's take this page on our website:

http://www.mjmmagic.com/store/pennomenon-by-steven-x-trick-p-22866.html

If you look at the source code, I have the META tag set to index,follow.

However, when I ran a quick SEO check and I get the following error:

[Attachment no longer available]

It says that the page is NOT indexable because of a canonical link (whatever that means).

Here is what my ROBOTS.txt file looks like:

User-agent: *
Disallow: /download/
Disallow: /cache/
Crawl-Delay: .5
User-agent: Mediapartners-Google

Can you help me to fix this so that it will say my product pages are INDEXABLE? My METATAGS.PHP for my template doesn't restrict product pages in the ROBOTS_PAGES_TO_SKIP either, so that shouldn't be causing a problem.

Should I remove the "<link rel="canonical">" section from my website? Would that allow the links to be indexed?

I should mention that I am using Ultimate URLs module.

For the record, I just removed the following section in my html_header.php file. Can you tell me if that will cause any unforeseen errors?

<?php if (isset($canonicalLink) && $canonicalLink != '') { ?>
<link rel="canonical" href="<?php echo $canonicalLink; ?>" />
<?php } ?>

NOTE: By removing this, I am now seeing that the page IS INDEXABLE, which is what I was wanting. I mainly need to know if you can see any adverse effects by not having that <link rel"canonical"> tag in there.

Do you have any linked products (i.e. products that "live" in more than one category)? If so, that's where the canonical link is going to help you, since those products' information pages' canonical links are going to be generated in a category-less manner so that all instances of the product have the same canonical link.

Without that, Google "sees" each of those linked products as individual products and you'll get "dinged" for duplicate content.

lat9:

Do you have any linked products (i.e. products that "live" in more than one category)? If so, that's where the canonical link is going to help you, since those products' information pages' canonical links are going to be generated in a category-less manner so that all instances of the product have the same canonical link.

Without that, Google "sees" each of those linked products as individual products and you'll get "dinged" for duplicate content.

Well, yes.....by linked, I certainly have a product that is linked (not duplicated) in more than one category. For example, if I have a DVD that deals with card magic, then it will be in the DVD and the Card Magic categories (even though there is only one database entry for the product itself).

So let me ask you this. If I add the reference to "<link rel="canonical">" on my pages, then it says that the page IS NOT indexable.

https://s14.postimg.org/o5r7f9ncx/8_31_2016_12_30_13_PM.png

But if I REMOVE that canonical reference, then it says that the page IS indexable.

https://s13.postimg.org/errnqyiav/8_31_2016_3_02_16_PM.png

Do you know how I could have the best of both worlds (allowing the page to be indexed AND not be dinged for duplicate entries)?

Sounds like a possible problem with your template's meta_tags.php module (that's where the no-index would be added).

That's what I thought too......but when I go to /includes/languages/english/custom/metatags.php, this is my entry:

define('ROBOTS_PAGES_TO_SKIP','login,logoff,create_account,account,account_edit,account_history,account_history_info,account_newsletters,account_notifications,account_password,address_book,advanced_search,advanced_search_result,checkout_success,checkout_process,checkout_shipping,checkout_payment,checkout_confirmation,cookie_usage,create_account_success,contact_us,download,download_timeout,customers_authorization,down_for_maintenance,password_forgotten,time_out,unsubscribe,info_shopping_cart,popup_image,popup_image_additional,product_reviews_write,ssl_check,shopping_cart,back_in_stock_notification_subscribe,back_in_stock_notification_unsubscribe');

Nothing in there seems to be blocking my product pages.

And again, it doesn't say that the problem is with the META tags themselves. It says the page isn't indexed because of a canonical url. In other words, it seems the only reason why it's not being indexed (according to that image in my first post) is because it has a canonical url. But if that is the case, how can anything ever be indexed if it has a canonical url?

See my confusion?

So if the results of that tool can be trusted (questionable), then the solution would be to only apply a canonical link when the current product/category combination is not the master category for that product. The problem with that solution though comes with things like sales/specials where such actions apply to entire categories such that individual product may need to be placed in a category that is not displayed but is put on sale and the product linked to that sale category. Thus every displayed version of the product would have a canonical link and therefore per that tool it would not be indexed.

Now, on the issue of being indexed though, having some form of webmaster tool associated with the search engine of choice should show what has/has not been indexed which is a far better tool than something found on the Internet attempting to identify what criteria must be met or excluded in order to be indexed. Understand that how high in a ranking is not the same as being indexed. If it ranks low, that's an indication of its presence but with low quality. If it can't be found at all? Well, then possibly not indexed or there just is not enough time to find where it is ranked. (those webmaster tools would be better at addressing the presence.)

jeff,
lat9's first post is telling you something. i do NOT think you understand cannonical, which is where your confusion comes in.

i will attempt to explain how i see this situation. i "think" it is correct, but i could be wrong.

#1 - it has NOTHING to do with your robots.txt file. NOTHING.
#2 - it "looks" like it might have been correct to start. but you have removed the canonical tag as far as i can tell, which i think is a mistake.
#3 - because you are NOT showing up high on google's search could be due to any number of factors, removing the canonical link in the header will ensure that you will come out LOWER, IMHO.
#4 - you need to decide what your canonical link should be!

the canonical link is the link that you want google to index.

please understand that.

when google says, we are NOT indexing this page due to a canonical link; they already have that page in their index using a different URL. that is what you want.

for example, look at the following URLs:

http://www.mjmmagic.com/store/selenium-shift-by-chris-severson-and-shin-lim-presents-video-download-p-22933.html
http://www.mjmmagic.com/store/selenium-shift-by-chris-severson-and-shin-lim-presents-video-download-p-22933.html#.V8eIltwrKV4
http://www.mjmmagic.com/store/index.php?main_page=product_info&products_id=22933&cPath=15
http://www.mjmmagic.com/store/index.php?main_page=product_info&products_id=22933
http://www.mjmmagic.com/store/index.php?main_page=product_info&products_id=22933&cPath=25
http://www.mjmmagic.com/store/index.php?main_page=product_info&products_id=22933&cPath=25#.V8eI8twrKV4
http://www.mjmmagic.com/store/index.php?main_page=product_info&products_id=22933&cPath=45#.V8eI8twrKV4
http://www.mjmmagic.com/store/carlwhat-p-22933.html

everyone of those links goes to basically the same place. you only want google to index that page once! what page do you want google to index that references all of those pages? THAT should be your canonical link. and it should be in the header of every page with the same canonical tag.

look at the last link. suppose i had a competing magic shop and i wanted to mess with you. i could construct a website on magic with links like that last one; and if you did not have a canonical link in the header for that link, google would index that page and BANG your get penalized for duplicate content.

so the idea that a URL does not get indexed due to a canonical link is a GOOD idea, not a bad one.

when you have a product in multiple categories, you need to figure out what the canonical link should be for any product (include master_category or not; linked categories should NOT be included in any canonical IMHO; and use your URL rewriting or not) and code accordingly.

the web is like the wild wild west. people can create links to your site that you never thought of; ZC may even do so if you are not careful with your coding! google and other search engines will find those links. and they will index them unless the canonical tells them ONLY use this URL to get to the content on that page.

i hope that makes sense. ... and more importantly, i hope it is correct! (...i think it is!)

best.

First off, thank you all for hopping on and helping me to understand canonical links better. In addition to your posts, I have also been reading up on them and I think I have a far better understanding.

The tool that I was using (to diagnose the SEO stuff) is a free add-on in Firefox called SEO Doctor:

https://addons.mozilla.org/en-US/firefox/addon/seo-doctor/

What led me to this wild goose chase is that I took a look at some new competition and noticed their SEO ranking was higher. Ok, no big deal. So that made me try this SEO Doctor on my own site, just as a test, to see what it diagnosed. The very first thing I saw was that my page was NOT Indexable because it pointed to a canonical link (which was the same formatted link that I was visiting).

I then looked at my competition, pulling up the source code for their page, and I noticed a couple of things:

1. They did NOT have any reference to a <link rel="canonical"> on their pages.
2. The SEO Doctor said that their page WAS indexable

That is why I jumped to the conclusion that since I had the <link rel="canonical"> reference on MY page, and that MY page was not being indexed because of it, then that must be the reason. By removing that <link rel="canonical"> reference and rechecking with the SEO Doctor plugin, it now said my page WAS indexable.

However, I certainly don't want to be penalized (like you illustrated, Carlwhat) for having multiple links.

SO I PUT THE <link rel="canonical"> BACK ON MY SITE.

I just worry now because this SEO Doctor is back to saying the page isn't indexable, and now I don't know if that's a "false positive" or if it even matters.

The best way to tell is to investigate what has been indexed. If it has, then yes false positive, if not then we'll, learned something. :)

I don't know if this will help but just in case... another site I work on is on v. 1.5.1

all the pages have the "<link rel="canonical">" on them

this one is not being indexed:
http://couturetricotrepentigny.com/boutique/

this one is being indexed:
http://couturetricotrepentigny.com/boutique/cours/broderie-machine

the meta tags are pretty much the same but for one word

Another site I am working on presently on v.1.5.5.a has the same issues... one category will be indexed, another is not and I haven't filled the meta tags yet and I can confirm that both were indexed but can't tell you about they are doing as far as rank.