Eliminating dual content due to missing cPath variable

[Diavire]

They lead to the same product... Is this a problem I should worry about?

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

[all4coffee]

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links. Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

[Diavire]

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links.

It may take some time for google to check all links, so more might be taken out with time. For me the number has been steadily declining over the past few days.

Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is really weird..
I've compared your headers with the one from my own site, and they're no different, except for the 301 OK message (which is fine according to all search results I've found about it), and server specific info.

They're also the same messages as other redirects have given that I've been using for months on our site.

The only thing I've noticed is that these zen-cart redirects give two results on this header check site: http://www.seoconsultants.com/tools/headers.asp.
The first is the page that redirects and it's code, the second is the result of the page it was redirected to.

I made three small test php files that redirect to eachother (1 to 2, 2 to 3, 3 shows some text and it ends there), and those show two things.
1: When redirecting from the first testfile, it only shows info about the page it's redirected to, and no redirects after that. So if your site was redirecting multiple times, it should (I think), show only the first redirect. Since the header checker shows the correct redirected page for your site, it would seem that it doesn't redirect multiple times, as google seems to imply.

2: The test pages show only one result, where zen-cart redirects show two. The test page shows only the result from the page redirecting, and not the page it redirected to, while zen-cart links do.

To be honest I have no idea what any of that means, I'm just hoping someone more knowledgable can do something to help with that info.

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

Dunno about Ultimate SEO, I've read nothing but trouble from those kinds of mods...

I don't see anything odd in the sitemap file either, linkwise, but upgrading might help, no idea though.

....hold on, just compared it to mine, and while the only real difference is that the graphical edges around the links look a little different... None of the links in my or your sitemap have any cPaths, what the hell? o_O

[Website Rob]

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is a common fallacy amongst most people. Although each Search Engine has their own Rules they go by, for the most part, the URL one is Redirected to is not indexed. This is evidenced by the msg. Google provided, you.

When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

In other words; Replace the URLs that are being redirected with the 'real' URLs; the ones you end up on after being redirected.


I'm not sure why your 'xml' Site map has so many invalid URLs but you should replace them with actual/correct URLs; no redirection involved.

[all4coffee]

In other words; Replace the URLs that are being redirected with the 'real' URLs; the ones you end up on after being redirected.

I'm not sure why your 'xml' Site map has so many invalid URLs but you should replace them with actual/correct URLs; no redirection involved.

I used the piece of code provided on the first page of this thread which changes/redirects URLs without "cPath" into ones with "cPath" and gives Google 301 redirect message... Sitemap was created with Google Sitemap automatically so I have no idea how would I replace "invalid URLs with actual/correct URLs" as you suggest... Should the URLs in the sitemap include the "cPath" bit or not? I'm really lost here trying to eliminate duplicate content on one side (which slowly seems to work with the aforementioned code provided in this thread), and on the other side fighting the Google too many redirects message...

I now submitted another sitemap generated with www.xml-sitemaps.com
utility, Google accepted it but hasn't indexed it yet so I don't know what their "comments" would be on this.
Here is the link to it:
http://www.all4coffee.com/sitemap.xml

Does this one look better in your opinion?

[DrByte]

Is it possible that all your problems stem from what might be a bug in your Google Sitemaps addon, and maybe this whole dual-content thing is caused by that, and not by Zen Cart at all?

[FluffyTheCat]

Is it possible that all your problems stem from what might be a bug in your Google Sitemaps addon, and maybe this whole dual-content thing is caused by that, and not by Zen Cart at all?

I think we have a winner!


The Google Sitemap Addon creates sitemapcategories.xml so all the products on the category pages gets crawled (with cpath).

Then sitemapproducts.xml (created by the addon) lists all the products without cpath.

Hmmm... why is there "duplicate" content.
I use the same addon and have the same problem but I dont worry too much because Google is smart and know its not intentional and I am not trying to be deceptive. But maybe this can be patched in the mod?

[all4coffee]

Is it possible that all your problems stem from what might be a bug in your Google Sitemaps addon, and maybe this whole dual-content thing is caused by that, and not by Zen Cart at all?

Yes, I think this might be it. I should know within few days - I hope. I created another yet sitemap (details in the next post in response to Website Rob). If this works the Google Sitemap addon could go good-bye... especially so many people are having such a hard time installing it correctly... I will report back

[Website Rob]

I now submitted another sitemap generated with www.xml-sitemaps.com
utility, Google accepted it but hasn't indexed it yet so I don't know what their "comments" would be on this.
Here is the link to it:
http://www.all4coffee.com/sitemap.xml

Does this one look better in your opinion?

A World of difference, with regard to the URLs used in the second one. That one is using actual and correct URLs which should vastly improve your standings with "all" Search Engines.

[all4coffee]

A World of difference, with regard to the URLs used in the second one. That one is using actual and correct URLs which should vastly improve your standings with "all" Search Engines.

The only thing I dicovered later that this free service does only up to 500 pages so the sitemap might look good but it is incomplete. I searched and found Inspyder Sitemap Creator, a free standing piece of software which allows not only to create but also to edit sitemaps (priority, frequency). Nice, simple interface, exclusions and rules list, ftp, auto scheduling, ping major engines and most importantly duplicate content detection.
It does cost 39.95 but if it works its worth every penny. It found over 2200 urls on my site and sitemapped around 1800 (half of which I excluded)

I just uploaded it to Google.
http://www.all4coffee.com/sitemap.xml

Being able to create rules for specific type of URLs I found especially useful since Google started complaining (when I changed priority for my products on Google Sitemap addon to 0.8 from default 0.5) that all urls "have the same and not default priority...

While I'm waiting for Googles reaction to my new sitemap I do have a question. Would I need any more the code change proposed at the start of this thread to fight the duplicate content? Or this code is an improvement of how things work/supposed to work anyway. I'm asking because if it serves no purpose I would rather "go back to the ZenCart original...

I will report back with results of the new sitemap submission.