Eliminating dual content due to missing cPath variable

[all4coffee]

I've now put it live on my site, tested with some other linked products and it uses the main category for cPath in all cases, so far so good. I'd like to hear back from you wether or not using this is a good idea or not and for what reasons.

Maybe you also have some thoughts on what could be going wrong for all4coffee below?



Glad you're happy with the code, but there seem to be some problems... somewhere...

First off, the first link seems to have an incomplete cPath? When following the second link it redirects to cPath 6_19, while the first link is just 19, did you mean to post it like that?

Basically, the cPath 19 is a problem, because since a cPath is given, the code doesn't think twice about it and doesn't check if the cPath is actually correct.
You could code around that and make it check the cPath even if one is given, but I don't know how heavy that's going to be on the server...

Secondly, when checking my headers using this code, I get a 301 Moved Permanently. When checking the redirect on your site, you get a 301 OK. That's not right. For a normal page you're supposed to get a "200 OK", for a permanent redirect that's "301 Moved Permanently", this is some weird combination of the two. I'd think that's something you need to contact your hoster about, because search engines aren't getting the right header, or even a valid one, when redirecting now.

Thanks for responding. The quoted links just pasted this way but when you look on display at the bottom of the browser they differ - anyway your code redirects perfectly the one without "cPath". Now when I know how to look up the server header info and that it returns the intended 301 (thanks Website Rob for explanation) I did some additional testing on my site and after checking all possible combinations (I also used the duplicate content links from Google webmaster tools - all seems to be working smoothly and seamlessly! Thank you so much for the code - I'm using 1.3.7 BTW and don't want to upgrade if I don't have to.

Maybe it is coincinental but until Google started pointing out duplicate tags, titles, content our site listed much higher. Thanks again everybody.

[Diavire]

Thanks for responding. The quoted links just pasted this way but when you look on display at the bottom of the browser they differ - anyway your code redirects perfectly the one without "cPath". Now when I know how to look up the server header info and that it returns the intended 301 (thanks Website Rob for explanation) I did some additional testing on my site and after checking all possible combinations (I also used the duplicate content links from Google webmaster tools - all seems to be working smoothly and seamlessly! Thank you so much for the code - I'm using 1.3.7 BTW and don't want to upgrade if I don't have to.

Maybe it is coincinental but until Google started pointing out duplicate tags, titles, content our site listed much higher. Thanks again everybody.

Great to hear it worked out well for you >)

One thing though, I wasn't talking about how the links looked, but the full links.
The first one you posted has "19" as cPath, and the second one (without cPath) redirected to "6_19", check it out.

[all4coffee]

Great to hear it worked out well for you >)

One thing though, I wasn't talking about how the links looked, but the full links.
The first one you posted has "19" as cPath, and the second one (without cPath) redirected to "6_19", check it out.

They lead to the same product... Is this a problem I should worry about?

[Diavire]

They lead to the same product... Is this a problem I should worry about?

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

[DrByte]

IMHO, all the above is just more reasons why people should not worry about Duplicate Content.

Reference: Duplicate content

Duplicate content generally refers to substantive blocks of content within or across domains that either completely match other content or are appreciably similar. Mostly, this is not deceptive in origin. Examples of non-malicious duplicate content could include:

* Discussion forums that can generate both regular and stripped-down pages targeted at mobile devices
* Store items shown or linked via multiple distinct URLs
* Printer-only versions of web pages

Note the highlighted comments above, from Google's own documentation:

Examples of non-malicious duplicate content could include:
* Store items shown or linked via multiple distinct URLs


I would add the following, taken from the same referenced document, which explains the reason for the hype:

However, in some cases, content is deliberately duplicated across domains in an attempt to manipulate search engine rankings or win more traffic. Deceptive practices like this can result in a poor user experience, when a visitor sees substantially the same content repeated within a set of search results.

...

Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results.

Since your store is not attempting to be deliberately deceptive in manipulating search engine results, there is less reason to be overly concerned about the matter. Google is not going to severely penalize you or bump you out of their indexed pages just because of having multiple links to the same content on your site.
That doesn't minimize the fact that it's prudent to keep duplication to a minimum. Google goes on to say that one of their reasons for desiring less duplicate content is so shoppers viewing search engine results don't have to sort through multiple listings of the same thing when looking at google pages, theoretically making their shopping experience more pleasant ... in Google's eyes anyway.

[Diavire]

Note the highlighted comments above, from Google's own documentation:

Examples of non-malicious duplicate content could include:
* Store items shown or linked via multiple distinct URLs


I would add the following, taken from the same referenced document, which explains the reason for the hype:

Since your store is not attempting to be deliberately deceptive in manipulating search engine results, there is less reason to be overly concerned about the matter. Google is not going to severely penalize you or bump you out of their indexed pages just because of having multiple links to the same content on your site.
That doesn't minimize the fact that it's prudent to keep duplication to a minimum. Google goes on to say that one of their reasons for desiring less duplicate content is so shoppers viewing search engine results don't have to sort through multiple listings of the same thing when looking at google pages, theoretically making their shopping experience more pleasant ... in Google's eyes anyway.

I did read those, indeed, but as you said, always good to minimize potential problems, also, google isn't the only search engine, even if it's the by far most used one, and I don't know how other search engines deal with that kinda thing.

I do agree with the fact that, since none of this is deliberate, it's unlikely that it'll do bad things to your ratings ;D

[all4coffee]

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links. Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

[Diavire]

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links.

It may take some time for google to check all links, so more might be taken out with time. For me the number has been steadily declining over the past few days.

Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is really weird..
I've compared your headers with the one from my own site, and they're no different, except for the 301 OK message (which is fine according to all search results I've found about it), and server specific info.

They're also the same messages as other redirects have given that I've been using for months on our site.

The only thing I've noticed is that these zen-cart redirects give two results on this header check site: http://www.seoconsultants.com/tools/headers.asp.
The first is the page that redirects and it's code, the second is the result of the page it was redirected to.

I made three small test php files that redirect to eachother (1 to 2, 2 to 3, 3 shows some text and it ends there), and those show two things.
1: When redirecting from the first testfile, it only shows info about the page it's redirected to, and no redirects after that. So if your site was redirecting multiple times, it should (I think), show only the first redirect. Since the header checker shows the correct redirected page for your site, it would seem that it doesn't redirect multiple times, as google seems to imply.

2: The test pages show only one result, where zen-cart redirects show two. The test page shows only the result from the page redirecting, and not the page it redirected to, while zen-cart links do.

To be honest I have no idea what any of that means, I'm just hoping someone more knowledgable can do something to help with that info.

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

Dunno about Ultimate SEO, I've read nothing but trouble from those kinds of mods...

I don't see anything odd in the sitemap file either, linkwise, but upgrading might help, no idea though.

....hold on, just compared it to mine, and while the only real difference is that the graphical edges around the links look a little different... None of the links in my or your sitemap have any cPaths, what the hell? o_O

[Website Rob]

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is a common fallacy amongst most people. Although each Search Engine has their own Rules they go by, for the most part, the URL one is Redirected to is not indexed. This is evidenced by the msg. Google provided, you.

When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

In other words; Replace the URLs that are being redirected with the 'real' URLs; the ones you end up on after being redirected.


I'm not sure why your 'xml' Site map has so many invalid URLs but you should replace them with actual/correct URLs; no redirection involved.

[all4coffee]

In other words; Replace the URLs that are being redirected with the 'real' URLs; the ones you end up on after being redirected.

I'm not sure why your 'xml' Site map has so many invalid URLs but you should replace them with actual/correct URLs; no redirection involved.

I used the piece of code provided on the first page of this thread which changes/redirects URLs without "cPath" into ones with "cPath" and gives Google 301 redirect message... Sitemap was created with Google Sitemap automatically so I have no idea how would I replace "invalid URLs with actual/correct URLs" as you suggest... Should the URLs in the sitemap include the "cPath" bit or not? I'm really lost here trying to eliminate duplicate content on one side (which slowly seems to work with the aforementioned code provided in this thread), and on the other side fighting the Google too many redirects message...

I now submitted another sitemap generated with www.xml-sitemaps.com
utility, Google accepted it but hasn't indexed it yet so I don't know what their "comments" would be on this.
Here is the link to it:
http://www.all4coffee.com/sitemap.xml

Does this one look better in your opinion?