Eliminating dual content due to missing cPath variable

[all4coffee]

I've now put it live on my site, tested with some other linked products and it uses the main category for cPath in all cases, so far so good. I'd like to hear back from you wether or not using this is a good idea or not and for what reasons.

Maybe you also have some thoughts on what could be going wrong for all4coffee below?



Glad you're happy with the code, but there seem to be some problems... somewhere...

First off, the first link seems to have an incomplete cPath? When following the second link it redirects to cPath 6_19, while the first link is just 19, did you mean to post it like that?

Basically, the cPath 19 is a problem, because since a cPath is given, the code doesn't think twice about it and doesn't check if the cPath is actually correct.
You could code around that and make it check the cPath even if one is given, but I don't know how heavy that's going to be on the server...

Secondly, when checking my headers using this code, I get a 301 Moved Permanently. When checking the redirect on your site, you get a 301 OK. That's not right. For a normal page you're supposed to get a "200 OK", for a permanent redirect that's "301 Moved Permanently", this is some weird combination of the two. I'd think that's something you need to contact your hoster about, because search engines aren't getting the right header, or even a valid one, when redirecting now.

Thanks for responding. The quoted links just pasted this way but when you look on display at the bottom of the browser they differ - anyway your code redirects perfectly the one without "cPath". Now when I know how to look up the server header info and that it returns the intended 301 (thanks Website Rob for explanation) I did some additional testing on my site and after checking all possible combinations (I also used the duplicate content links from Google webmaster tools - all seems to be working smoothly and seamlessly! Thank you so much for the code - I'm using 1.3.7 BTW and don't want to upgrade if I don't have to.

Maybe it is coincinental but until Google started pointing out duplicate tags, titles, content our site listed much higher. Thanks again everybody.

[Diavire]

Thanks for responding. The quoted links just pasted this way but when you look on display at the bottom of the browser they differ - anyway your code redirects perfectly the one without "cPath". Now when I know how to look up the server header info and that it returns the intended 301 (thanks Website Rob for explanation) I did some additional testing on my site and after checking all possible combinations (I also used the duplicate content links from Google webmaster tools - all seems to be working smoothly and seamlessly! Thank you so much for the code - I'm using 1.3.7 BTW and don't want to upgrade if I don't have to.

Maybe it is coincinental but until Google started pointing out duplicate tags, titles, content our site listed much higher. Thanks again everybody.

Great to hear it worked out well for you >)

One thing though, I wasn't talking about how the links looked, but the full links.
The first one you posted has "19" as cPath, and the second one (without cPath) redirected to "6_19", check it out.

[all4coffee]

Great to hear it worked out well for you >)

One thing though, I wasn't talking about how the links looked, but the full links.
The first one you posted has "19" as cPath, and the second one (without cPath) redirected to "6_19", check it out.

They lead to the same product... Is this a problem I should worry about?

[Diavire]

They lead to the same product... Is this a problem I should worry about?

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

[all4coffee]

I wrote about that in an earlier post in this thread, what happens is, nomatter what the cPath, as long as the product ID is the same, it'll show that product, the cPath decides what category is shown along with the product, so, by changing the cPath, the product is shown in a category that it's not supposed to be in.

I'll try and explain with example.

The numbers in the cPath variable tell the system what category to show.
"6" is "Coffee, Tea & Chocolate"
"19" is "Loose Tea Blends"
19 is a subcategory of 6, so "6_19" tells the system to show an opened "Coffee, Tea & Chocolate -> Loose Tea Blends"
Just "19" shows only "Loose Tea Blends", without "Coffee, Tea & Chocolate".

Functionally, this doesn't pose a problem, but the dual content issue isn't fixed, it's replaced with a different dual content issue.
First you had duplicates with a link with and without a cPath.
Now you have two different cPath links, one correct and one incomplete.

How the incomplete one got there I have no idea

I hope you understand what I'm trying to say, I had a bit of difficulty putting it into words ;)

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links. Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

[Diavire]

I think I uderstand but this is really beyond my technical knowledge. Your code still seems to work and it did eliminate about 300 out of 1600 reported by Google "duplicate content" links.

It may take some time for google to check all links, so more might be taken out with time. For me the number has been steadily declining over the past few days.

Today however I got this sitemap warning from Google:

URLs not followed
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

And here are the "guilty" URL listed by Google (first URL in each pair is as it is listed by Google, the ones below in each pair I added as they are being displayed after being redirected by your code)

http://www.all4coffee.com/index.php?...roducts_id=158

http://www.all4coffee.com/index.php?...roducts_id=158

-------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=199

http://www.all4coffee.com/index.php?...roducts_id=199
--------------------------------------------------------------
http://www.all4coffee.com/index.php?...roducts_id=159

http://www.all4coffee.com/index.php?...roducts_id=159
--------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=91

http://www.all4coffee.com/index.php?...products_id=91
----------------------------------------------------------------------
http://www.all4coffee.com/index.php?...products_id=64

http://www.all4coffee.com/index.php?...products_id=64
---------------------------------------------------------------------

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is really weird..
I've compared your headers with the one from my own site, and they're no different, except for the 301 OK message (which is fine according to all search results I've found about it), and server specific info.

They're also the same messages as other redirects have given that I've been using for months on our site.

The only thing I've noticed is that these zen-cart redirects give two results on this header check site: http://www.seoconsultants.com/tools/headers.asp.
The first is the page that redirects and it's code, the second is the result of the page it was redirected to.

I made three small test php files that redirect to eachother (1 to 2, 2 to 3, 3 shows some text and it ends there), and those show two things.
1: When redirecting from the first testfile, it only shows info about the page it's redirected to, and no redirects after that. So if your site was redirecting multiple times, it should (I think), show only the first redirect. Since the header checker shows the correct redirected page for your site, it would seem that it doesn't redirect multiple times, as google seems to imply.

2: The test pages show only one result, where zen-cart redirects show two. The test page shows only the result from the page redirecting, and not the page it redirected to, while zen-cart links do.

To be honest I have no idea what any of that means, I'm just hoping someone more knowledgable can do something to help with that info.

Here is the link to my products Sitemap which Google took "samples" from:

http://www.all4coffee.com/sitemapproducts.xml


I'm using Google Sitemaps module (v.1.3.14) to generate my sitemaps. Should I upgrade to the latest vesion of Sitemaps (maybe generated links will look different on updated version of this mode? - I couldn't find any info on it). Also there is an Ultimate SEO mode - would this help? I'm lost.

Dunno about Ultimate SEO, I've read nothing but trouble from those kinds of mods...

I don't see anything odd in the sitemap file either, linkwise, but upgrading might help, no idea though.

....hold on, just compared it to mine, and while the only real difference is that the graphical edges around the links look a little different... None of the links in my or your sitemap have any cPaths, what the hell? o_O

[Website Rob]

The above confuses me even more... I was under impression that the code redirects duplicates with 301 permanent redirect so Google takes a note of this and indexes only the one w/ cPath

That is a common fallacy amongst most people. Although each Search Engine has their own Rules they go by, for the most part, the URL one is Redirected to is not indexed. This is evidenced by the msg. Google provided, you.

When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot because they contained too many redirects. Please change the URLs in your Sitemap that redirect and replace them with the destination URL (the redirect target). All valid URLs will still be submitted

In other words; Replace the URLs that are being redirected with the 'real' URLs; the ones you end up on after being redirected.


I'm not sure why your 'xml' Site map has so many invalid URLs but you should replace them with actual/correct URLs; no redirection involved.