Hoping to pick the brains of some old pros!

SPH:

RodG, with all respect (truly), don't Feznizzle's indexed CEON sites that are, to my understanding, not using an xml sitemap for the test product, prove that a xml sitemap is not necessary? Or is zencart's "human" default sitemap with its categories, and the category navigation system itself, sufficient?

My first test site, promdressesforless.us, does not have an xml sitemap at all. Yet products show at google. Now, how long that takes...perhaps we'll see with the test product.

BTW, don't yet see anything at Yahoo/Bing. Do we care about them? And are some carts perhaps more optimized for Google (and what would be that difference, one wonders.)

You are reading too much into this, there are many other factors at play here. If Google bots are constantly crawling a site due to an existing xml sitemap telling it that there are frequent changes then it will get indexed a lot faster since the bot is always there crawling. Their bot is on our site almost non-stop due to our sitemap saying there are daily changes. A new product on our site will index on Google in hours, I don't consider it something overly special. But, I am of the opinion that an xml sitemap is better for business and gets things in the SEs faster.

You also seem to be confusing the issue with regards to the bigger sites "rewriting". There very likely is NO rewriting going on. The script natively generates those links on it's own, there is NO redirection (rewriting) going on as is the case with the discussed ZC mods. I mean no disrespect, but I don't know if you are understanding what we've been talking about, or perhaps it's simply a case of misunderstanding from the written word (which can be quite flat in it's projection of the author's intent/meaning). From what you are posting, you seem to be under the impression that any URL with human-friendly text is "rewritten". That is simply not the case. A rewritten link is when a (usually) dynamic link is transformed through code rewriting into some type of human-readable text with or with dynamic aspects. I would bet heavily that these referenced sites practice no rewriting at all.

With regard to the native zen cart sitemap, that is purely for human consumption. It has 0 to do with SEs and to my knowledge (which like every other person's is imperfect) it has 0 impact with regards to a SE.

IMHO, the fact that one site has indexed this early on is pretty meaningless. It just shows that the bots crawl that site more frequently, likely due to the xml sitemap instructions, so it shows up faster. That is totally irrelevant to this experiment although it may prove beneficial to those sites without them in terms of being crawled more often and getting new/changed products indexed faster.

I am going to further our test a little tomorrow when I add this product to 2 fairly dormant sites. One will be the product in a random category and one will be the same product in a category called Wurldbitzer Debibulator. I am curious to see how this A/Bs in the results. I am under the impression that the SEs give more weight to categories with relevant keywords but haven't ever tried to prove it to myself. I won't add anything to the category description, and I don't believe this will negatively impact the overall results and can give us more insights into overall SE behavior/trends. If there is a consensus that this will be detrimental to our test then I will refrain and do this on my own with other fictitious products.

RodG:

Steve. Please don't panic over this, and please DO NOT add a sitemap to the site (just yet). I'm curious as to how long it will take to be indexed (assuming it gets indexed at all). I'd like to suggest that if it doesn't show up in 7 days (from the time the product was added) it isn't going to show at all. In which case, if you then add a sitemap and it shows within the following 8hours or so, we should be able to conclude a sitemap is essential.
Does anyone disagree with this?

Yes, I have to respectfully disagree. His link will show up eventually, perhaps just as fast as the others. For this reason:

SPH:

RodG, with all respect (truly), don't Feznizzle's indexed CEON sites that are, to my understanding, not using an xml sitemap for the test product, prove that a xml sitemap is not necessary? Or is zencart's "human" default sitemap with its categories, and the category navigation system itself, sufficient?

My first test site, promdressesforless.us, does not have an xml sitemap at all. Yet products show at google.

Rod, I was honestly not trying to insult your intelligence or compare apples to oranges by pointing out that a native (read not xml, human friendly) sitemap lives on all ZCs (http://shop.vcsweb.com/index.php?main_page=site_map). Because of this, an xml sitemap is not 100% necessary. All the google bot has to do is find a single link in a chain that will eventually lead to a product you are trying to list.

An xml sitemap lets you make suggestions to SE's about what pages are more important and how often they should crawl your site. But it is not the only way to make your link known to SE's.

Steve, I could kiss you for pointing that out. I was beginning to think I was speaking gibberish that no one could understand. :) Seriously, I thought I was loosing my mind.

And Rod, I don't mean that you aren't smart or you aren't capable of understanding. I was just worried I wasn't correctly assigning words to thoughts.

The fact that Steve understood what I was trying to say makes me think I am speaking the right words, they just aren't being read the same way that I am speaking them sometimes. I hope we can not let missunderstandings take the fun from the conversation!

RodG:

Sure our templates may have an effect, but by definition, the templates should only affect the way the cart 'looks'. The underlying content/functioning doesn't change.

I am not sure I completely agree here. The templates, and thus the stylesheet, can greatly change the way a page's HTML is generated which can have an effect on the weight of different page elements and thus relevance to the SE. For instance, I can make a class or ID be bold or larger font. While not carrying the same weight as an H1 tag it still has more weight than standard text since the SE interprets this as the owner specifying that it's more important/relevant which is why it's bold or larger. All else being equal, a page with the keywords in bold should be given a higher score (how much I really do not know but a number of little things can add up to tip the scales). This is an example of how limited this test will be since we are not all using the same template with the same category structure, etc. and so it can really be open to interpretation in many different ways.

If the water is so muddied when we review the results I propose to retry the experiment with everyone using the stock template, it would eliminate another variable. In fact, I might go so far as to try Niccol's suggestion and do this on subdomains using the stock template on a third dormant domain - one with the native dynamic links and another with a rewriter, this can give us more insight into the barebones of the argument.

Right, it's almost 1am and I am off to bed, I will try to get my end sorted tomorrow, should be a more normal day work-wise so I will have some time to devote to this.

lankeeyankee:

With regard to the native zen cart sitemap, that is purely for human consumption. It has 0 to do with SEs and to my knowledge (which like every other person's is imperfect) it has 0 impact with regards to a SE.

Just a thought, but a link is a link. If there is no sitemap (xml or otherwise) bots will follow all links that appear on the index and subsequent linked pages. If you have some sort of sitemap that can be found (via robot text or via index page) the bots will crawl this as well (and all subsequent links). This includes the human 'friendly' ZC native sitemap. So while it might have been designed initially for humans, it is also an effective means for SE indexing. :)

lankeeyankee:

I am going to further our test a little tomorrow when I add this product to 2 fairly dormant sites. One will be the product in a random category and one will be the same product in a category called Wurldbitzer Debibulator. I am curious to see how this A/Bs in the results. I am under the impression that the SEs give more weight to categories with relevant keywords but haven't ever tried to prove it to myself. I won't add anything to the category description, and I don't believe this will negatively impact the overall results and can give us more insights into overall SE behavior/trends. If there is a consensus that this will be detrimental to our test then I will refrain and do this on my own with other fictitious products.
^ I like this a lot. I use the strategy all the time, but have never tested it. I don't think adding it to the mix will skew anything. Not like we are following an even remotely scientific approach here anyway. ;-)

lankeeyankee:

You are reading too much into this, there are many other factors at play here. If Google bots are constantly crawling a site due to an existing xml sitemap telling it that there are frequent changes then it will get indexed a lot faster since the bot is always there crawling. Their bot is on our site almost non-stop due to our sitemap saying there are daily changes. A new product on our site will index on Google in hours, I don't consider it something overly special. But, I am of the opinion that an xml sitemap is better for business and gets things in the SEs faster.

Then perhaps more than anything this test will establish the desirability of having an xml sitemap with daily updates. Regardless of dynamic url or rewriting.

You also seem to be confusing the issue with regards to the bigger sites "rewriting". There very likely is NO rewriting going on. The script natively generates those links on it's own, there is NO redirection (rewriting) going on as is the case with the discussed ZC mods. I mean no disrespect, but I don't know if you are understanding what we've been talking about

No, I think I now have the jist. Let me substitute "rewriting" for "human readable url". Big sites use them so one conclusion is that they, at least, see some benefit for a "human readable url". That's all that interests me there.

Now, this distinctions with "native human readable urls" interests me with regard to some of our main competition in the fashion/dress biz. My sister's ex-SEO guy thought that some were using custom carts. So maybe they are? Generating "human readable urls" but "natively".

Apparently the best of both worlds?

I want one of those carts. ;)

SPH:

I'd agree that is a difference. But as a non-tech guy, don't see why some urls in an existing xml sitemap would enhance a test product not yet in it. Actually, to my thinking, not being in the xml sitemap would indicate to google that the test product doesn't exist on the site. But then, I'm learning on this thread. :)

No panic here. I'm like you: just want to learn something, if something works or not.

Feznizzle has already helped explain this, but in the interests of furthering your knowledge, the sitemaps are more than a list of URL's for google to index/crawl in that they can contain other data, such as when the page (or site) was last modified, and how frequently you want it indexed.

SPH:

But my second site - fashion-mart.biz - and my recently added prommart.com do both have xml sitemaps, just not regenerated for the test product. So if my test products there still do not show, despite the xml sitemaps, then perhaps we can conclude: 1. CEON is the superior re-writer, or 2. my sites just s u c k for google...or maybe both.

I see nothing amiss with that possible conclusion. I wouldn't have said it in quite the same way though. :)
'Course, then we'll need to consider which of those conclusions is most likely to be correct (unless someone proposes a 3rd). Then that hypotheses can be tested too :)

SPH:

As I recall Feznizzle also has a dynamic url site in the test. What is the reason it has not shown thus far? Are perhaps the other two sites just more favorable for google, apart from CEON? Or is it because it lacks an xml sitemap,.

All good questions, but I can't say, on account of I haven't looked/compared yet. Must do this quickly before the 'natural order of events' kick in because its only a matter of time before all test sites are indexed.

OTOH, as long as there is a bit of discussion as each site turns up we'll have this forum as a record of events.

SPH:

which apparently even dynamic urls need?.

Mind if we standardise this, and not call them "dynamic urls"?

Regardless of whether the URL is generated "on the fly" (dynamic) or whether they are hard coded (static) in both cases they are generated by the application. What appears in the URL is something that we each have our own names for. It is both simpler and more accurate if we try to not name them, but instead refer to them as the Scenario A, B or C format.

What you have refered to as "dynamic urls" could be either scenario B or Scenario C, but never(?) scenario A.

SPH:

(Like with rewriting, maybe the Zencart team doesn't consider xml sitemaps that useful or important.)

1. Where has it been said the Zencart team don't consider rewrites/redirects important? The fact is, ZenCart does use redirects, in several places. The most obvious example is, if not logged in and you point your browser to http://mystore.com/admin/ you'll be redirected to http(s)://mystore.com/admin/login.php instead.

2. I'm sure the ZenCart team are all for XML sitemaps too. These are a worthy addition to any site. It's not something that can be 'built in' to ZenCart though (well, I suppose it could) but it has no more to do with ZenCart than (say) the PayPal module(s). Very desirable, but not a core function.

Please review the three possible scenarios' I fear you are heading off the path a little.

SPH:

EDIT: by different carts for google, I meant zencarts. Some just more "google SEO optimized".

Thanks for the clarification. I was prepared to assume you meant presta, zen, osc, etc, and been wrong. This is why I keep trying to insist that some things be clarified. It is far too easy lose track of things otherwise).

Cheers
Rod

SPH:

Then perhaps more than anything this test will establish the desirability of having an xml sitemap with daily updates. Regardless of dynamic url or rewriting.

I concur with desirability of the XML sitemap for Google.

Before jumping on the 'daily updates' as also being good, this is treating on shaky ground
.
Firstly if you don't actually update the site daily all you will achieve is to have Google re-index the exact same pages(s) again. Additional site traffic for zero benefit.

Secondly I don't know if I read it somewhere, or simply dreamed it, but its possible that if you 'tell' google to (re)index the site every day, but it constantly fails to find any changes, the site will actually be penalised as a result.

SPH:

No, I think I now have the jist. Let me substitute "rewriting" for "human readable url".

No Steve, you CANNOT substitute "rewriting" for "human readable url" This is akin to substituting an Apple for an Orange. Simply not the same thing at all.

SPH:

Big sites use them so one conclusion is that they, at least, see some benefit for a "human readable url". That's all that interests me there.

This is once again back to the B & C scenario comparisons. You won't get any useful data about this from these tests/experiments. There are far too many other variables that will come into play, and we simply don't have the level of control needed to prove anything worthwhile.

This is the reason why I changed my original tact in hoping/expecting the redirect supporters to be able to prove the benefits of "human readable urls". It is impossible(?) for them to do so. In contrast, showing that the redirects are ineffective is something that I'm still hoping is provable. If I'm unable to prove this, it still says nothing about any possible benefits of "human readable url"s, and this aspect of SEO will remain forever in debate.

Putting this another way, if I am unable to prove the negative, and the others are unable to prove the positive, we have a status quo.

If either of us can prove our case, then the other case must be false/invalid/incorrect and beyond dispute.

Ideally, it should be those using and recommending the technology to prove its value, and not for me to disprove it.

I stated very early in this thread that if I created a module that I claim will speed up a site by 50%, would it be sensible to install and use the module even though the benefits I claim can't be proven? In MY opinion, installing a module with no provable benefits is a waste of time. plain and simple.

Cheers
Rod

SPH:

Now, this distinctions with "native human readable urls" interests me with regard to some of our main competition in the fashion/dress biz. My sister's ex-SEO guy thought that some were using custom carts. So maybe they are? Generating "human readable urls" but "natively".

Apparently the best of both worlds?

I want one of those carts. ;)

Don't forget, any gains made with "native human readable urls" is still hotly in debate. There may still be negative consequences!

Cheers
Rod

lankeeyankee:

I am going to further our test a little tomorrow when I add this product to 2 fairly dormant sites. One will be the product in a random category and one will be the same product in a category called Wurldbitzer Debibulator. I am curious to see how this A/Bs in the results. I am under the impression that the SEs give more weight to categories with relevant keywords but haven't ever tried to prove it to myself. I won't add anything to the category description, and I don't believe this will negatively impact the overall results and can give us more insights into overall SE behavior/trends. If there is a consensus that this will be detrimental to our test then I will refrain and do this on my own with other fictitious products.

I see nothing detrimental.

I also see that we now appear to be having a lot of individual sub testing going on (or planned). This is good. Please everyone document any/all findings and observations though, and when the experiments 'end' I plan to go through the forum taking notes and trying to consolidate them into a single document that will hopefully prove some things, disprove others, and leave many others hotly debatable.

At the end of it all we'll probably end up with a few 'discussions' as we'll each have our own ideas as to what has caused what result, bit nonetheless I'm convinced we'll see some patterns emerge that we will all benefit from no matter what out own points of views may be.

Cheers
Rod

Feznizzle:

Just a thought, but a link is a link.)

And never anything else. :)

Feznizzle:

If there is no sitemap (xml or otherwise) bots will follow all links that appear on the index and subsequent linked pages.

Exactly.

Feznizzle:

If you have some sort of sitemap that can be found (via robot text or via index page) the bots will crawl this as well (and all subsequent links).
This includes the human 'friendly' ZC native sitemap. So while it might have been designed initially for humans, it is also an effective means for SE indexing. :)

Yes, BUT, they still aren't in the same ballgame. The xml sitemap is a specific file that the SE's specifically seek out. (but not all of them I must add)

The 'sitemap' via an index page (or similar) is just another collection of links on the page that the crawlers will automatically follow. There is a very good chance that the site product without the XML map will be indexed** via the 'New Products' links **on the site before the crawler gets to read the main_page=sitemap links.

This is yet another variable that will affect** initial** indexing. Does it affect rankings? I've no idea.

Cheers
Rod

ok hopefully ive done this ok,

Date/Time Begin: 3rd January 08:21 uk time

Your name/handle: bn17311

Test product URL: <http://www.onyourbike.co.uk/index.php?main_page=product_info&cPath=1_65&products_id=1336>

SEO redirection tool/method: none

Sitemap in use? yes xml updated after product added

Age of store (if known): domain age 10yrs + various different types of stores, last 2 years approx zen 1.39 with ceon, with fresh install 1.50 approx oct 2012 
without ceon

hopefully thats all you need

bn

Feznizzle:

Yes, I have to respectfully disagree. His link will show up eventually, perhaps just as fast as the others. For this reason:

Thanks for coming back to this, and yes, I do agree that eventually it'll show up for the very reasons given.

Mind if I amend my original statement because on re-reading I know I didn't quite make it clear what I was suggesting/proposing.

Original : I'd like to suggest that if it doesn't show up in 7 days (from the time the product was added) it isn't going to show at all..............

Amended: I'd like to suggest that if it doesn't show up in 7 days (from the time the product was added) we should assume it isn't going to show at all to be of any use for current testing, In which case, if you then add a sitemap and it shows within the following 8hours or so, we should be able to conclude a sitemap is essential (or perhaps just coincidence)

My question 'does anyone dissagree' was pertaining to the 7 day wait. Should we wait longer before trying to give it a kickstart, or perhaps 7 days is too long?

Apologies for the confusion.

BTW, how about when we disagree with each other we just state our cases, the "respectfully disagree" should be a given (and in some circles, "respectfully disagree" is read as "I think you're an acehole and here's why..... ")

Seriously Guys I have no disrespect for any of you participating in this thread. In fact quite the opposite. The way things have progressed is something I think we should all be proud of. It's not easy keeping emotions in check with such a hotly debated topic. To be perfectly frank, I'm still amazed that we are being civil to each other.

Feznizzle:

I hope we can not let missunderstandings take the fun from the conversation!

If it weren't for the misunderstandings we wouldn't be having this conversation in the first place :)

I'm having more fun with this now than when we first started... Not only do we all have a clear picture of each others points of view, but we are working well enough together that we can finally test our various hypothesis... All is good as far as I can see.

Cheers
Rod

bn17311:

ok hopefully ive done this ok,

All is good. Welcome aboard.

Please document any findings/observations and post into the forum.

Cheers
Rod

RodG:

Secondly I don't know if I read it somewhere, or simply dreamed it, but its possible that if you 'tell' google to (re)index the site every day, but it constantly fails to find any changes, the site will actually be penalised as a result.

I think that you are on to something. My sister's professionally SEO'd site uses the commercial Magic SEO URL*, which is incompatible with the zencart sitemap xml plugin. So my sister's ex-SEO guy used automapit for xml...and set nothing for when google should return. Left that blank. Perhaps for same reason you're thinking.

He also suggested the cross-sell plugin to show on the products (main column), with the idea that the dresses rotating through would make the products appear to be "fresh" to google.

This does lead me to wonder what the optimum google revisit setting should be when new content isn't added that often.

Or for that matter, what constitutes "new content." Probably adding a new product, for sure. Perhaps also simply sprucing-up a category or product description.

*Magic SEO URL does come with a free xml sitemap generator, which we are only now using in conjunction with automapit. I'm seeing some issues with automapit and we may get rid of it on those sites.

Feznizzle:

That is why I mistakenly thought you were asking us to generate our own gibberish.
Mega bummer.
Maybe we should start over, this time using the caveats I outlined when I thought you wanted us to generate our own content (limit # of references to the prod words, etc)

Man. I don't know if I have it in me to continue this... :no:

No, No, No, No No.

Wasn't the VERY FIRST and MOST IMPORTANT thing that we ALL agreed on was "Content is King"

If we all used different content then all other testing is worthless, all we'd be doing is seeing who could write the best content description for Google to find.

You've said several times that we 'aren't using the scientific method' and, yes, that is true, there are too many other variables that we have no control over, or will simply sway the results one way or the other no matter what we try to do, BUT, that doesn't mean we should throw** all **of our controls out of the window, especially the one that we all agree is the most important above all else.

By all means, feel free to create your own unique description to see if it does better or worse than the gibberish (all else being equal).

Don't place too much emphasis over the fact that Google has hidden the second site, at least not yet.

If all of the test sites suffer the same fate

1. I will be somewhat surprised, and
2. We'll have shown in a big way what happens with too much duplicate content, so from there we can simply opt to do the search again with the omitted links, and we'll once again be back on relatively level ground for the other planned comparisons.

Cheers
Rod

According to my data we currently have eight sites under test. Data so far

Notify me of corrections or ommisions, etc
Cheers
Rod

RodG:

Secondly I don't know if I read it somewhere, or simply dreamed it, but its possible that if you 'tell' google to (re)index the site every day, but it constantly fails to find any changes, the site will actually be penalised as a result.

I think we had the same dream. I leave most everything on a monthly (what google would typically do anyway), bump up my categories or anything else I expect to change to week. I don't request dailies anywhere (I think), because of that shared dream.

Thanks for organizing the data into a chart. Any chance you could choose a standard date/time zone? This project was initiated by you, so Ausie time will be fine. If you go US, would you please go eastern standard (New York; Steve, Yankee, and me are all in that zone)?

I notice that my time/date is missing. My 3 listings were done within 5 minutes of each, immediately before I posted:
EST (New York): 1pm; Jan 2nd
PST (Calif): 4pm; Jan 2nd
Sth.Aust: 4:30am; Jan 3rd

That was painful to figure out! First I used the time stamp on lankeeyankee's post where he said it was one 1am, meaning the site is 5 hours ahead. Then I figured out my posts occurred, thus my site listings. Then I figured out you were 15hrs 30min ahead (http://www.happyzebra.com/timezones-worldclock/difference-between-New%20York-and-Adelaide.php), then…

sigh

That is annoying. If you decide to skip it, I certainly won't blame you!

Side note, did a search and now see promdressesforless.us. Odd. Link is to a dif product. Maybe his depopulator got featured on that product page (as a new prod or something) right at the moment google was busy crawling? Interesting. But at least he is showing up, my guess is that his actual prod will get indexed pretty soon.
++++

Whoops! I need to make 2 confessions.

First, I just noticed that one site links to another. My Ceon site (the one that got indexed first) provides a link to my Native site. There are no other cross links I am aware of. This will not hurt consideration of Ceon and will only help my non-modded site to get picked up faster. Theoretically, this should cause my non-SEO site to out perform the control non-SEO site. It will have zero effect on the rankings for my optimized site.

Second (this one really irritates me), my optimized site (also the one that got listed first) automatically appended meta title (Wurldbitzer Debibulator [Wurldbitzer Debibulator] : etc). !@#$!!! Perhaps I should have manually changed the meta data, I just didn't think of it. Sorry guys, I just left it to default behavior, it was not intentional. I did the same thing on the listing on the SEO url site (ZC 139h), but no meta was added to it.

In my opinion, these things won't detract from the thrust of the discussion much, but I am wide open to other opinion.

Well, my first entry promdressesforless.us is showing, but indirectly to the search product term.

BTW, I notice this morning that Feznizzle's first "articles" entry is now on Yahoo/Bing.

Feznizzle:

Second (this one really irritates me), my optimized site (also the one that got listed first) automatically appended meta title (Wurldbitzer Debibulator [Wurldbitzer Debibulator] : etc). !@#$!!! Perhaps I should have manually changed the meta data, I just didn't think of it. Sorry guys, I just left it to default behavior, it was not intentional. I did the same thing on the listing on the SEO url site (ZC 139h), but no meta was added to it.

In my opinion, these things won't detract from the thrust of the discussion much, but I am wide open to other opinion.

Same here with my first entry, promdressesforless.us My other products there are custom metas (biohazard symbol in admin), but for this test I let it go to "whatever zencart defaults to". Got the product name, price and apparently the general meta.php. I'd rather not have that, just the product name in title like my other two sites (and they both "defaulted" to just that.)

Perhaps an additional benefit will be seeing if extra words in the meta title help or hinder?