SitemapXML v.2

Hello,

I have the mod running on my 1.5.0 shop however i'm constantly getting emails (via the email on 404 mod) that search engines reached a page-not-found page, and it is when they are trying to find the sitemap,

Here is one of the emails i get (this one is from bing bot)

A user at host address [msnbot-65-55-213-192.search.msn.com] with IP address
[65.55.213.192] generated a 404 using the query string
[/sitemap/sitemap.xml].

Hi,

ShopVille:

Hello,

I have the mod running on my 1.5.0 shop however i'm constantly getting emails (via the email on 404 mod) that search engines reached a page-not-found page, and it is when they are trying to find the sitemap,

Here is one of the emails i get (this one is from bing bot)

Indeed, the file should not be - /sitemap/sitemap.xml. Why a robot looking for it you have there?
What is the address you gave sitemap search engine?
What is the address specified in the sitemap robots.txt?

a_berezin:

Indeed, the file should not be - /sitemap/sitemap.xml. Why a robot looking for it you have there?

I was going to post the same thing, as I have noticed in my admin who's online google trying to look at sitemap/sitemap.xml and when I click on it from admin it goes to a page not found. So right now I don't know that my google sitemap it doing me any good, and maybe even doing harm.

Is this because the product, category, etc. sitemap files are now in the sitemap folder instead of the root?

ZC 1.5.1

SPH:

I was going to post the same thing, as I have noticed in my admin who's online google trying to look at sitemap/sitemap.xml and when I click on it from admin it goes to a page not found. So right now I don't know that my google sitemap it doing me any good, and maybe even doing harm.

Is this because the product, category, etc. sitemap files are now in the sitemap folder instead of the root?

ZC 1.5.1

It is difficult to suggest anything if you do not even want to answer the questions:

a_berezin:

What is the address you gave sitemap search engine?
What is the address specified in the sitemap robots.txt?

I looked at your http://www.prommart.com/robots.txt:

User-agent: *
Disallow: /cgi-bin/
Disallow: /*zenid=

User-agent: ##########spider
Disallow: /

Sitemap: http://www.prommart.com/sitemap.xml

Sitemap: http://www.prommart.com/sitemap/sitemapproducts.xml

Sitemap: http://www.prommart.com/sitemap/sitemapcategories.xml

Who advised you to do it? Why three sitemap? You did not read that you recommend to the admin sitemapxml?

a_berezin:

Who advised you to do it? Why three sitemap? You did not read that you recommend to the admin sitemapxml?

Thank you. I removed the products and categories sitemap urls from my robots.txt and also removed the products sitemap from google webmasters (which did go to the correct link.)

So now my robots.txt says:

Sitemap: http://www.prommart.com/sitemap.xml

And sitemap.xml is the only sitemap in google webmasters.

Thanks for pointing out my operator error. :)

a_berezin:

What is the address you gave sitemap search engine?
What is the address specified in the sitemap robots.txt?

I submitted to the search engines sitemap.xml without a directory,
however i did not have anything in the robots.txt but i now added this line to my robots.txt:

Sitemap: http://**************.com/sitemap.xml

How can I support UTF8 symbols in the sitemaps?

Currently I have installed Ultimate SEO (latest version) and it's working fine with cyrillic symbols at first sight.

When I try to make sitemap the indexed link is something like that: Свещ-Уют-p-183.html

It opens just fine but the link stays that way in the browser and probably google will have problems with it?

Do I have to manually make all files from this contribution UTF8?

I'm running 1.5.1 version of Zen.

Hi.

ShopVille:

I submitted to the search engines sitemap.xml without a directory,
however i did not have anything in the robots.txt but i now added this line to my robots.txt:

Sitemap: http://**************.com/sitemap.xml

Give a link to the site.

Hi,

the.rampage.rado:

How can I support UTF8 symbols in the sitemaps?

Currently I have installed Ultimate SEO (latest version) and it's working fine with cyrillic symbols at first sight.

When I try to make sitemap the indexed link is something like that: Свещ-Уют-p-183.html

It opens just fine but the link stays that way in the browser and probably google will have problems with it?

Do I have to manually make all files from this contribution UTF8?

I'm running 1.5.1 version of Zen.

SitemapXML uses the standard zen-cart function which should generate the address of the appropriate standard w3c.
If Ultimate SEO creates not valid Cyrillic address - it's a Ultimate SEO problems. Can you give a link to you site?

a_berezin:

Hi,

SitemapXML uses the standard zen-cart function which should generate the address of the appropriate standard w3c.
If Ultimate SEO creates not valid Cyrillic address - it's a Ultimate SEO problems. Can you give a link to you site?

http://www.rampagehockey.eu/neo/graphics-cards-c-1_4/%D0%A1%D0%B2%D0%B5%D1%89-%D0%97%D0%B8%D0%BC%D0%BD%D0%B0-%D0%BF%D1%80%D0%B8%D0%BA%D0%B0%D0%B7%D0%BA%D0%B0-p-182.html here's the link for the problematic item with Bulgarian SEO url in browser but with junk in sitemap.

This is a test installation of 1.5.1 to test out things like cyrillic support and modules I'll be using.

Hi,

the.rampage.rado:

http://www.rampagehockey.eu/neo/graphics-cards-c-1_4/%D0%A1%D0%B2%D0%B5%D1%89-%D0%97%D0%B8%D0%BC%D0%BD%D0%B0-%D0%BF%D1%80%D0%B8%D0%BA%D0%B0%D0%B7%D0%BA%D0%B0-p-182.html here's the link for the problematic item with Bulgarian SEO url in browser but with junk in sitemap.

This is a test installation of 1.5.1 to test out things like cyrillic support and modules I'll be using.
You have the wrong address.
For example, this URL

/neo/graphics-cards-c-1_4/Свещ-Зимна-приказка-p-182.html

must be

/neo/graphics-cards-c-1_4/%D0%A1%D0%B2%D0%B5%D1%89-%D0%97%D0%B8%D0%BC%D0%BD%D0%B0-%D0%BF%D1%80%D0%B8%D0%BA%D0%B0%D0%B7%D0%BA%D0%B0-p-182.html

rfc3986 Uniform Resource Identifier (URI): Generic Syntax. 2. Characters

A URI is composed from a limited set of characters consisting of
digits, letters, and a few graphic symbols. A reserved subset of
those characters may be used to delimit syntax components within a
URI while the remaining characters, including both the unreserved set
and those reserved characters not acting as delimiters, define each
component's identifying data.

A good example is the URL's in Wikipedia - http://bg.wikipedia.org/wiki/Кирилица
Please note, the URL is

bg.wikipedia.org/wiki/%D0%9A%D0%B8%D1%80%D0%B8%D0%BB%D0%B8%D1%86%D0%B0

but not the

bg.wikipedia.org/wiki/Кирилица

Of course, I can add additional processing URL, make it valid with rfc3986.
But:

1. Standard function zen_href_link() is not needed additional processing. It return valid URLs.
2. This will not save you with the troubles indexing your site with an non valid URLs.

OK, If I leave it that way it won't be good for sitemap but I really need those cyrillic letters in the url. What can I do?

the.rampage.rado:

OK, If I leave it that way it won't be good for sitemap but I really need those cyrillic letters in the url. What can I do?
You do not understand. You can not have Cyrillic letters in the URL.

OK I understand that all non latin letters get encoded.

But Google understands that and support cyrillic searches and when you search it still gives good results on 'exact match' with links you have in your site so most cyrillic writing people use cyrillic in URLs. Russia, Bulgaria and few more.

It's a common practice to have cyrillic symbols in the links (when you copy/paste they copy/paste as encoded) but when you follow the link it get 'readable' for the user in their browser address bar.

My problem is that in the sitemap it doesn't get the proper link (the encoded version you pasted above) but some hieroglyphs as I pasted in my first post in the thread. If I follow that link the proper page opens but the address is still the same (it doesn't change neither to cyrillic or encoded version of the link).

the.rampage.rado:

But Google understands that and support cyrillic searches and when you search it still gives good results on 'exact match' with links you have in your site so most cyrillic writing people use cyrillic in URLs. Russia, Bulgaria and few more.
This is misleading. I showed you the example of Wikipedia in fact formed the Cyrillic addresses.

the.rampage.rado:

My problem is that in the sitemap it doesn't get the proper link (the encoded version you pasted above) but some hieroglyphs as I pasted in my first post in the thread. If I follow that link the proper page opens but the address is still the same (it doesn't change neither to cyrillic or encoded version of the link).
Use another sitemap generator.

Or modify it as you need.

OK, I like your generator and I've been using it at site without Friendly URLs for few years now - it's robust and gives good results.

Could you advise how I can fix that to show at least the encoded version?

the.rampage.rado:

Could you advise how I can fix that to show at least the encoded version?
Encoded version I can do - this is not contrary to my principles and standard rfc3986. :smile:
Wait until tomorrow.

If you can do that you'll be my god! :D

Or at least give me directions on how to implement it because your plug-in is really top notch!

the.rampage.rado:

Or at least give me directions on how to implement it because your plug-in is really top notch!

be patience.

the.rampage.rado, try the following change to "Ultimate SEO URLs" and let me know what happens.

change line 185 of "seo.url.php" from

		return htmlspecialchars($link, ENT_QUOTES, CHARSET, false);

to

		return rawurlencode(htmlspecialchars($link, ENT_QUOTES, CHARSET, false));