SitemapXML v.2

mario75:

Hi!
I have searched the thread for different keywords and phrases and I don't seem to find my answers in 223 pages. Sorry if it was answered before, but I couldn't find it.

I'm running Zen-Cart v1.5.7c and for that reason I have installed version 3.9.9 of the plugin. I also have learned that "Sitemaps ping is deprecated." when I checked the ping Search Engines checkbox and hit send. I have also seen that mentioned in the plugin's version 4.0.0 description. But from that version upwards there is no support for my 1.5.7.c Zen-Cart.

Now, without updating my Zen-Cart store, what are my options?

I have already:

• Submited my Sitemap via the search engine's submission interface
• Specified the Sitemap location in my robots.txt file.
• the sitemap plugin was installed without errors and is working as it should, also without errors.

Questions:

1. How do I notify crawlers of the update to my XML sitemap?
2. How do I set up a Cron job via my host's control panel to automatically update sitemaps and notify crawlers, but not to ping Google...?

Sorry if the questions seem silly but I'm just a little familiar with php, and all I have learned is from here, from you. Thank you!

The site is: https://mainilemamei.ro/shop/
And the sitemap is at this adress: https://mainilemamei.ro/shop/sitemap.xml
Having the sitemap location identified in your robots.txt file will (eventually, when they drop by) let the crawlers know what's been updated. For a specific search-engine, I'll suggest checking to see if there's an automated way -- perhaps in your SE's management console?

For the cron setup, you can navigate to your admin's Tools :: SitemapXML and view the instructions there.

Thank you!

Bug Report: Sitemap XML v4.0.x - URL Double Encoding with Non-ASCII (e.g., Chinese) Characters in Pretty URLs

Hello Andrew / Sitemap XML Development Team,

I hope this email finds you well.

I am a user of your Sitemap XML module (Version 4.0.x, specifically 3.9.7 base version with updates from May 2023) for Zen Cart, and I've encountered an issue that I believe is a bug related to URL encoding, specifically when using non-ASCII characters (like Chinese) in Pretty URLs.

Problem Description: When generating the sitemapproducts.xml file, product URLs containing non-ASCII characters (e.g., Chinese product names in Pretty URLs) are being double-encoded. This results in URLs like https://hz.com.tw/tanko%25E5%25A4%25A9...-p-4495.html instead of the correct single-encoded format https://hz.com.tw/tanko%E5%A4%A9...-p-4495.html.

This double encoding causes Google Search Console to report these URLs as "Excluded" due to "Blocked by 403 (Forbidden)" errors, as the web server correctly rejects the malformed URLs.

Steps to Reproduce:

Zen Cart setup with "Pretty URLs" enabled (e.g., using Ultimate SEO URLs or similar functionality).

Have products with non-ASCII characters (e.g., Chinese) in their product names/titles, which are used to generate the Pretty URL slug.

Install and configure Sitemap XML module (v4.0.x) with Use cPath parameter in Links set to false.

Generate sitemaps from the admin panel.

Inspect the generated sitemapproducts.xml file (after unzipping if compressed).

Expected Outcome: URLs in the <loc> tags should be single-encoded, e.g., ...%E5%A4%A9...html.

Actual Outcome: URLs in the <loc> tags are double-encoded, e.g., ...%25E5%25A4%25A9...html.

Root Cause Analysis (Proposed Solution):

I've traced the issue to includes/classes/sitemapxml.php. The SitemapWriteItem method is performing a redundant URL encoding step.

In sitemapxml_products.php, zen_href_link() is called, which correctly generates the initial (single) URL encoding for Pretty URLs.

The SitemapWriteItem method then receives this already correctly encoded URL.

Inside SitemapWriteItem, the line $loc = $this->_url_encode($loc); is called.

The _url_encode method, when processing the path component, re-encodes the already existing % characters (from the first encoding) into %25, leading to the double encoding.

Proposed Fix:

To resolve this, I have successfully fixed the issue by commenting out or removing the redundant $loc = $this->_url_encode($loc); line in the protected function SitemapWriteItem(...) method within includes/classes/sitemapxml.php

// sitemapxml.php, inside protected function SitemapWriteItem(...)
// ...
// if ($this->magicSeo) {
// $href = '<html><body><a href="' . $loc . '">loc</a></body></html>';
// $out = unMagicSeoDoSeo($href);
// if (preg_match('@<a[^>]+href=(["'])(.*)\1@isU', $out, $m)) {
// $loc = $m[2];
// }
// }
// !!! PROBLEM LINE: This line causes double encoding for non-ASCII characters in Pretty URLs.
// It should be removed or commented out as zen_href_link() already handles the encoding correctly.
// $loc = $this->_url_encode($loc); // <-- COMMENT OUT OR REMOVE THIS LINE

    if (!$this->_checkDuplicateLoc($loc)) {
        return false;
    }

// ...

This fix ensures that the URL is only encoded once by zen_href_link(), resulting in correct URLs in the sitemap.

I hope this detailed report is helpful for improving the module. Thank you for your hard work on this useful tool!

Best regards,

Suggestion: create a video sitemap as well. Feature request added to Github URL:
https://github.com/lat9/sitemapxml/issues/61

Thank you for keeping this great plugin alive!

showren:

Bug Report: Sitemap XML v4.0.x - URL Double Encoding with Non-ASCII (e.g., Chinese) Characters in Pretty URLs

Hello Andrew / Sitemap XML Development Team,

I hope this email finds you well.

I am a user of your Sitemap XML module (Version 4.0.x, specifically 3.9.7 base version with updates from May 2023) for Zen Cart, and I've encountered an issue that I believe is a bug related to URL encoding, specifically when using non-ASCII characters (like Chinese) in Pretty URLs.

Problem Description: When generating the sitemapproducts.xml file, product URLs containing non-ASCII characters (e.g., Chinese product names in Pretty URLs) are being double-encoded. This results in URLs like https://hz.com.tw/tanko%25E5%25A4%25A9...-p-4495.html instead of the correct single-encoded format https://hz.com.tw/tanko%E5%A4%A9...-p-4495.html.

This double encoding causes Google Search Console to report these URLs as "Excluded" due to "Blocked by 403 (Forbidden)" errors, as the web server correctly rejects the malformed URLs.

Steps to Reproduce:

Zen Cart setup with "Pretty URLs" enabled (e.g., using Ultimate SEO URLs or similar functionality).

Have products with non-ASCII characters (e.g., Chinese) in their product names/titles, which are used to generate the Pretty URL slug.

Install and configure Sitemap XML module (v4.0.x) with Use cPath parameter in Links set to false.

Generate sitemaps from the admin panel.

Inspect the generated sitemapproducts.xml file (after unzipping if compressed).

Expected Outcome: URLs in the <loc> tags should be single-encoded, e.g., ...%E5%A4%A9...html.

Actual Outcome: URLs in the <loc> tags are double-encoded, e.g., ...%25E5%25A4%25A9...html.

Root Cause Analysis (Proposed Solution):

I've traced the issue to includes/classes/sitemapxml.php. The SitemapWriteItem method is performing a redundant URL encoding step.

In sitemapxml_products.php, zen_href_link() is called, which correctly generates the initial (single) URL encoding for Pretty URLs.

The SitemapWriteItem method then receives this already correctly encoded URL.

Inside SitemapWriteItem, the line $loc = $this->_url_encode($loc); is called.

The _url_encode method, when processing the path component, re-encodes the already existing % characters (from the first encoding) into %25, leading to the double encoding.

Proposed Fix:

To resolve this, I have successfully fixed the issue by commenting out or removing the redundant $loc = $this->_url_encode($loc); line in the protected function SitemapWriteItem(...) method within includes/classes/sitemapxml.php

// sitemapxml.php, inside protected function SitemapWriteItem(...)
// ...
// if ($this->magicSeo) {
// $href = '<html><body><a href="' . $loc . '">loc</a></body></html>';
// $out = unMagicSeoDoSeo($href);
// if (preg_match('@<a[^>]+href=(["'])(.*)\1@isU', $out, $m)) {
// $loc = $m[2];
// }
// }
// !!! PROBLEM LINE: This line causes double encoding for non-ASCII characters in Pretty URLs.
// It should be removed or commented out as zen_href_link() already handles the encoding correctly.
// $loc = $this->_url_encode($loc); // <-- COMMENT OUT OR REMOVE THIS LINE

    if (!$this->_checkDuplicateLoc($loc)) {
        return false;
    }

// ...

This fix ensures that the URL is only encoded once by zen_href_link(), resulting in correct URLs in the sitemap.

I hope this detailed report is helpful for improving the module. Thank you for your hard work on this useful tool!

Best regards,
GitHub tracking issue created: https://github.com/lat9/sitemapxml/issues/62

Thank you, @showren, for the detailed explanation and proposed correction.

v4.1.0 of Sitemap XML is now available for download: https://www.zen-cart.com/downloads.php?do=file&id=367

This release contains changes for these GitHub issues:

#62: Correct double-encoding for non-ASCII characters in "pretty" URLs
#63: Add support for zc300's zen_config functionality.
#64: Don't include disabled EZ-Pages that are part of a chapter.