Ampersands Not Being Converted

[Feznizzle]

Here is my staging site to demonstrate the problem:
http://2staging.apswittcosales(DOT)com/

To quickly see the problem, run it here:
https://validator.w3.org/

As you can see, the ampersand is being spit out into href's as a title w/o first being scrubbed.

ZC has a built in scrubber that can be seen here:
/includes/functions/functions_general.php
(I opened a clean copy of zc151 and compared that to my current version. They were identical.)

I don't think that file is the source of the problem. I think somewhere else (where the data is being called) is simply failing to use that set of functions replace special characters with html.

In this particular case, replace "&" with "&".

Those titles are stored in my database as text, not html.

Can somebody point me in the right direction?

[Feznizzle]

NEW: BOUNTY ON THE TABLE! :)

Bump!

I think I need to open a bounty on this problem. If you think you can figure this problem out, PM me your paypal address so I can buy you some coffee.

Regarding the problem, it seems to be output related only, not input. I was able to put a bandaid on the problem by going into the actual data (category title and product title fields) and replacing "&" with "&".

Test the results, run that validator for the actual site:
http://smtsupplies(DOT)com

But that site is about to grow massively and my clients will be doing most of the work. Forcing them to use html specials is less than ideal!

Thanks!

[mesnitu]

Can somebody point me in the right direction?

html5 ?

[mc12345678]

Huh, I'd say that "correcting" the output, could be a huge chase, there are several places that output or could use the data that represents the category title. Just from the samplng provided there's the category name(s) in the category listings and then once navigating the site, there's the bread-crumbs at the "top" of the screen, then there's the sitemap just to name a few places. Fixing the "input" well, then there is the potential issue of using other software to modify the database contents, also not sure what the effect is/would be on the on-site search functionality for those product. There's also the possibility of not allowing the & symbol at least to let them know they shouldn't use it. But yes there's a function in the ZC arsenal that could change it to the html equivalent.

[kobra]

I see &'s on your staging site
Using FireFox v44.0

[lat9]

There was some discussion on the ampersand topic, as applicable to HTML validation, a couple of months ago. IIRC, the answer was if you want the information you're entering to validate, enter valid data!

[Feznizzle]

I see &'s on your staging site
Using FireFox v44.0

True. Because on that site, the database contains &'s. On the live site, you will also see &'s but those are actually html (which validates).

There was some discussion on the ampersand topic, as applicable to HTML validation, a couple of months ago. IIRC, the answer was if you want the information you're entering to validate, enter valid data!

That's what I did on the live site. But I have several problems with that. First, I'm lazy! That's a lot of extra work when dealing with thousands of products and hundreds of categories. And it's dangerous, counting on all users to remember to clean special characters when building content. On the flip site, what happens when the data is needed elsewhere? It has to be cleaned in the opposite direction, replacing html specials with text versions.

Then there is the possible complications that MC12345678 points out!

html5 ?

That's very scary to me. I'm going to be looking at it, but its very scary!


++++++++++++++++++++++++++
I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

////
//CLR 030228 Add function zen_decode_specialchars
// Decode string encoded with htmlspecialchars()
  function zen_decode_specialchars($string){
    $string=str_replace('>', '>', $string);
    $string=str_replace('&lt;', '<', $string);
    $string=str_replace('&#039;', "'", $string);
    $string=str_replace('&quot;', "\"", $string);
    $string=str_replace('&amp;', '&', $string);
    $string=str_replace('&deg;', '°', $string);

    return $string;
  }

////

I'm not sure what's going on there, but I like the idea that I can add specific characters, based upon needs. I added that last one as an example.
++++++++++++++++++++++++++


Guys, I did find this bit of code (from here: http://php.net/manual/en/function.htmlspecialchars.php)

string htmlspecialchars ( string $string [, int $flags = ENT_COMPAT | ENT_HTML401 [, string $encoding = ini_get("default_charset") [, bool $double_encode = true ]]] )

I wish I could program! I think I need something like that to clean the output anywhere that an alt tag or href tag gets displayed (sent to a browser). As MC12345678 points out, there are lots of places where such output gets created.

But what if that scrubber got placed at a top level, like the main header? One place, not a zillion.

What if there was an observer that just waited for href's and alt's? If it saw one, it scrubbed it before display?

Anyway, can this be done? Can a little snippet be inserted somewhere that listens for alt/href and then cleans before output?

How hard would that be?

[Feznizzle]

I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

Sorry, that was a reference to something I removed! :)

[mc12345678]

True. Because on that site, the database contains &'s. On the live site, you will also see &'s but those are actually html (which validates).




That's what I did on the live site. But I have several problems with that. First, I'm lazy! That's a lot of extra work when dealing with thousands of products and hundreds of categories. And it's dangerous, counting on all users to remember to clean special characters when building content. On the flip site, what happens when the data is needed elsewhere? It has to be cleaned in the opposite direction, replacing html specials with text versions.

Then there is the possible complications that MC12345678 points out!




That's very scary to me. I'm going to be looking at it, but its very scary!


++++++++++++++++++++++++++
I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

////
//CLR 030228 Add function zen_decode_specialchars
// Decode string encoded with htmlspecialchars()
  function zen_decode_specialchars($string){
    $string=str_replace('>', '>', $string);
    $string=str_replace('&lt;', '<', $string);
    $string=str_replace(''', "'", $string);
    $string=str_replace('&quot;', "\"", $string);
    $string=str_replace('&amp;', '&', $string);
    $string=str_replace('&deg;', '°', $string);

    return $string;
  }

////

I'm not sure what's going on there, but I like the idea that I can add specific characters, based upon needs. I added that last one as an example.
++++++++++++++++++++++++++


Guys, I did find this bit of code (from here: http://php.net/manual/en/function.htmlspecialchars.php)

string htmlspecialchars ( string $string [, int $flags = ENT_COMPAT | ENT_HTML401 [, string $encoding = ini_get("default_charset") [, bool $double_encode = true ]]] )

I wish I could program! I think I need something like that to clean the output anywhere that an alt tag or href tag gets displayed (sent to a browser). As MC12345678 points out, there are lots of places where such output gets created.

But what if that scrubber got placed at a top level, like the main header? One place, not a zillion.

What if there was an observer that just waited for href's and alt's? If it saw one, it scrubbed it before display?

Anyway, can this be done? Can a little snippet be inserted somewhere that listens for alt/href and then cleans before output?

How hard would that be?

That type of operation (htmlspecialchars) is provided with a few of the functions in the includes/functions/functions_general.php file:

function zen_output_string($string, $translate = false, $protected = false) {

Or more specifically/easily:

function zen_output_string_protected($string) {

As to the listening for everything aspect... That's a different subject/topic than the above two functions. Would have to prevent any content from being pushed to the display before being processed by the listener. Otherwise the HTML5 aspect just basically is that HTML5 is more "intelligent" about the validation of characters such as & (ampersand), allowing coders/authors to be just as indicated more lazy. :)

Generally the little "snippet" is what is added to the code where it might be expected that html will be included in the content to be displayed. The use of the & in strings such as categories, attributes, and even product (if I remember correctly) has not been a specific character to be addressed in past versions. I haven't looked at ZC 1.5.5 to see how it behaves with that additional character, but when I have helped others to validate their site, I have on occasion made the suggestion that they use & in their original data.

Lot's of ways to go about it though. :)

[mesnitu]

If this is so hard to do in zencart because there are a lot of places.... One thing.... probably to get around, but with with some extra steps, it's if you can use 'and', to do a search and replace in a calc or csv file.
I mean, if you use EP4, exporting all that stuff, and replacing & by 'and'. I use a lot EP4 and libreoffice, search and replace... never had much problems. Of course, it's a extra step, so, errors could occur.

Or, just insert the name as you would, and time to time, the same thing, but using search and replace the '&' by the '&'
This would be a admin process, not the users....