Forums / General Questions / Ampersands Not Being Converted

Ampersands Not Being Converted

Locked

Views: 2,482

Results 1 to 12 of 12
This thread is locked. New replies are disabled.
5 Feb 2016, 12:43 AM
#1
feznizzle avatar

feznizzle

Totally Zenned

Join Date:
Apr 2010
Posts:
900
Plugin Contributions:
0

Ampersands Not Being Converted

Here is my staging site to demonstrate the problem:
http://2staging.apswittcosales(DOT)com/

To quickly see the problem, run it here:
https://validator.w3.org/

As you can see, the ampersand is being spit out into href's as a title w/o first being scrubbed.

ZC has a built in scrubber that can be seen here:
/includes/functions/functions_general.php
(I opened a clean copy of zc151 and compared that to my current version. They were identical.)

I don't think that file is the source of the problem. I think somewhere else (where the data is being called) is simply failing to use that set of functions replace special characters with html.

In this particular case, replace "&" with "&".

Those titles are stored in my database as text, not html.

Can somebody point me in the right direction?

5 Feb 2016, 12:27 PM
#2
feznizzle avatar

feznizzle

Totally Zenned

Join Date:
Apr 2010
Posts:
900
Plugin Contributions:
0

Re: Ampersands Not Being Converted

NEW: BOUNTY ON THE TABLE! :)

Bump!

I think I need to open a bounty on this problem. If you think you can figure this problem out, PM me your paypal address so I can buy you some coffee.

Regarding the problem, it seems to be output related only, not input. I was able to put a bandaid on the problem by going into the actual data (category title and product title fields) and replacing "&" with "&".

Test the results, run that validator for the actual site:
http://smtsupplies(DOT)com

But that site is about to grow massively and my clients will be doing most of the work. Forcing them to use html specials is less than ideal!

Thanks!

5 Feb 2016, 12:30 PM
#3
mesnitu avatar

mesnitu

Totally Zenned

Join Date:
Aug 2014
Location:
Lisbon
Posts:
597
Plugin Contributions:
0

Re: Ampersands Not Being Converted

Feznizzle:

Can somebody point me in the right direction?

html5 ? :smile:

“Though the problems of the world are increasingly complex, the solutions remain embarrassingly simple.” ― Bill Mollison

5 Feb 2016, 12:38 PM
#4
mc12345678 avatar

mc12345678

Totally Zenned

Join Date:
Jul 2012
Posts:
16,908
Plugin Contributions:
2

Re: Ampersands Not Being Converted

Huh, I'd say that "correcting" the output, could be a huge chase, there are several places that output or could use the data that represents the category title. Just from the samplng provided there's the category name(s) in the category listings and then once navigating the site, there's the bread-crumbs at the "top" of the screen, then there's the sitemap just to name a few places. Fixing the "input" well, then there is the potential issue of using other software to modify the database contents, also not sure what the effect is/would be on the on-site search functionality for those product. There's also the possibility of not allowing the & symbol at least to let them know they shouldn't use it. But yes there's a function in the ZC arsenal that could change it to the html equivalent.

ZC Installation/Maintenance Support <- Site
Contribution for contributions welcome...

5 Feb 2016, 1:13 PM
#5
kobra avatar

kobra

Black Belt

Join Date:
Aug 2005
Location:
Arizona
Posts:
31,500
Plugin Contributions:
4

Re: Ampersands Not Being Converted

I see &'s on your staging site
Using FireFox v44.0

Zen-Venom Get Bitten

5 Feb 2016, 1:55 PM
#6
lat9 avatar

lat9

Administrator

Join Date:
Sep 2009
Location:
Stuart, FL
Posts:
14,109
Plugin Contributions:
56

Re: Ampersands Not Being Converted

There was some discussion on the ampersand topic, as applicable to HTML validation, a couple of months ago. IIRC, the answer was if you want the information you're entering to validate, enter valid data!

5 Feb 2016, 3:51 PM
#7
feznizzle avatar

feznizzle

Totally Zenned

Join Date:
Apr 2010
Posts:
900
Plugin Contributions:
0

Re: Ampersands Not Being Converted

kobra:

I see &'s on your staging site
Using FireFox v44.0

True. Because on that site, the database contains &'s. On the live site, you will also see &'s but those are actually html (which validates).

lat9:

There was some discussion on the ampersand topic, as applicable to HTML validation, a couple of months ago. IIRC, the answer was if you want the information you're entering to validate, enter valid data!

That's what I did on the live site. But I have several problems with that. First, I'm lazy! That's a lot of extra work when dealing with thousands of products and hundreds of categories. And it's dangerous, counting on all users to remember to clean special characters when building content. On the flip site, what happens when the data is needed elsewhere? It has to be cleaned in the opposite direction, replacing html specials with text versions.

Then there is the possible complications that MC12345678 points out!

mesnitu:

html5 ? :smile:

That's very scary to me. I'm going to be looking at it, but its very scary!

++++++++++++++++++++++++++
I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

////
//CLR 030228 Add function zen_decode_specialchars
// Decode string encoded with htmlspecialchars()
  function zen_decode_specialchars($string){
    $string=str_replace('>', '>', $string);
    $string=str_replace('<', '<', $string);
    $string=str_replace(''', "'", $string);
    $string=str_replace('"', "\"", $string);
    $string=str_replace('&', '&', $string);
    [B]$string=str_replace('°', '°', $string);[/B]

    return $string;
  }

////

I'm not sure what's going on there, but I like the idea that I can add specific characters, based upon needs. I added that last one as an example.
++++++++++++++++++++++++++

Guys, I did find this bit of code (from here: http://php.net/manual/en/function.htmlspecialchars.php)

string htmlspecialchars ( string $string [, int $flags = ENT_COMPAT | ENT_HTML401 [, string $encoding = ini_get("default_charset") [, bool $double_encode = true ]]] )

I wish I could program! I think I need something like that to clean the output anywhere that an alt tag or href tag gets displayed (sent to a browser). As MC12345678 points out, there are lots of places where such output gets created.

But what if that scrubber got placed at a top level, like the main header? One place, not a zillion.

What if there was an observer that just waited for href's and alt's? If it saw one, it scrubbed it before display?

Anyway, can this be done? Can a little snippet be inserted somewhere that listens for alt/href and then cleans before output?

How hard would that be?

5 Feb 2016, 3:59 PM
#8
feznizzle avatar

feznizzle

Totally Zenned

Join Date:
Apr 2010
Posts:
900
Plugin Contributions:
0

Re: Ampersands Not Being Converted

Feznizzle:

I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

Sorry, that was a reference to something I removed! :)

5 Feb 2016, 6:04 PM
#9
mc12345678 avatar

mc12345678

Totally Zenned

Join Date:
Jul 2012
Posts:
16,908
Plugin Contributions:
2

Re: Ampersands Not Being Converted

Feznizzle:

True. Because on that site, the database contains &'s. On the live site, you will also see &'s but those are actually html (which validates).

That's what I did on the live site. But I have several problems with that. First, I'm lazy! That's a lot of extra work when dealing with thousands of products and hundreds of categories. And it's dangerous, counting on all users to remember to clean special characters when building content. On the flip site, what happens when the data is needed elsewhere? It has to be cleaned in the opposite direction, replacing html specials with text versions.

Then there is the possible complications that MC12345678 points out!

That's very scary to me. I'm going to be looking at it, but its very scary!

++++++++++++++++++++++++++
I don't know if that code above is really what I want. I noticed this little routine (/includes/functions/functions_general.php)

////
//CLR 030228 Add function zen_decode_specialchars
// Decode string encoded with htmlspecialchars()
function zen_decode_specialchars($string){
$string=str_replace('>', '>', $string);
$string=str_replace('<', '<', $string);
$string=str_replace(''', "'", $string);
$string=str_replace('"', """, $string);
$string=str_replace('&', '&', $string);
[B]$string=str_replace('°', '°', $string);[/B]

return $string;

}

////

> 
> I'm not sure what's going on there, but I like the idea that I can add specific characters, based upon needs. I added that last one as an example.
> ++++++++++++++++++++++++++
> 
> 
> Guys, I did find this bit of code (from here: <http://php.net/manual/en/function.htmlspecialchars.php>)
> ```
string htmlspecialchars ( string $string [, int $flags = ENT_COMPAT | ENT_HTML401 [, string $encoding = ini_get("default_charset") [, bool $double_encode = true ]]] )

I wish I could program! I think I need something like that to clean the output anywhere that an alt tag or href tag gets displayed (sent to a browser). As MC12345678 points out, there are lots of places where such output gets created.

But what if that scrubber got placed at a top level, like the main header? One place, not a zillion.

What if there was an observer that just waited for href's and alt's? If it saw one, it scrubbed it before display?

Anyway, can this be done? Can a little snippet be inserted somewhere that listens for alt/href and then cleans before output?

How hard would that be?

That type of operation (htmlspecialchars) is provided with a few of the functions in the includes/functions/functions_general.php file:

function zen_output_string($string, $translate = false, $protected = false) {

Or more specifically/easily:

function zen_output_string_protected($string) {

As to the listening for everything aspect... That's a different subject/topic than the above two functions. Would have to prevent any content from being pushed to the display before being processed by the listener. Otherwise the HTML5 aspect just basically is that HTML5 is more "intelligent" about the validation of characters such as & (ampersand), allowing coders/authors to be just as indicated more lazy. :)

Generally the little "snippet" is what is added to the code where it might be expected that html will be included in the content to be displayed. The use of the & in strings such as categories, attributes, and even product (if I remember correctly) has not been a specific character to be addressed in past versions. I haven't looked at ZC 1.5.5 to see how it behaves with that additional character, but when I have helped others to validate their site, I have on occasion made the suggestion that they use & in their original data.

Lot's of ways to go about it though. :)

ZC Installation/Maintenance Support <- Site
Contribution for contributions welcome...

5 Feb 2016, 8:12 PM
#10
mesnitu avatar

mesnitu

Totally Zenned

Join Date:
Aug 2014
Location:
Lisbon
Posts:
597
Plugin Contributions:
0

Re: Ampersands Not Being Converted

If this is so hard to do in zencart because there are a lot of places.... One thing.... probably to get around, but with with some extra steps, it's if you can use 'and', to do a search and replace in a calc or csv file.
I mean, if you use EP4, exporting all that stuff, and replacing & by 'and'. I use a lot EP4 and libreoffice, search and replace... never had much problems. Of course, it's a extra step, so, errors could occur.

Or, just insert the name as you would, and time to time, the same thing, but using search and replace the '&' by the '&'
This would be a admin process, not the users....

“Though the problems of the world are increasingly complex, the solutions remain embarrassingly simple.” ― Bill Mollison

6 Feb 2016, 2:25 PM
#11
feznizzle avatar

feznizzle

Totally Zenned

Join Date:
Apr 2010
Posts:
900
Plugin Contributions:
0

Re: Ampersands Not Being Converted

Thanks for the good suggestions!

Regarding admin vs user process... for sites I admin, no big deal. But this particular site I built and handed over. I come back in occasionally on a project basis. So I was just trying to find a way to prevent mistakes from occurring as they expand. To make it foolproof, as it were.

MC's suggestion looks interesting. It doesn't seem too pervasive to track down and fix. Mainly, I need to php find the file that generates output strings for:
• Product names (inserted into alt/href)
• Category names (inserted into alt/href)
• Product listing pages (inserted into alt/href)
• Meta data (1 for category, 1 for product)

Only five references to fix. I'm mainly concerned with the first three, the other would be a bonus.

If I can track that down, then I can figure out how to wrap the titles in the zen_output_string_protected string.

Anybody know wear to start? Which php files generate the references?