special characters and search

[earmsby]

I noticed something a bit odd with a client's ZenCart store recently. When running a search for a word that has a special character in it, some characters are found but some are not. Since I don't normally do much searching on the client's site, I'm sure it's always been like this but I just never noticed. I wondered if there was some logical explanation for this that is just escaping me.

Here's and example of what I mean:

Search term: "Bayoan"
Title Found: "Bayoán" [note A with accent above]

Search term: "uber"
Title NOT Found: "Übers Gebirg' Maria" [note umlaut U]

I haven't tried doing similar searches for foreign characters (like Ç or Ñ) that are like regular characters but it seems at least initially that some substitutions are made in the search but maybe not others? Is there a way to find out which are?

[mc12345678]

Trying to look up the area of code for this, but was also wondering what charset(s) are in play here? Ie. Includes/configure.php, admin/includes/configure.php and includes/languages/english.php as well as the actual database itself (phpmyadmin).

[earmsby]

The charset for ZenCart is iso-8859-1 and the collation for the database is Latin-1 in case that helps with this question.

[mc12345678]

The charset for ZenCart is iso-8859-1 and the collation for the database is Latin-1 in case that helps with this question.

It could, but there is a more likely "cause" depending on what text is entered where.

The bulk of the search is performed in: includes/modules/pages/advanced_search_result/header_php.php

Look for the section that uses $keywords.

The search goes out and looks at multiple parts of the product for the information. So, the un-formatted version that is returning results may have the text entered in one of the searched fields other than what was posted above. Ie. Metatags, description, etc....

[earmsby]

Right. I thought of that at first but with my "Bayoán" example, the word without the diacritical mark "Bayoan" is not anywhere is the record. Maybe I'll poke around in the PHP file that does the search to see if I can find any clue. My gut feeling is that is has something to do with the charset itsel and less to do with the way the ZenCart search works. But my client thinks that I have set something to automatically translate these characters which of course I haven't. ;)

[lat9]

Another thing to look for is whether the database uses HTML symbols instead of the actual characters themselves: é is not going to match é when searching.

[earmsby]

Another thing to look for is whether the database uses HTML symbols instead of the actual characters themselves: é is not going to match é when searching.

Right, I also thought of that but neither of my examples are using the HTML symbols. Looking at the data directly in the database (via MySQL workbench) and also looking at the source code, I don't see html symbols.

Puzzling indeed. I will keep digging and post if I figure out what causes the difference.

[lat9]

Check (as mc12345678 indicated previously) three values:

1. /includes/configure.php, DB_CHARSET value
2. /includes/languages/{YOUR_TEMPLATE}/english.php or, if a template-override doesn't exist, /includes/languages/english.php, CHARSET value
3. using phpMyAdmin, check the default collation associated with the to-be-searched element.

The values associated with the 3 settings should be similar, e.g. utf8, utf-8 and utf8_general_ci __OR__ latin1, iso-8859-1, latin1_general_ci.

[earmsby]

The charset in both places in ZenCart is: iso-8859-1 and the collation is latin-1 as previously stated.

Further digging on this and I discovered two new things:

1. The search works for at least some other characters (besides accents):

Search term: "sueno"
Title Found: "El Sueño de Tartini"

2. If I actually paste "Sueño" into the search box the search fails and the main home page is displayed - not even the usual no products found search results page.

More digging is next on the actual search code to see what's going on.

[lat9]

I'm not sure what "both places" means relative to the CHARSET: Do you mean both admin and store? What about the DB_CHARSET, present in the configure.php files?