UTF8 Special Characters not coming thru in Meta Title/Description

[Feznizzle]

Hi,

I can use special characters in product descriptions, no problem. If I want to describe angles, I can say:
90° to 45°

And it will come thru as:
90° to 45°

However, if I put that into a meta description or meta title, this is what I get:
90° to 45°

Er, I mean that is what I see when I view the URL here:
seocentro.com/tools/search-engines/metatag-analyzer.html

Here is an example I've been playing with:
http://2staging.apswittcosales**DOT*...e_for_EB-5000S

Incidentally, if I try to go straight up ascii °, I get the same result: °

I opened up phpMyAdmin, checked that I am using utf8_general_ci. And I opened the Operations of meta_tags_products_description table and saw that Collation is set to utf8_general_ci as well.

I'm getting desperate! Can someone tell me what I'm doing wrong?

Thanks!

[lhungil]

For me it is appearing correctly (and is using the HTML entities) in meta content as well as in the page body. Does it appear correctly to you in a web browser (not all free tools play well with character encoding or HTML entities; most major search engines and browsers work fine).


If you want to go a little down the character set rabbit hole, read on!


Word / WordPerfect / Etc
If you are copy / pasting characters from a word processor application... Be aware these often use characters sets other than ASCII or UTF-8 (unless configured otherwise). This can sometimes cause some issues with garbled characters when using "copy / paste" (how does the browser know what character set the data in the clipboard is using). Usually this is fine with modern OS / Browser combinations, but I have seen issues in the past.

Database Character Set
What DB_CHARSET is defined in your configure.php files? These are used for communication with the database (to tell the database server and php how to encode character data and what format to expect character data to be provided as). For best compatibility it should match the collation used by the database. So for "utf8_general_ci", you should probably define "DB_CHARSET" as "utf8".

HTTP(S) Character Set
What CHARSET is defined in your store's core Zen Cart language file(s)? For example the core English language file for the customer facing side is located at "/includes/languages/english.php" and can be overridden by the theme "sheffield_blue" in "/includes/languages/sheffield_blue/english.php". The core English language file for the admin side is located at "/your_admin_folder/includes/languages/english.php".

Currently your website is not using utf-8 (UTF8) as the character encoding for HTTP content. Your website appears to be using iso-8559-1 (latin-1).

This is evident in the HTTP reply:

Content-Type	text/html; charset=iso-8859-1

And in the HTML head element:

<meta content="text/html; charset=iso-8859-1" http-equiv="Content-Type">

Zen Cart 1.5.x will encode character data using the defined CHARSET in your language file(s) before outputting the data (usually contained in HTML or JSON). So in your case it should be "iso-8859-1" based upon what is being returned by the web server and in the HTML head element.

If the defined CHARSET used by Zen Cart does not match the charset you are telling browsers to expect, issues will ensue (such as strange characters). The charset will impact both the customer facing and admin side of the store (and how character data is treated before being sent to the database).

NOTE: If you want to use UTF8 configure CHARSET as "utf-8" (I usually recommend this - although it can be a pain to convert all the data if multiple languages are involved). You may also need to make changes to the theme if the theme does not make use of CHARSET. You may also want to move the CHARSET meta element in the HTML head to appear before any other character data. More information can be found here.

[Feznizzle]

Awesome, thanks lhungil for that most thorough reply!

I initially thought my issue was related to garbage characters scooped up from Word / WordPerfect / Etc, so I have switched away to using to text files. Also, I'm mainly using EP4 for mass population via OpenOffice Calc (CSV files set to UTF8). I thought maybe EP4 was not filtering (or whatever the native filtering process), so maybe it was a garbage in garbage out problem.

But I no longer think that. I entered the above information directly via admin. I mean, I went to Admin, went to the product, clicked on the meta disk and put this in:
90° to 45°

And still got:
90° to 45°

So it happens even when I manually put it in.

I have verified that both of my Config files are set to UTF8.

I will work my way thru everything you said under HTTP(S) Character Set next.

BTW, on my end I see no garbage characters (both admin or front end). Only seems to show up in meta results. Attaching sample result screen shot.

[lhungil]

So that app shows it wrong, but does your browser show it correctly? Firefox and Edge are displaying it correctly for me (and when I checked the HTML source shows entities being used, so would not be a character set issue - most likely an issue with the tool).

That said, I'd probably still check what the tools provided by the major Search Engines show. I don't think it would be a bad idea to switch to UTF-8 for the HTML output, but right now I do not think it is the issue (and you may be spending time tracking down a non-issue). If the tools provided by the major SEs also show an issue, may be simpler to just replace with the word "degree".

[Feznizzle]

...you may be spending time tracking down a non-issue.

Crud.

You are right!!!

I've used that tool for so long, it just never occurred to me that it could glitch!