Forums / Upgrading from 1.3.x to 1.3.9 / UTF8 and special characters after upgrade

UTF8 and special characters after upgrade

Views: 2,742

Results 1 to 7 of 7
9 Apr 2011, 7:09 PM
#1
aeolidia avatar

aeolidia

Zen Follower

Join Date:
Mar 2005
Posts:
166
Plugin Contributions:
0

UTF8 and special characters after upgrade

Hello,

I have a UTF8 site set up. Everything was working fine until we upgraded to the latest version of Zen Cart. When doing that, all the special characters started showing strangely.

é is now é for example.

For a quick fix, I did a find and replace in the database to change these characters to their escape characters, but now "é" is showing in the plain text emails sent from the site.

Database collation is utf8_general_ci and products_name is "varbinary," while products_description is "blob."

To try to solve this, I added this file:

includes/extra_configures/my_db_charset.php - set to define('DB_CHARSET', 'utf8');

I also set the character set in zc_install when upgrading, as per a post I saw on the forums.

I also see this:

includes/functions/functions_general.php

Line #1513 : * convert supplied string to UTF-8, dropping any symbols which cannot be translated easily
Line #1518 : function charsetConvertWinToUtf8($string) {
Line #1520 : $string = htmlentities($string, ENT_QUOTES, 'UTF-8');
Line #1530 : if (CHARSET == 'UTF-8') return $string;
Line #1532 : $string = htmlentities($string, ENT_QUOTES, 'UTF-8');

What can I do to get these characters showing correctly? Thanks!

9 Apr 2011, 7:18 PM
#2
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: UTF8 and special characters after upgrade

Two things:

a) replacing the characters in your database just breaks things worse
b) when you upgraded, did you keep your existing database intact and just run the zc_install upgrade against it, as per the instructions? Or did you import it to another database and take that database live? If so, then the import process likely lost all the correct character handling, resulting in bad data.

.
Zen Cart - putting the dream of business ownership within reach of anyone!
Donate to: DrByte directly or to the Zen Cart team as a whole

Remember: Any code suggestions you see here are merely suggestions. You assume full responsibility for your use of any such suggestions, including any impact ANY alterations you make to your site may have on your PCI compliance.
Furthermore, any advice you see here about PCI matters is merely an opinion, and should not be relied upon as "official". Official PCI information should be obtained from the PCI Security Council directly or from one of their authorized Assessors.

9 Apr 2011, 8:00 PM
#3
aeolidia avatar

aeolidia

Zen Follower

Join Date:
Mar 2005
Posts:
166
Plugin Contributions:
0

Re: UTF8 and special characters after upgrade

Hello!

No importing, I just ran the database upgrade! That's what's so puzzling.

10 Apr 2011, 6:15 PM
#4
kobra avatar

kobra

Black Belt

Join Date:
Aug 2005
Location:
Arizona
Posts:
31,500
Plugin Contributions:
4

Re: UTF8 and special characters after upgrade

No importing, I just ran the database upgrade!
Did you backup the database before doing this??

Restore it and determine the character set you were originally using

What is it?

Zen-Venom Get Bitten

10 Apr 2011, 7:32 PM
#5
aeolidia avatar

aeolidia

Zen Follower

Join Date:
Mar 2005
Posts:
166
Plugin Contributions:
0

Re: UTF8 and special characters after upgrade

Yes I did.

Before the upgrade:
DEFAULT CHARSET=utf8

After the upgrade:
DEFAULT CHARSET=utf8

When I use WinMerge to compare the two databases (before and after upgrade exports), I don't see any differences to the character set between the two.

During the upgrade, it stopped partway through and gave me the error mentioned in this thread:
http://www.zen-cart.com/forum/showthread.php?t=157297

I tried all the tips, but none of them worked, so I just ran the last few commands on the database by hand. Ideas?

10 Apr 2011, 8:04 PM
#6
aeolidia avatar

aeolidia

Zen Follower

Join Date:
Mar 2005
Posts:
166
Plugin Contributions:
0

Re: UTF8 and special characters after upgrade

Additionally, if I edit a product from the admin (instead of the database) to add a é to a word, the accented e turns into a question mark.

I don't care too much how this happened, but would like to know the quickest/simplest way to get special characters working again, thanks!

10 Apr 2011, 8:29 PM
#7
torvista avatar

torvista

Totally Zenned

Join Date:
Aug 2007
Location:
Gijón, Asturias, Spain
Posts:
2,876
Plugin Contributions:
7

Re: UTF8 and special characters after upgrade

Steve
github.com/torvista: BackupMySQL, Structured Data, Multiple Copy-Move-Delete, Google reCaptcha, Image Checker, Spanish Language Pack and more...