UTF-8 Zen-Cart Tutorial for v1.3.x

Sorry, I'm not an expert on configuring for Hebrew language.

Maybe this will help? http://www.zen-cart.com/forum/showthread.php?p=584302

If it's a server issue with your PHP configuration, then it's something your hosting company will have to fix. It's NOT in php.ini.

Regarding the date format and day and month names outputted by php, you may find this thread of use regarding the available locales on your server, which may be related to your problem.

http://www.zen-cart.com/forum/showthread.php?t=133111

regards
Steve

Thanks Steve. I knew there was another discussion around here about that.

Thank you Crystal Koi for your great article! I have tried to read it very carefully, but I'm afraid I don't totally understand. (I never had to work with charsets before).

First I will explain the problem. I am maintaining a few zen cart stores for my boss. Within these stores we have german language installed. German language contains a lot of umlaut characters, which are not displayed well in the product information or in the product titles in the menu.

My boss is not happy with it (I can't blame him) and wants me to change it. But I don't know how to change it on a working store. Will your fix work on a running store? And can I convert the db tables to utf-8 without problems? If so, how should I proceed?

Thank you in advance!

I see lots of talk about changing query_factory.php to accommodate UTF-8, but for admin there is also query_factory.php

I see no mention of modding this as well. I did and all seems to work there fine now too.

Also for changing your database after the fact, instead of going in and changing each field in each table, the following script works nicely. It will output a rather large list of alter table sql that you can copy and paste into phpmyadmin and change all at once. I didn't write this, but have used it and my confidence is high. It will also pickup and generate sql for any tables your other mods have added.

Use at your own risk.

<?php
 
// this script will output the queries need to change all fields/tables to a different collation
// it is HIGHLY suggested you take a MySQL dump prior to running any of the generated
// this code is provided as is and without any warranty
 
die("Make a backup of your MySQL database then remove this line");
 
set_time_limit(0);
 
// collation you want to change:
$convert_from = 'latin1_swedish_ci';
 
// collation you want to change it to:
$convert_to   = 'utf8_general_ci';
 
// character set of new collation:
$character_set= 'utf8';
 
$show_alter_table = true;
$show_alter_field = true;
 
// DB login information
$username = 'user';
$password = 'pass';
$database = 'dbname';
$host     = 'localhost';
 
mysql_connect($host, $username, $password);
mysql_select_db($database);
 
$rs_tables = mysql_query(" SHOW TABLES ") or die(mysql_error());
 
print '<pre>';
while ($row_tables = mysql_fetch_row($rs_tables)) {
    $table = mysql_real_escape_string($row_tables[0]);
    
    // Alter table collation
    // ALTER TABLE `account` DEFAULT CHARACTER SET utf8
    if ($show_alter_table) {
        echo("ALTER TABLE `$table` DEFAULT CHARACTER SET $character_set;\r\n");
    }
 
    $rs = mysql_query(" SHOW FULL FIELDS FROM `$table` ") or die(mysql_error());
    while ($row=mysql_fetch_assoc($rs)) {
        
        if ($row['Collation']!=$convert_from)
            continue;
 
        // Is the field allowed to be null?
        if ($row['Null']=='YES') {
            $nullable = ' NULL ';
        } else {
            $nullable = ' NOT NULL';
        }
 
        // Does the field default to null, a string, or nothing?
        if ($row['Default']=='NULL') {
            $default = " DEFAULT NULL";
        } else if ($row['Default']!='') {
            $default = " DEFAULT '".mysql_real_escape_string($row['Default'])."'";
        } else {
            $default = '';
        }
 
        // Alter field collation:
        // ALTER TABLE `account` CHANGE `email` `email` VARCHAR( 50 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL
        if ($show_alter_field) {
            $field = mysql_real_escape_string($row['Field']);
            echo "ALTER TABLE `$table` CHANGE `$field` `$field` $row[Type] CHARACTER SET $character_set COLLATE $convert_to $nullable $default; \r\n";
        }
    }
}
 
?>

Make sure to edit in your proper username and passwd and stuff and do a backup before running the output of this script.

Kudos to whoever wrote this, it has saved me much time.

I have installed recently Zen Cart and follow the instructions for installing the Japanese Pack given on srw2d.com and made the utf8 encoding conversions. The only thing I might have made wrong is that I have overwritten the files: admin/index.php, admin/invoice.php, includes/modules/payment/paypal.php, includes/functions/functions_general.php, includes/classes/order.php after installing Zen cart in English. I don't know if the order is important?

Every things seems to work fine except the Japanese informations in the admin customers panel (next to modules). The characteres are strange, for example 田中 (Tanaka) become 縺輔→縺ｿ . In my SQL Data base the names and address are written correctly in Japanese and I hope that there would not be any problem during the payment?

Any suggestion to correct that problem? I have read all this thread to find a solution but I couldn't find it.

@ kiddo:

Sorry for my late response, I was on holiday so hadn't checked back for a while.

Thank you very much for the script! I will use it later on the week and report back on how it went!

jvanree:

I am maintaining a few zen cart stores for my boss. Within these stores we have german language installed. German language contains a lot of umlaut characters, which are not displayed well in the product information or in the product titles in the menu.

I solved the same problem with this thread's help.

As my database was already online I followed these steps to convert it to UTF-8:

1. Complete export of database from phpmyadmin (including Add DROP TABLE / VIEW / PROCEDURE / FUNCTION setting)
2. Then export file editing to change every instances of CHARSET=latin1 to CHARSET=utf8
3. Import of modified file to recreate database from scratch with every previous content (but in utf8)

@ moosesoom.. Thank you very much for your answer!!

moosesoom:

I solved the same problem with this thread's help.

As my database was already online I followed these steps to convert it to UTF-8:

2. Complete export of database from phpmyadmin (including Add DROP TABLE / VIEW / PROCEDURE / FUNCTION setting)

3. Then export file editing to change every instances of CHARSET=latin1 to CHARSET=utf8

4. Import of modified file to recreate database from scratch with every previous content (but in utf8)

Hello Moosemoon,

Could you write of resume of the steps to follow for an already installed Zen-Cart 1.38 shop.

Thank you.

Having 1.3.9a installed and you code don't work for me maybe my file are never could you help me to fix it..to utf8 thanks to every body..

NOTE:
With v1.3.9 you don't have to touch the query_factory.php file.
You should use this method instead: http://www.zen-cart.com/forum/showthread.php?t=153582

Hi DrByte,

Sorry, I have been busy. We just put out our own open source project, http://mediafront.org/ . I am going to try and update my site with the correct info this coming week. The whole site actually.

Thanks for taking over the thread there Dr.

-Shaun

Hi Everyone,

I am new to ZenCart, i am looking to install Japanese to the site, can anyone comment on which of the two JP Language Pack is the best to use?

CrystalKoi or the one from the Japanese ZenCart page?

japanese_localization_pack_based_on_zencarv_zencartv1-3-0-2l10njp1

Or should i donwload direct from the Japanese website?

PS...does the latest version of Zen-Cart need to be configured for UTF8 still?

Thanks,

Richard

DrByte:

NOTE:
With v1.3.9 you don't have to touch the query_factory.php file.
You should use this method instead: http://www.zen-cart.com/forum/showthread.php?t=153582

Thank you Dr Byte!

There is also a note in that post:

NOTES:

• This is only applicable to v1.3.9x. It will NOT work in older versions, and is not applicable to the future version v2.x.

Future v2.x. will support utf8 won't it? Probably using another method...

Anyway, these are the steps I have followed to make zen-cart-1.3.9e work with utf8 characters:

• 1->I created my database and user.
  2->From phpMyAdmin->Operations.
  I changed Collation to "utf8_general_ci", and then I hit "Go".

Optionally, "perfection maniacs" can try the following guide:
http://journal.suteki.nu/2008/12/25/mysql-to-pure-utf8-character-set-configuration/

• zc_install/includes/installer_params.php

• Before installing *

I adjusted the relevant entry

// optionally set this to 'utf8':
  define('DB_CHARSET', 'utf8');

• includes/extra_configures/my_db_charset.php
  admin/includes/extra_configures/my_db_charset.php

I created the above mentioned directories and files that contain the following code:

<?php define('DB_CHARSET', 'utf8');

• I followed the renowned guide except for the part regarding the query_factory.php file :
  http://www.srw2d.com/content/database-and-collation-important
• For Greek language, I downloaded the "Greek language pack"
  http://www.zen-cart.com/index.php?main_page=product_contrib_info&products_id=1316

I adjusted the relevant files according to the above mentioned guide but I also had to convert the .php files using the following command adjusted in a small bash script:

(I am using Linux, preferably Gentoo)

iconv --from-code=ISO-8859-7 –to-code=UTF-8 "iso.file" > "utf.file"

n=1
find ./ -name '*.php' -type f | sort |  {
		while read files; do
echo "$files"  
fullpath=`dirname "$files"`
filename=`basename "$files"`
new_filename=utf__encoded_$filename
echo $fullpath/utf_$filename
iconv --from-code=ISO-8859-7 --to-code=UTF-8 $files  > $fullpath/$new_filename
mv $fullpath/$new_filename $fullpath/$filename
echo "$n"
let "n += 1"
done 
	}

:clap: Now, everything seems to be working great! :clap:

Thank you for your help converting to utf8!

Kind regards,
orange_juice

Hi. I keep having "cold feet" about what seems to be a straightforward process of converting a database. I thought I had selected all the options to set up a database using UTF8, but after the database was created and I had configured my cart, I saw that most of the files were in latin1_swedish_ci. It appears that changing the collation and entering UTF8 in the boxes that appear in PHPMyAdmin does not actually change the underlying type of the file in PHPMyAdmin.

I find this not adequately explained, even though processes have been detailed on this thread for changing the latin1 to UTF8. I keep puzzling about whether every single latin1 instance is actually detrimental to the smooth operation of the cart, and am concerned that latin1 is 1/3 the size of a UTF8 character and is a space-saver that would in theory speed up a cart.

While I have not fully digested the below, I feel others who are puzzling over this conversion may find these articles of interest. I would be interested in any comments from persons who actually do understand these articles fully, as to (1) why latin1 would not be fine for -- say -- the latin languages, and (2) why some tables could not easily be in latin 1, for example, if one did not need Admin files to be in another language.

This quoted section is from a discussion of the mysql group at [email protected], which came my way in an email from the mysql group on 8/10/10:

Simply creating a new table and copying the data will not work in all
situations.
You need to ensure your client communications also match the character set
of the output and input data.
Using mysqldump is the most foolproof why. The following articles give you a
detailed introduction into the process and the pitfalls.

http://ronaldbradford.com/blog/migrating-mysql-latin1-to-utf8-preparation-2010-02-11/
http://ronaldbradford.com/blog/migrating-mysql-latin1-to-utf8-preparation-2-2010-02-22/
http://ronaldbradford.com/blog/migrating-mysql-latin1-to-utf8-the-process-2010-03-06/

Regards

Ronald

Your post sounds like you're not actually needing UTF8 support. So, why are you dabbling?

Dr. Byte,

Thanks for your comment. Please remove my off-topic post.

I hope to add a Spanish Language Pack to my site. I guess I had better just roll up my sleeves and follow the instructions here and stop worrying about my lack of understanding of MySql.

Believe it or not, your comment helps quite a lot.

Notageek

I followed the steps in Post 88 posted by moosesoom in this thread, since my cart was already set up, and the procedure worked beautifully.

Moosesoom stated:

I solved the same problem with this thread's help.

As my database was already online I followed these steps to convert it to UTF-8:

1. Complete export of database from phpmyadmin (including Add DROP TABLE / VIEW / PROCEDURE / FUNCTION setting)
2. Then export file editing to change every instances of CHARSET=latin1 to CHARSET=utf8
3. Import of modified file to recreate database from scratch with every previous content (but in utf8)

The instruction to check DROP TABLE/VIEW/PROCEDURE/FUNCTION as well as keep the other items which are checked by default in the PHPMyAdmin Export procedure was a critical instruction.

COLLATE=latin1 and COLLATE latin1 also needed to be replaced, but a global search and replace in Notepad++ replaced all of the latin1 occurrences with utf8. My latin1 version was latin1_general_ci (not latin1_swedish_ci), so that fit in with just a replacement of latin with utf8.

I accepted the default name DB on export, selected "save as file" and then after the search and replace named the file to import the same name as the exported file, having been reminded recently that was necessary for the import to succeed.

To be sure the prior table would not interfere with the new table, I dropped the tables, all of which had remained in the database, before importing the changed file.

I mention the above extra detail because someone asked moosesoom to describe the steps in more detail. People more familiar with PHPMyAdmin would automatically know the above.

My site is FASTER now, really noticeably faster.

Thanks to moosesoom and also to Dr. Byte who is always so "on target."

I really thought changing the latin1 would be a fundamental change in the database that would slow my site down, but this is a simple word-replacement procedure that improved the site's speed.

Notageek

Your speed improvements are likely a result of deleting and restoring your data, more than changing the collation method. Similar performance gains could be accomplished using the Optimize Tables approach to tidy the database.