Zen Cart Logo
Forums / General Questions / Japanese Data stored in MySQL as Latin1

Japanese Data stored in MySQL as Latin1

Views: 1,159

Results 1 to 4 of 4
14 Aug 2014, 2:22 AM
#1
urgentprog avatar

urgentprog

New Zenner

Join Date:
Jun 2011
Posts:
32
Plugin Contributions:
0

Japanese Data stored in MySQL as Latin1

I have customer data in Japanese stored in the mysql database as Latin1. In the zen-cart admin, it shows fine.

I am trying to import it into a new zen cart install. The site design has been applied, but it looks like they used a different zen cart version. In the customer table there are new fields like "customers_firstname_kana" and it's in eucjpms_japanese_ci . If I import the data directly using phpmyadmin import/export, it does not show up properly on the new site.

What would be the best way to proceed? Should I switch site back to latin1 and try to use first setup which works correctly? Or import with special characters and question marks and then change table encoding in mysql? I also thought of using a plugin to try exporting directly from the zen cart admin.

14 Aug 2014, 3:08 PM
#2
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: Japanese Data stored in MySQL as Latin1

First need some more information, including:
What is the ZC version for the old site and for the new site?
What is the CHARSET set to in the old site and in the new site?
What is the database collation set to for the products_description field in the products table, on both the old site and the new site?
What plugins are installed on the old site?
What PHP version on old and new site?
What MySQL version on old and new site?

18 Aug 2014, 3:17 PM
#3
urgentprog avatar

urgentprog

New Zenner

Join Date:
Jun 2011
Posts:
32
Plugin Contributions:
0

Re: Japanese Data stored in MySQL as Latin1

DrByte:

First need some more information, including:
What is the ZC version for the old site and for the new site?
What is the CHARSET set to in the old site and in the new site?
What is the database collation set to for the products_description field in the products table, on both the old site and the new site?
What plugins are installed on the old site?
What PHP version on old and new site?
What MySQL version on old and new site?

Old site:
v1.5.3, recently upgraded from v1.2. New upgrade works without problems.
New site:
v1.5.3 clean install

Old Site Char Set:
In admin/includes/languages/japanese.php
mb_language('Japanese');
mb_internal_encoding("EUC-JP");
mb_http_output("EUC-JP");
MySQL zen_customers table and field collation:
latin1_swedish_ci

New Site Char Set:
In admin/includes/languages/japanese.php
mb_internal_encoding("EUC-JP");
mb_http_output("EUC-JP");
MySQL zen_customers table and field collation:
eucjpms_japanese_ci

PHP version
5.3.27

MySQL version both sites
5.5.37-cll

Please let me know if you still require list of plugins.

19 Aug 2014, 9:57 PM
#4
drbyte avatar

drbyte

Sensei

Join Date:
Jan 2004
Posts:
63,513
Plugin Contributions:
176

Re: Japanese Data stored in MySQL as Latin1

I can't speak to the exact steps you need to take, but I would probably advise converting all your custom fields to use the same collation as the original core ZC tables.

Just remember that when converting between latin1 and utf8 you can't merely change the setting in the database or the individual fields without first preserving the data inside each field, else you run the risk of corrupting the contents of each record. I imagine the same problem exists when converting between eucjpms and utf8 or latin1.