Question about changing database to Utf-8

[kcb410]

Hello everyone,

Currently my database in PHPMyadmin is UTF8__general_ci and I backup and restore the database using utf8
but in Zen Cart I've got Internal Character Encoding set to ISO-8859-1 and DB Collation set to latin1. I've never had any issues with the database but I understand that I should switch to Utf8 to ensure stable and clean importing when using Easy Populate. I've downloaded the MySQL-latin1-to-UTF8-Converter.

Are there any major issues or pitfalls to doing this?
Any help is appreciated. Thank you.

Zen 139h
MySQL 5.0.95
PHP Version: 5.3.10

[DrByte]

Yes, there are significant risks involved. The script you mention is intended for converting a latin1 database to utf8, and that is NOT what you have. So don't use it.

But if you do, TEST IT FIRST on a COPY of your site. NOT ON YOUR LIVE SITE DIRECTLY.

Right now you have latin1 characters in your utf8 tables.
If you retrieve latin1 data from your utf8 tables but treat it as utf8 data on output then it may not display at all, or it may show odd symbols instead. You'll need to clean up all those scenarios by hand. DO NOT USE THAT CONVERSION SCRIPT FOR THIS PURPOSE.

[kcb410]

Yes, there are significant risks involved. The script you mention is intended for converting a latin1 database to utf8, and that is NOT what you have. So don't use it.

But if you do, TEST IT FIRST on a COPY of your site. NOT ON YOUR LIVE SITE DIRECTLY.

Right now you have latin1 characters in your utf8 tables.
If you retrieve latin1 data from your utf8 tables but treat it as utf8 data on output then it may not display at all, or it may show odd symbols instead. You'll need to clean up all those scenarios by hand. DO NOT USE THAT CONVERSION SCRIPT FOR THIS PURPOSE.

Hello DrByte,

Thank you for the reply!

As I mentioned above the database hasn't given any grief other than my ongoing battles with Easy Populate (every version). I'm able to import csv files using ISO-8859-1 with some success.

The only other possible consideration is that I'll be upgrading to Zen 1.5 in the few months.

So am I best to leave current database as it is?

Thanks again!

[DrByte]

In an ideal world you want your database to use the same collation/charset as your PHP and HTML are using.

Since yours is already a hybrid, you're in a complicated situation.

Upgrading to v1.5 won't solve the situation you're in right now. v1.5 isn't a "magic pill" to fix mixed-up data in your database.

If you want to make them both the same, you'll need to manually fix all the data by hand, or else force you and your customers to "put up with" the strange characters you'll encounter.

[kcb410]

In an ideal world you want your database to use the same collation/charset as your PHP and HTML are using.

Since yours is already a hybrid, you're in a complicated situation.

Upgrading to v1.5 won't solve the situation you're in right now. v1.5 isn't a "magic pill" to fix mixed-up data in your database.

If you want to make them both the same, you'll need to manually fix all the data by hand, or else force you and your customers to "put up with" the strange characters you'll encounter.

Hi DrByte,

I guess I'll take an 'If it's not broken... don't fix it' approach for now as the site's perfectly functional. When I have the extra time I'll attempt the change to Utf-8 on a copy of the site.

Thank you for your input.

[torvista]

Just an idea, an alternative may be export/import, while you convert the db inbetween.

ie
You export all the products details (Apsona/Easypopulate) to a csv.
On opening that with Openoffice you can specify the charset (latin1).
On saving it you can specify the charset too (as utf8), then import it back into to the converted db.
You should be able to see any potential conversion problems and fix them in one pass.

I'm able to import csv files using ISO-8859-1 with some success.

"Some success" sounds like failure to me..I import and export continually (utf-8, spanish accents) with Apsona and Openoffice with total success.

[kcb410]

Just an idea, an alternative may be export/import, while you convert the db inbetween.

ie
You export all the products details (Apsona/Easypopulate) to a csv.
On opening that with Openoffice you can specify the charset (latin1).
On saving it you can specify the charset too (as utf8), then import it back into to the converted db.
You should be able to see any potential conversion problems and fix them in one pass.


"Some success" sounds like failure to me..I import and export continually (utf-8, spanish accents) with Apsona and Openoffice with total success.

Thanks Torvista,

Good suggestions. I just need to set aside a decent amount of time to do it....I'm assuming it may not go as smoothly as I would like it to!

[kcb410]

Hello all,

I've now converted my database and required php files to UTF-8 from ISO-8859-1(did it prior to upgrading)and upgraded to Zen Cat 1.5. I've only encountered one glitch regarding strange characters...My products are 'Priced by Attributes and my 'Starting at:$222.222 ' has gone a little crazy with some products, an example can be seen here:
http://www.kcbgraphics.com/index.php...oducts_id=2076

When I look at the attribute pricing in Zen Admin I have this � showing up in 'Attribute Weight Prefix' and 'Price Prefix' in some locations but not all. I assume this is the source of the problem.

When I export product attributes using Apsona and open it in Open Office as UTF8 I have either a 0 or nothing in 'Attribute Weight Prefix' and 'Price Prefix' columns.

What would be the correct strategy to resolve this issue without causing further problems?
Any help is appreciated. Thank you.

Zen 1.5
PHP Version: 5.3.10
Database: MySQL 5.0.95