Special characters messed up in ZC Admin

[moullas]

Hello there.

Thanks to icouto for the excellent post explaining how to move the whole shop to UTF8

I managed to do it thanks to that guide, then it is my turn to give something back to the community,..
Get the file from darthnet40.com/zencart1.3.7-UTF8.zip

This file is ONLY FOR A NEW INSTALLATION (It creates the correct UTF8 Collation and character sets in the new tables).

[U]To use it, follow the steps:
1) Copy the whole Zencart distribution on your server
2) Put the files i provide on your server, overwriting the default ZC ones
3) Run setup as usual
4) Enjoy

[miep]

I have this some problem when I have to move a shop from one to another server. A lot of charaters are messed up.

The sql file Moullas posted takes only care of a new installation. How can I import an database form a running shop and not run into trouble. Is there a sql file for to run over an existing imported database?

I hope you will understand my question, my native speech is Dutch.

[moullas]

Ok, i don't know if this helps.
The command you have to run is
ALTER TABLE TableName DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;


This changes the collation of the Table. However, i think the fields will stay to their former values. So you'll have to get all your table names and run the command from PHPMyAdmin to change the Table collation (again, time consuming).

This page looks like it could help a bit more, however my PHP and MySQL skills are non-existent, i don't know if someone could make a script out of this to convert an existing structure

http://www.phpwact.org/php/i18n/utf-8/mysql

[pisceandelusions]

WOW! icouto, I owe you BIG TIME! Your solution WORKS PERFECTLY! (see icouto's solution below)

Background on my issue, which was special characters not displaying properly:

I have been running a Zen Cart installation at http://www.GermanFoodatJosies.com for just over 2 years now. We set everything up in English originally thinking it would save time and resources. The store works great and has been all this time, but lately I've really been putting a lot of extra time into it in an effort to take advantage of a lot more of the features such as Manufacturers, and now Languages.

All of our product descriptions were in English for the most part, and we have been going through the products and trying to add the German description in below the English one. The umlauts and special characters always worked for the English site.

I downloaded and installed the German Language pack yesterday and everything worked great right away, for the most part. I had to go in and redo page 1, page 2, and page 3, and also translated my privacy policy, shipping & other information in the info box because they weren't set up at all for German. I'm not sure why the installation required me to recreate those pages, but it wasn't too much trouble with Google Translate.

After all those were set up, I also had to then manually put in the category names in German, because they didn't translate either. No big deal, just a little time consuming.

Now all of the Sidebox headings translated over fine, just like the home and login links.

Everything is working great now at this point, except for one more thing, and this is where icouto saves the day for me.

I spent hours researching this everywhere, btw, and somehow finally found this thread just when I was about to give up. I will go back to the other threads and post icouto's answer so they're all informed as well.

Ok, so back to the final problem, and that was the product descriptions. As I said, after a few manual translations, everything for the most part was working great but the product descriptions - for some reason, the special characters worked fine on the English side that also had German language in it, but on the German side, the special characters were inverted question marks (not working).

I followed icouto's steps and now everything is working perfectly. Actually, I did one thing differently which saved a TON of time - since only my product descriptions needed fixed, I only edited the "products description" table and the text fields within it using pHpMyAdmin. Once inside there, under "products description", I edited the collation value to utf8_general_ci. There was no need for me to go in and edit all of the other tables.

Also, make sure you do step 3, because 1 & 2 will not do it alone.

Thanks again, icouto.

***please note, icouto's instructions say to change it to utf8_generic_ci - I did not see a "generic" and assumed that he meant "general", which is working fine for me.

[ivanbr]

OK. I followed all the instructions to have everything with utf-8. All my input in admin and in website with special characters registers fine and is accessed fine from DB but now all data requested from browser to php files like "privacy notice" comes out garbled when it came out fine with iso-8859-1.

Can anyone help?

[flix]

Thanks a lot to icouto!!!

I have done exactly how you described and my product and admin pages now show German characters perfectly! German special characters are also saved correctly in the database.

But there are still 3 issues I have been trying to solve for days now (I am using ZC v1.3.8):

----------------------------------------------------------------------------
1) PROBLEM WITH DATABASE BACKUP / RESTORE

When I produce a backup of my database and restore the backup using phpMyAdmin, I see via phpMyAdmin that special characters are now saved wrongly in the database (e.g. in the initial database I had 'Zürich' but in the restored db it is written 'Zürich') and therefore special characters are also displayed wrongly in the shop interface. Any idea how I can solve this issue?

----------------------------------------------------------------------------
2) PROBLEM HANDING OVER SPECIAL CHARACTERS TO PAYPAL IPN

And this is now a very mean thing where I am completely lost: I want my clients to use PayPal IPN - Website Payments Standard (my shop uses the preinstalled module). But the thing is that whenever a client clicks on the confirmation button and is redirected to the PayPal site, the special characters of the client's address in the prefilled PayPal form are displayed wrongly (this time, 'Zürich' is not shown as 'Zürich' but instead the 'ü' is replaced by a small questionmark in a black square).

I checked that in the PayPal Admin panel, profile->language encoding->more options->encoding is set to UTF-8. So I would think that throughout Zen Cart and PayPal I am using UTF-8.

I next deleted the code line $s = preg_replace ('/&([a-zA-Z])(uml|acute|elig|grave|circ|tilde|cedil|ring|quest|slash|caron);/', '$1', $s); from the function replace_accents($s) in the file includes/functions/functions_general.php.

This actually seemed to solve my problem: Now 'Zürich' was displayed correctly as 'Zürich' in the PayPal address form.

----------------------------------------------------------------------------
3) UNSUPPORTED CHARACTERS MESSAGE IN PAYPAL

When I thought that I had solved issue 2), issue 3) poped up immediately: Now PayPal is displaying everything nicely and correctly and clients are also able to process the payment with PayPal, but unfortunately they are displayed the following message in red letters: "You have entered unsupported characters for this field. Current available language character types are: European, Chinese, Korean, Japanese, and Thai. Please try again".
I am now completely lost. I have read that this issue should have been solved years ago in an earlier version of Zen Cart!?!?! Furthermore, PayPal support told me that I am their only client facing such an issue and they would not be able to help me.
-> Does anyone have a clue how I could end my misery????


I am using the following settings in the PayPal IPN module:
Enable PayPal Module
True

Business ID
[email protected]

PDT Token (Payment Data Transfer)


Transaction Currency
Selected Currency

Payment Zone
--none--

Set Pending Notification Status
Pending [1]

Set Order Status
Processing [2]

Set Refund Order Status
Pending [1]

Sort order of display.
1

Page Style
iSupply

Mode for PayPal web services

Default:
www.paypal.com/cgi-bin/webscr
or
www.paypal.com/us/cgi-bin/webscr
or for the UK,
www.paypal.com/uk/cgi-bin/webscr
www.paypal.com/cgi-bin/webscr

Debug Mode
Off

[flix]

Yesssss!!!!!!!!!!!!! I have got the solution!!!

I have tested the following browsers and found out that only Chrome shows the error message! Seems that Chrome has a bug here.

Crome 5.0 Error
Firefox 3.6 OK
IE 8.0, 7.0 OK
Safari 4.0 OK
iPhone/Safari OK


I am actually a little bit surprised that no one else seems to have spotted this bug. Could anyone of you shorty test your PayPal IPN integration with Chrome and let us know whether you are experiencing the same problems?

[frank18]

OK. I followed all the instructions to have everything with utf-8. All my input in admin and in website with special characters registers fine and is accessed fine from DB but now all data requested from browser to php files like "privacy notice" comes out garbled when it came out fine with iso-8859-1.

Can anyone help?

Open /includes/languages/YOUR_TEMPLATE/english.php

Around lines 56-57 you find

// charset for web pages and emails
define('CHARSET', 'utf-8');

set to

// charset for web pages and emails
define('CHARSET', 'iso-8859-1');

Now do the same in admin/includes/languages/english.php around lines 65-66

No more 'funny' characters