Special characters messed up in ZC Admin

I recently installed a fresh copy of 1.3.0.1 on a new webhost (linux virtual server) The special danish characters (å, ø, æ) are not displaying correctly in ZC admin. They are replaced by ? and gobbledy gook.

I was plagued with similar problems in my database and on the webpages since installing, due to not using the right collation when creating the database. Over the last few days, I've managed to clear that up (with Absolute's kind help) but the problem in ZC Admin remains.

If the info in the database is "Tilbehør", it'll display correctly on the webpage, but in ZC admin, it'll read "Tilbeh?>" If I edit that category and then save without touching the name, it'll then display on my website as "Tilbeh?>". If I fix the name in admin, it'll be incorrect on the webpage and in the database.

I've tried to change the locale and charsets to everything I can think of, but nothing's worked.

Here's some info on my new setup:
PHP version: 4.4.2
MySQL Version Reported = 4.1.18-standard
HTTP_ACCEPT_LANGUAGE= "en-us,en;q=0.5"
HTTP_ACCEPT_CHARSET= "ISO-8859-1,utf-8;q=0.7,*;q=0.7"
SERVER_SOFTWARE= "Apache/2.0.51 (Fedora)"

Please, anybody with some help or suggestions -- I would really appreciate it.

Nobody has any ideas here? Any suggestions would be appreciated.

I realized you guys could use more information. Here's what I have in the main language files:

admin/includes/languages/danish.php:
setlocale(LC_TIME, 'da_DK.ISO_8859-1');
define('CHARSET', 'iso-8859-1');

includes/languages/danish.php:
@setlocale(LC_TIME, 'da_DK.ISO_8859-1');
define('CHARSET', 'iso-8859-1');

Let me know if any other info will help. Thanks, MP

I'm sorry to bump but I'm really desperate for some kind of direction on this problem. I'm willing to pay to have this fixed, because I'm not sure if all the work I'm continuing to do on the site won't be undermined by some initial error in encoding, collating, etc. - It'd be disastrous if I had to start from scratch now, even worse later.

Does anyone have any clue what's going on? Suggestions on things to check, places to look, rumors or innuendos? Anything...

Are the characters displaying properly in the catalog?

How about a guess?

I haven't had this problem with ZC, but other PHP scripts - if a .php file is saved with the wrong encoding/charset, it can mess up the special chars... Since you say it's working in the shop, but not admin, maybe a .php file in admin/ is saved w/the wrong character set?

Second guess - is there anything different in your web server's config between yourstore/ and yourstore/admin. .htaccess, etc?

Sorry - I'm not much help!
Chris

Yes the characters display correctly in the catalog if it's set correctly in the database. The admin is messed up. But, I can correct it in the admin, and then the database and catalog are messed up.

Chris, no -- your guess is helpful! It gets me looking in the right places. I'll check out what you recommend and get back with the answers.

I replaced my admin folder with a fresh 1.3.0.1 copy -- merged in my configure settings. The problem's still there.

I don't have any htaccess files in my admin. What else should I check?

I noticed in my database that some fields (category_description, etc) were collated as utf-8_danish_ci and some other fields were latin1_danish_ci. So, I created a new database as utf-8, exported my database, changed all the fields to utf-8_danish_ci in a text editor. I ran this exported file into my new database. It worked fine, my site is back up but... Nothing changed! The characters are still displaying incorrectly.

Is it okay to just switch the collation of these "fields" (what are they called??) in a text editor?

And, of course... anything else I should try to narrow it down?

magicpants:

So, I created a new database as utf-8

How are you creating the database - in cpanel, etc?

What options for character set (collation?) are you given when you make a new database?

Is it possible to install software on your host? I use phpMyAdmin to access the mysql databases directly - if you can install it (or it's already installed), it might give you a clue to what character sets are available.

Chris

Yeah, I'm using phpmyadmin already -- that's how i have been creating the databases, etc. I'm using phpMyAdmin 2.6.4-pl4, and MySQL 4.1.18-standard.

When I create a database, I can set the collation -- I've tried both latin1_danish_ci and utf8_danish_ci with it set appropriately in the zen cart files. But, ALSO, on the right hand side of phpmyadmin, there's a few more settings: I can set the "language" (currently en-utf-8) and the "mySQL connection collation" (currently set to utf8_danish_ci). Both of these can be set by me. However, a third setting : "MySQL charset" can not be changed by me and it's currently set to UTF-8 Unicode (utf8). These are all the settings I see on this page, and I've tried many different combinations to no avail.

Further, I noticed that when I create a new database with a collation, let's say latin1_danish_ci, then export my current database, fix all the weird characters and change all the references to latin1_danish_ci in a text editor, then import that sql file into the new database, it looks okay. But, then if I immediately export to a text file, the sql file has the characters all screwed up again! HOWEVER, if I export the file WITHOUT SAVING A FILE -- just opening it on the screen -- it's fine! This could be an unrelated quirk of mySQL or phpmyadmin, I don't know...

I went through my php info and extracted all lines that had something to do with language, charsets, collations, etc.... I'll list them below. Maybe it'll shed some light on what's going on? Thanks so far, Chris...

Apache Environment

HTTP_ACCEPT_LANGUAGE en-us
HTTP_REFERER http://MYSITE.COM/pma/main.php?lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
HTTP_COOKIE pma_theme=original; pma_collation_connection=utf8_danish_ci; pma_lang=en-utf-8; pma_charset=iso-8859-1
QUERY_STRING lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
REQUEST_URI /pma/phpinfo.php?lang=en-utf-8&server=1&collation_connection=utf8_danish_ci

HTTP HEADERS INFORMATION

Referer http://MYSITE.COM/pma/main.php?lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
Accept-Language en-us
Cookie pma_theme=original; pma_collation_connection=utf8_danish_ci; pma_lang=en-utf-8; pma_charset=iso-8859-1
Content-Type text/html; charset=UTF-8

PHP Variables

_REQUEST["collation_connection"] utf8_danish_ci
_REQUEST["pma_collation_connection"] utf8_danish_ci
_REQUEST["pma_collation_connection"] utf8_danish_ci
_REQUEST["pma_lang"] en-utf-8
_REQUEST["pma_charset"] iso-8859-1
_GET["lang"] en-utf-8
_GET["collation_connection"] utf8_danish_ci
_COOKIE["pma_collation_connection"] utf8_danish_ci
_COOKIE["pma_lang"] en-utf-8
_COOKIE["pma_charset"] iso-8859-1
_SERVER["HTTP_REFERER"] http://MYSITE.COM/pma/main.php?lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
_SERVER["HTTP_ACCEPT_LANGUAGE"] en-us
_SERVER["HTTP_COOKIE"] pma_theme=original; pma_collation_connection=utf8_danish_ci; pma_lang=en-utf-8; pma_charset=iso-8859-1
_SERVER["QUERY_STRING"] lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
_SERVER["REQUEST_URI"] /pma/phpinfo.php?lang=en-utf-8&server=1&collation_connection=utf8_danish_ci
_SERVER["argv"] Array([0] => lang=en-utf-8&server=1&collation_connection=utf8_danish_ci)

I feel your pain - I had similar issues with a db-driven spanish site...

I'm running out of suggestions, but if you could export a couple of entries with the problem into a sql file, i'd be happy to take a look at it and see if I can reproduce/fix the problem...

I'll PM you my email addr.

Chris

Further testing this issue, I just installed a fresh 1.3.0.1 cart, separate from my real one. Chose latin1_danish_ci for the database collation and MySQL connection collation. (Of course, the MySQL charset was set to UTF-8 Unicode, because I can't change that setting)

I manually added one product with danish characters through the admin - and it shows up wrong in the database and on the website.

It must be a problem with the server or my php version or mysql version or who knows! I'd appreciate it if someone from the zen cart team could chime in here. Because I suspect this is a compatibility issue and it's already pushed us way off schedule.

Chris, thanks again -- i'll contact you by email...

Hi,

Is it possible for you to post a full phpinfo for your site. If you do not want to do this publically, then please pm me or drbyte.

I'm also looking at the possibility this may be an mbstring problem.

BTW wher in admin are you entering this value Tilbehør,

Is it as a product name, a category name or something else. ??

Just the results of my initial fiddlings

Initially I was having problems where admin worked fine with Tilbehør, used as a ategory name. It would display OK in admin, and I could repeatedly edit/save it without any problems.

However it always dislplayed incorrectly in Catalog.

I set character encoding in catalog and admin english.php to UTF-8 and everything works fine.

Note this is a default mysql5 install using the detalt latin1_swedish collation

Okay, I finally got my webhost to clear up the mbstring problem. It's installed and I thought this would finally solve the problem for us. But, it didn't.

After resolving mbstring -- I exported the database AGAIN, fixed the errors in Win32Pad, imported into a new database with latin1_danish_ci. Tried both iso-8859-1 and utf-8 in the language files, and the EXACT SAME PROBLEM.

I'm going to send another PM to wilt and dr. byte right now -- giving you guys full access to our system. I'm about two days away from throwing in the towel and looking into organized crime as a career option. It is %¤#%¤&@$ crazy how many hours I have wasted from my life trying to investigate this, and not even moving one small step toward solving it.

The man of the hour is Wilt! He fixed it up all nice and easy.

For anybody having this kind of problem in the future, it was because our server was somehow overriding the charset set by Zen Cart.

In our case, it was solved by adding one line to admin/includes/init_includes/init_sessions.php.

Find:

if (!defined('IS_ADMIN_FLAG')) {
  die('Illegal Access');
}

Add this line under it:

header("Content-Type: text/html;charset=ISO-8859-1");

Just want to say, that this represents a bug in zen-cart.

The problem is this, Zen cart does send out correct html and sets in both admin and catalog

meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

or whatever the charset maybe.

The problem is that Apache also sends out headers setting the charset and if these are different to the html charset, your browser will listen to apache and not ZC.

We do already send override headers in the catalog, but forgot to put them in admin. :(

Needless to say, thye will be there in the next release. (albeit, in a different file than you mentioned above)

However, glad to have helped :)

I am having a similar problem.
I just moved a store from one host to another and now my spanish special characters aren't displaying correctly in the admin and catalog.