No content in product editor after upgrading 139-151

Hello chaps

I've just upgraded my old 1.3.9b site to 1.5.1. That went fine and I'm back up and running again.

However there is one big problem - The editor doesn't show any content for nearly all products and also for the define/ezy pages. A few have content, such as the original unedited define_page_4.php, but it's not consistent and the vast majority show up with nothing. The data does exist in the DB.

I've read this post:
http://www.zen-cart.com/content.php?313-should-i-convert-my-store-from-iso-8859-1-to-utf8

And have applied these:
@setlocale(LC_TIME, 'en_US.ISO_8859-1');
define('CHARSET', 'iso-8859-1');

And in the config files I added:
define('DB_CHARSET', 'latin1');

It hasn't fixed the issue. Is there something else I can do?

Thanks for any advice on this.
Rob

Zen Cart 1.5.1
Database Patch Level: 1.5.1
v1.5.1 [2013-10-25 16:43:45] (Version Update 1.5.0->1.5.1)
v1.5.0 [2013-10-25 16:43:45] (Version Update 1.3.9->1.5.0)
v1.3.9b [2010-05-18 18:34:34] (Version Update 1.3.8->1.3.9b)
v1.3.8 [2008-07-30 02:07:56] (Fresh Installation)

PHP Version: 5.2.17
MySQL 5.5.21
Microsoft-IIS/7.0

1. Turn off the editor and try to edit product and view your data.
2. If your data is good when you turn off editor, try to install ckeditor then turn it on.

Thanks for the reply Chu Hai

I've done all the known tests and fixes including checking text-only and I also installed CKEditor, but it's not editor-related. It's to do with the encoding, but the published fix doesn't fix it. The data is all there in the DB and on the product pages, just can't edit it.

Any further suggestions by anyone are much appreciated.
Regards
Rob

Since you said that "the data is all there in the DB", I'm assuming that you are familiar with phpMyAdmin. What shows as the collation when you view the products_description table's structure?

Also, are there any myDebug*.log files present in your store's /logs folder?

Hi lat9

Thanks for the reply, I really appreciate it!

I've got MySQL Administrator installed for managing databases on the server.

When I check that table and look at the collation, it oddly says "latin1_swedish_ci"

Screenshot: http://s23.postimg.org/wrb236y4b/collation.jpg

I've checked this against the previous 139 database and that is the same.

Is there a clue in this?

When we open various products for editing in the store, a few will show content in the editor, but most do not. There is no obvious pattern to this. Some are very old products and some are new, but all were created before the upgrade.

Adding a new product of course works and content is there in the editor. However we are not editing any products until this is fixed.

Is there perhaps a manual method for converting the tables and everything to the new UTF8 or should we be pursuing a fix for Latin?

Thanks for any further advice on this.
Regards
Rob

p.s. no debug logs are present.
The site has been extremely reliable in the past and the upgrade went very smoothly as well.

I've installed the latest PHPMyAdmin to check this as well and the tales and relevant columns are all latin1_swedish_ci

There's no trace of utf8 anywhere that I can see yet the product editor shows nothing.

This is strange, since most of the improper collation-configurations that result in blank product edit pages in the admin also create a debug logfile since it's usually htmlentities complaining about its inability to convert one or more characters.

Are you using an unedited version of /YOUR_ADMIN/products.php? One of the major changes from 1.3.9->1.5.x was the expansion of the arguments to htmlentities and htmlspecialchars calls due to a change in the PHP default. Speaking of PHP, what version are you running on?

Thanks for the reply again lat9

PHP Version: 5.2.17
MySQL 5.5.21

I've used Winmerge to compare myadmin\product.php with one from the 151 install zip and it is identical.

Ok bingo! I reapplied suitable permissions to the logs folder and now a debug file was written as soon as I opened a product.

It contains this:

[01-Nov-2013 01:58:08] PHP Warning: htmlspecialchars() [<a href='function.htmlspecialchars'>function.htmlspecialchars</a>]: Invalid multibyte sequence in argument in C:\inetpub\wwwroot\mydomain\myadmin\includes\modules\product\collect_info.php on line 401

Note that not all products seem to have the problem. Some do have content in the editor and some do not. I can't see any pattern and all products were created prior to the upgrade. Any product without content logs the same error.

Related to this is that when I use the define pages editor to edit define_main_page.php I also get no content in the editor and get the same type of error logged:

[01-Nov-2013 02:38:18] PHP Warning: htmlspecialchars() [<a href='function.htmlspecialchars'>function.htmlspecialchars</a>]: Invalid multibyte sequence in argument in C:\inetpub\wwwroot\mydomain\myadmin\define_pages_editor.php on line 188

However when I edit define_page_3.php which has never been edited at all on the site, I get the original default content. This means that only the edited pages throw the error and not the age-old originals.

I also did a full text search of all files in the cart folder for "utf8" and only found one reference in a js file containing some Japanese language references. Nothing else.

Any further advice is very much appreciated.
Regards
Rob

Regarding the define_pages editor, if you use Notepad++ as your text editor, since you're using the non-utf-8 encoding on your site you should verify that none of your text files are being stored in utf-8 format.

Open one of the define pages that white-screens in the define-pages editor with Notepad++ and click the "Encoding" button in the screen header ... it should show ANSI. If it doesn't, click ANSI and I'm betting that you'll see some "funky" characters in the display. Change those to either spaces or some other place-holder and save the file. Re-upload to your hosted environment and you'll be able to edit with the define-pages editor again.

Thanks again for the reply.

I've installed Notepad++ and examined some files.

This has to be a clue:

As I mentionedearlier, some define-pages have editable content and some are blank. I have two here that have a clear difference:

Homepage - The define_main_page.php is blank in the editor and in Notepad++ status bar it shows as "ANSI". In the encoding menu it has a bullet next to "Encode in ANSI".

Sitemap page - The define_site_map.php is not blank in the editor and in the Notepad++ status bar it comes up as "ANSI as UTF8". In the encoding menu it has a bullet next to "Encode in UTF-8 without BOM".

Next I tried to save define_main_page.php to be the same as define_site_map.php, however Notepadd++ seem to think the file is unchanged when I switch it from "ANSI" to "Encode in UTF-8 without BOM" and doesn't offer it for saving. Forcing a save and then opening the file still shows it as "ANSI".

So the result is:

A file marked as "Encode in UTF-8 without BOM" works
And a file marked as just "ANSI" or even just "UTF8" does not (I saved one as UTF8 just to test).

Does that make any sense?
Regards
ROb

Quick update...

I missed the notepad++ option to conert before save, so I tried again.

When I select "Convert to UTF8 without BOM" in the Notepad++ encoding menu and then save, the define_main_page.php now opens in the editor. In notepad++ it shows as "ANSI as UTF8"

And if I use the same method to "Convert to UTF8", it also now works in the editor.

So it appears the editor is working in UTF8 mode or something.

I have double-checked my config files and language files for the nth time now and there is no trace of "UTF" in any of them, so I don't know how it is running as UTF8?

I've clicked up and down through the db tables and columns and everything in there is latin, no UTF.

I've full-text searched the file system and there is also no trace of utf.

Do you think it would be advisable to convert the db tables and relevant fields to UTF8 as I have seen in various blog and forum posts?

Regards
rob

I found the issue... Don't trust full-text search for a start!

It didn't find the utf8 reference in the site's custom template language folder. I found it after browsing files randomly and search for utf8 in each.

Now the site is back up and all's fine.. thanks for the efforts to help sort this out!
Regards
Rob

The Developer's Tool Kit is not designed to look outside standard Zen Card code/file structure.

Not free but UltraFinder has been a God send since I'm not real familiar with some of the Linux find codes. Works on the hard drive and over FTP.

I actually used the file explorer's own search on the server (Server 2008 R2) for the full text search and it seems odd to me that it didn't find it. It normally does well. I'm wondering if it ignores some filetypes.
I'll check out UltraFinder - I see they have a Windows version as well, thanks
Rob

Hi, Robax

I got same problem. Can you please explain exactly how you resolved this issue?

thank you

I eventually found a reference to UTF8 in an english.php somewhere in a custom template folder. The custom folders are scattered throughout the installation as part of the override template system.

If you have done this:
http://www.zen-cart.com/content.php?313-should-i-convert-my-store-from-iso-8859-1-to-utf8
and confirmed everything else mentioned in this thread, and it still doesn't work, either manually search all language file contents for 'utf8' or use a tool like the one suggested - UltraFinder (or maybe there's a free alternative).

I eventually found mine in Notepad just randomly opening files and searching.

I made the mistake of thinking the full text search facility in Windows file manager would work as one might expect, but it didn't; presumably it was only searching files that it considers to be text and not simply everything. It did find reference in a .js file, but not in the .php

Regards
Rob

UltraFinder has a free trial period.