Totally Zenned
- Join Date:
- Jul 2012
- Posts:
- 16,908
- Plugin Contributions:
- 2
EasyPopulate 4.0 Support Thread
Feznizzle:
Thanks, MC!
Would these changes do the trick?
if ($ep_uses_mysqli) {
$collation = mysqli_character_set_name($db->link); // should be either latin1 or utf8
} else {
$collation = mysql_client_encoding(); // should be either latin1 or utf8
}
if ($collation == 'latin1') {
if (function_exists('mb_internal_encoding')) {
mb_internal_encoding("iso-8859-1");
}
}
if (($collation == 'latin1') && ((substr($project, 0, 5) == "1.3.8") || (substr($project, 0, 5) == "[B]1.3.9[/B]"))) {
//mb_internal_encoding("iso-8859-1");
$category_strlen_max = $category_strlen_max / 3;
$categories_name_max_len = $categories_name_max_len / 3;
$manufacturers_name_max_len = $manufacturers_name_max_len / 3;
$products_model_max_len = $products_model_max_len / 3;
$products_name_max_len = $products_name_max_len / 3;
$products_url_max_len = $products_url_max_len / 3;
$artists_name_max_len = $artists_name_max_len / 3;
$record_company_name_max_len = $record_company_name_max_len / 3;
$music_genre_name_max_len = $music_genre_name_max_len / 3;
$zco_notifier->notify('EP4_COLLATION_UTF8_ZC13X');
}
>
> Also, the reason I highlighted the reference to **1.3.9** is that the site I'm working on is a 1.3.9**h**. Should I change that as well?
>
> Thanks again!
Couple of things. Let's start with what looks like a little bit of a history lesson on ZC (based on some independent review of old ZC code and system response to by a ZC 1.5.1 site) and why some of the above code is present. It would appear that back "in the day" (ie. before ZC 1.5.0), that the includes/configure.php and admin/includes/configure.php did not include a DB_CHARSET setting. Therefore part of the interface with the database was to respond/identify whatever characterset was otherwise associated or set in the database itself. A little bit about that though means that if the database responded that it was using UTF-8, then the maximum field length report for the associated fields would be reported related to UTF-8 sizing but expect that the software wouldn't be able to properly encode the characterset back into "multibyte" related utf8 (or in other words identify that there are more "letters" available than what the field would actually be able to take, basically reporting back the size of the field to be 3 times what it would accept). Now most of this was already written before I came along and looking back in the history these changes were incorporated ever more to maintain support for older systems while continuing to develop or provide for newer systems. The test for 1.3.9 looks at the first 5 characters of the $project variable.. Regardless of which sub-version of ZC 1.3.9, the first five characters are 1.3.9, so there is no change needed in that particular portion of the code.
Basically all that to say, that in the second part of the code, you wouldn't want to reduce the size of the available fields if the $collation was identified as latin1, therefore that section would remain unchanged.
A "bigger" problem though in using the more recent software is that it appears that there is a response indicating that the database may be responding back with utf8... And that is causing the "conversion" that is occurring of unusual characters in the csv file (when viewed in iso-8859-1 format)... If the problem is visible that soon in, then I have a few recommendations to "test" or check, but yes generally it looks like changing the code in this area is all that is necessary. (There may be an issue with the character used for the category name separator, but one step at a time).
So, here's what I suggest first:
if ($ep_uses_mysqli) {
$collation = mysqli_character_set_name($db->link); // should be either latin1 or utf8
} else {
$collation = mysql_client_encoding(); // should be either latin1 or utf8
}
echo 'collation: ' . $collation . "\n";
if ($collation == 'utf8') {
if (function_exists('mb_internal_encoding')) {
mb_internal_encoding("UTF-8");
}
}
if ($collation == 'latin1') {
if (function_exists('mb_internal_encoding')) {
mb_internal_encoding("iso-8859-1");
}
}
if (($collation == 'utf8') && ((substr($project, 0, 5) == "1.3.8") || (substr($project, 0, 5) == "1.3.9"))) {
//mb_internal_encoding("UTF-8");
$category_strlen_max = $category_strlen_max / 3;
$categories_name_max_len = $categories_name_max_len / 3;
$manufacturers_name_max_len = $manufacturers_name_max_len / 3;
$products_model_max_len = $products_model_max_len / 3;
$products_name_max_len = $products_name_max_len / 3;
$products_url_max_len = $products_url_max_len / 3;
$artists_name_max_len = $artists_name_max_len / 3;
$record_company_name_max_len = $record_company_name_max_len / 3;
$music_genre_name_max_len = $music_genre_name_max_len / 3;
$zco_notifier->notify('EP4_COLLATION_UTF8_ZC13X');
}
The first "addition" is present to see what the actual returned value is, because in order to accomplish what is desired it may be necessary to not let EP4 think that it should be able to communicate in utf8, but instead push/force latin1 style formatting as a result of the observed outcome. There are other factors to review/confirm as well in the setup of the store and associated collation information. Basically EP4 expects the store to be setup correctly.
After making the first above change, when viewing the EP4 screen you should see text above the standard ZC admin screen indicating either utf8 or latin1. I am expecting that it says latin1 based on all of the discussion.
Can you also indicate what is displayed on the EP4 admin screen associated with the screen item: 'Internal Character Encoding:'?
Does it provide a result or does it say something like: ''mb_internal_encoding not available"?
The next thing to do after reporting all of that information would be to attempt to export, review the data.
When successful, export something "small" like a single category (disabled with disabled product?) that may have a single product with a really long product name, then without opening the file, attempt to upload that one product back and see if there are length/truncation result(s) reported (not expected). Then also look at the product itself and see if/what got modified about it (Should be nothing). Again export the product, compare the two files to see what if any differences now exist (again expect nothing).
Then attempt to open a copy of one of the files, turn around and save the file, again remembering the encoding.
Upload and then import the file (should be no changes on site).
Again export and compare that export with one of the previous (again no differences expected).
You're in an "opportune" position to be able to test/validate the operation because of the setup involved.
