EasyPopulate 4.0 Support Thread

[eranariel]

Zen Cart 1.5.6 & Easy Populate 4.0.36.ZC bug uploading non latin UTF-8 characters


Multilingual with English and Hebrew characters:
1. I am able to download a CSV file in which characters in both languages appear correctly
2. On the upload of the CSV file, the English characters are correctly updated and added to the database , HOWEVER with Hebrew, text strings are cut short after the following two characters: "ג" (Gimmel) and "ע" (Ayn). The Hebrew language is RTL or written right to left.

If I have a product name "עגביות מספר 1" (tomato number 1), all of the text to the left of the character "ע" is omitted once uploaded. The same applies to category names.

I have been looking at the code easypopulate_4_import.php, and it contains reference to the mb_split function, however it isn't clear why only categories names to mention that the problem I am having persists on category names as well. Could the utf-8 conversion of smart-quotes, em-dash, and ellipsis be interfering in some manner?

I have contributed the Hebrew language packages a while ago that I intend to update shortly.


Thanks in advance,

Eran Ariel

[mc12345678]

Been trying to think about this. Need to look at the language pack to see its settings as relates to the database along with the returned data associated with the plugin. Note that generally speaking, as written mb_split and the other mb_ functions only really comes into play when utf8 is involved.

Would need to trace down execution about what happens to the imported text to see what operation is truncating it. I have seen that Russian also has had a similar issue.

Does the same problem occur in the product description or is it only in the product and category names?

[eranariel]

The problem occurs in both, the product and category names. Here is some code from the easy populate import file:

foreach ($langcode as $lang) {
if (!function_exists('mb_split')) {
// iso-8859-1
$categories_names_array[$lang['id']] = explode($categories_delimiter,$items[$filelayout['v_categories_name_'.$lang['id']]]);
} else {
// utf-8
$categories_names_array[$lang['id']] = mb_split(preg_quote($categories_delimiter), $items[$filelayout['v_categories_name_' . $lang['id']]]);
}

[mc12345678]

The problem occurs in both, the product and category names. Here is some code from the easy populate import file:

foreach ($langcode as $lang) {
if (!function_exists('mb_split')) {
// iso-8859-1
$categories_names_array[$lang['id']] = explode($categories_delimiter,$items[$filelayout['v_categories_name_'.$lang['id']]]);
} else {
// utf-8
$categories_names_array[$lang['id']] = mb_split(preg_quote($categories_delimiter), $items[$filelayout['v_categories_name_' . $lang['id']]]);
}

The question also asked about other fields such as products_description, is that field also affected?

Since you already have the language installed and having this issue, perhaps you could run a few tests for me.

Could you modify a portion of the above code to the following:

            } else {
            // utf-8
              $categories_names_array[$lang['id']] = mb_split(preg_quote($categories_delimiter), $items[$filelayout['v_categories_name_' . $lang['id']]]);
trigger_error('category: ' . $items[$filelayout['v_categories_name_' . $lang['id']]] . ' category array: ' . print_r($categories_names_array[$lang['id']], true), E_USER_WARNING);
            }

Then import at least one line of product information that contain the character(s) involved.
After import check the logs directory for a myDebug-adm-xxxx file generated at about the time of attempting to import.
Post the contents of the file, but omit or rename the admin directory (can use admin as the folder name).

Expectation if all is correct on the server is information related to the category name. If nothing is generated, then that too is a piece of useful information and will have to work backwards a little.
Note when posting the contents, please use [CODE][/CODE] tags around the contents. The tags can be generated by pressing the # button in the message box toolbar when responding.

[eranariel]

Yes, I will try...and thank you

[eranariel]

And yes, the question relates to all fields, however this is the only reference in the respective file that relates to the mb_split function.

[eranariel]

OK, I tested with the edited code on one product, which didn't upload to its respective category. Below is the log file:

[19-Feb-2019 21:20:10 America/Chicago] Request URI: /admin/easypopulate_4.php, IP address: 67.86.152.229
#1  fgetcsv() called at [public_html/admin/easypopulate_4_import.php:44]
#2  include_once(/public_html/admin/easypopulate_4_import.php) called at [/public_html/admin/easypopulate_4.php:289]
--> PHP Warning: fgetcsv() expects parameter 1 to be resource, null given in /public_html/admin/easypopulate_4_import.php on line 44.

[mc12345678]

OK, I tested with the edited code on one product, which didn't upload to its respective category. Below is the log file:

[19-Feb-2019 21:20:10 America/Chicago] Request URI: /admin/easypopulate_4.php, IP address: 67.86.152.229
#1  fgetcsv() called at [public_html/admin/easypopulate_4_import.php:44]
#2  include_once(/public_html/admin/easypopulate_4_import.php) called at [/public_html/admin/easypopulate_4.php:289]
--> PHP Warning: fgetcsv() expects parameter 1 to be resource, null given in /public_html/admin/easypopulate_4_import.php on line 44.

That appears to be caused by a an issue with the file. Specifically that is expected to be the case if the file is considered not there, specifically can occur with 4.0.36.ZC if the filename would be sanitized by strict sanitization by the ZC software because the appropriate sanitizer code has not yet been incorporated to the distribution.

So, I recommend using a filename that is very similar to what was provided on export, no spaces, no special characters. Afterall, this issue just reported appears different than what we are trying to work.

[shags38]

v1.5.5f php 7.1

I have the same products on different sites (.com and ccTLD) - the only difference is the description varies for each site, all else is the same (part number, image file etc.).

Question: can I do a full EP export from the master site, then delete all columns except for part number, quantity and status (in stock/out of stock) and then import that modified file into a second site such that it modifies/updates ONLY the Quantity and Status of a Part Number and maintains everything else in the database as it is/was?

cheers, Mike

[mc12345678]

v1.5.5f php 7.1

I have the same products on different sites (.com and ccTLD) - the only difference is the description varies for each site, all else is the same (part number, image file etc.).

Question: can I do a full EP export from the master site, then delete all columns except for part number, quantity and status (in stock/out of stock) and then import that modified file into a second site such that it modifies/updates ONLY the Quantity and Status of a Part Number and maintains everything else in the database as it is/was?

cheers, Mike

If by part number you mean products_model or products_id where whichever is used is unique (different for every product) then yes.

Where two rows of data have the same master key (products_model or products_id by default), the last one that is discovered will become the current information for all product that have that same master key.