EasyPopulate 4.0 Support Thread

brittainmark:

I have just got the latest version from git hub and wanted to check if it is now a requirement that the folder in which you keep the uploaded files is under [YOURADMIN] directory.
The reason I ask is that I was having trouble with all the files showing CSV Unknow Delimiter. I tracked the code down to module easypopulate_4.php lines 438-444

$basepath = "";
$realBase = realpath($basepath);
$userpath = $basepath . $file;
$realUserPath = realpath($userpath);
if ($realUserPath === false || strpos($realUserPath, $realBase) !== 0) {
return NULL; // return back to the function with a non-result?
}

> 
> The issue is that 
> $file         /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> $realBase     /MYSERVER/HOME/EASYPOPULATELOADDIR/MYADMIN
> $userpath     /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> $realUserPath /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> 
> does not pass the test unless EasyPopulateUploadDIR is under MYADMIN
> thus $file /MYSERVER/HOME/EASYPOPULATELOADDIR/MYADMIN/EasyPopulateUploadDIR/Myfile.csv
> 
> I can leave the folder under MYADMIN just checking that this is now a requirement, OR if i have set something up wrongly.
Was not intended to become a "requirement". I have a fix for the above issue that retains the expected security. 
It can be found at this commit: [Restore import from catalog side/remove CSV delimiter warning · mc12345678/EasyPopulate-4.0@84331eb (github.com)](https://github.com/mc12345678/EasyPopulate-4.0/commit/84331ebfabc2a56449363dcc7218475baaed841c)

brittainmark:

I have just got the latest version from git hub and wanted to check if it is now a requirement that the folder in which you keep the uploaded files is under [YOURADMIN] directory.
The reason I ask is that I was having trouble with all the files showing CSV Unknow Delimiter. I tracked the code down to module easypopulate_4.php lines 438-444

$basepath = "";
$realBase = realpath($basepath);
$userpath = $basepath . $file;
$realUserPath = realpath($userpath);
if ($realUserPath === false || strpos($realUserPath, $realBase) !== 0) {
return NULL; // return back to the function with a non-result?
}

> 
> The issue is that 
> $file         /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> $realBase     /MYSERVER/HOME/EASYPOPULATELOADDIR/MYADMIN
> $userpath     /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> $realUserPath /MYSERVER/HOME/EASYPOPULATELOADDIR/EasyPopulateUploadDIR/Myfile.csv
> 
> does not pass the test unless EasyPopulateUploadDIR is under MYADMIN
> thus $file /MYSERVER/HOME/EASYPOPULATELOADDIR/MYADMIN/EasyPopulateUploadDIR/Myfile.csv
> 
> I can leave the folder under MYADMIN just checking that this is now a requirement, OR if i have set something up wrongly.

> **mc12345678:**
>
> Was not intended to become a "requirement". I have a fix for the above issue that retains the expected security. 
> It can be found at this commit: [Restore import from catalog side/remove CSV delimiter warning · mc12345678/EasyPopulate-4.0@84331eb (github.com)](https://github.com/mc12345678/EasyPopulate-4.0/commit/84331ebfabc2a56449363dcc7218475baaed841c)

I can't seem to "properly" edit the previous post and it is not a "timeout" issue, the original content just doesn't appear in the editor.

Anyways, the above is to be incorporated into the next version which I'm near completing for forum posting but had some things to work out/on.

Thats great. Thanks. If i find anything else shall i post here or raise issue on git-hub.

brittainmark:

Thats great. Thanks. If i find anything else shall i post here or raise issue on git-hub.

For me to respond, either would work; however, generally in the spirit of the community, I would suggest posting here. If that is then determined better to capture/address in github such can be then be done.

I want to also say that the above posted "solution" considers the admin directory to be only one directory deep away from the "public" folder being used. E.g. if a store were found in https: // DOMAIN / sub-folder / 2nd sub-folder and the file were located at https: // DOMAIN / filename.csv Then I suspect/expect that the CSV unknown delimiter issue would occur... That also said, I was just about to indicate how I might change the message to reflect the condition causing this issue; however, I'm not so sure that I want to offer up that additional piece of info that the file is being retrieved from a location considered to be unauthorized...

The general principle behind the check is to consider the actual file path (location on the server) and in a way to ignore or not use database and/or constant related information when considering likeness.

mc12345678:

For me to respond, either would work; however, generally in the spirit of the community, I would suggest posting here. If that is then determined better to capture/address in github such can be then be done.

I want to also say that the above posted "solution" considers the admin directory to be only one directory deep away from the "public" folder being used. E.g. if a store were found in https: // DOMAIN / sub-folder / 2nd sub-folder and the file were located at https: // DOMAIN / filename.csv Then I suspect/expect that the CSV unknown delimiter issue would occur... That also said, I was just about to indicate how I might change the message to reflect the condition causing this issue; however, I'm not so sure that I want to offer up that additional piece of info that the file is being retrieved from a location considered to be unauthorized...

The general principle behind the check is to consider the actual file path (location on the server) and in a way to ignore or not use database and/or constant related information when considering likeness.

Think I understand now. The "Uploads Directory Admin/Catalog" setting determins where the upload file should be located. I think my issue was that this flag had been reset in the update from false to true. I will reinstall and old version and reupdate to see if it happens again.

Just for completeness "Show all EP4 Filetypes with Files" has true and false with a capital letter, all the others are lower case.

brittainmark:

Think I understand now. The "Uploads Directory Admin/Catalog" setting determins where the upload file should be located. I think my issue was that this flag had been reset in the update from false to true. I will reinstall and old version and reupdate to see if it happens again.

Just for completeness "Show all EP4 Filetypes with Files" has true and false with a capital letter, all the others are lower case.

Sused it. If you include the admin directory in the path of the "Uploads Directory" you set the "Uploads Directory Admin/Catalog" flag to true and remove the admin directory from the path. V clever!

I'll carry on testing.

brittainmark:

Think I understand now. The "Uploads Directory Admin/Catalog" setting determins where the upload file should be located. I think my issue was that this flag had been reset in the update from false to true. I will reinstall and old version and reupdate to see if it happens again.

Just for completeness "Show all EP4 Filetypes with Files" has true and false with a capital letter, all the others are lower case.

The false to true condition is further discussed below. It is possible "upon upgrade" that the software will look in the admin instead of the store side if the previous version did not contain this configuration setting as the default install/upgrade is to add the configuration setting with it set to true by default. Therefore, if one were using the catalog side in the old version (where the setting did not exist), then when the configuration switch is added it is by default set to true and therefore now the software looks in the admin for the folder that was intended to be on the catalog side.

"Show all EP4 Filetypes with Files", I don't see where a capital letter is used for true or false in my search of the extra functions file and the main file. A capital is used for "Hidden", but that appears to be consistently used. If there is a discrepancy in other use of capital/small letter, could you please point me in the direction where that problem exists? "True" != "true" per se... At least it is not intended to be equal.

brittainmark:

Sused it. If you include the admin directory in the path of the "Uploads Directory" you set the "Uploads Directory Admin/Catalog" flag to true and remove the admin directory from the path. V clever!

I'll carry on testing.
So, in an earlier effort to ensure that the admin directory remains as "obscured" as possible and recognizing that a default install of Zen Cart does not populate the database with the admin directory, I sought to remove storage of the admin directory in as "simple" of a way as possible. That is not to say that someone couldn't work around the issue and force the admin directory to be in the database, but I take that to be something that someone with the appropriate access and at that time authority wanted to include the admin directory and that it is on them, not necessarily the software.

Glad it was figured out though.. Thanks for feedback.

mc12345678:

@kernheimer, as you and I know, we have talked outside this forum. I would like to ask a few questions to find an answer.

In admin->Configuration->Easy Populate 4:

What are the settings for:

1. Import/Export Primary Key
2. Import Language Override

The problem described indicates a conflict.
To correct the issue I understand:
If 1. is products_id or blank_new, then 2. should NOT be products_model_only.
If 1. is products_model (historically the default), then 2. should NOT be products_id_only.

The above is for the problem described. The problem described has been, when importing a product file created by export, the system response is "No category provided for this new product".

I have more questions, but in this situation I want to slow down and address one issue at a time. Please continue to have patience.

Dear mc12345678

What are the settings for:
1. Import/Export Primary Key. - Products_model
2. Import Language Override. _ Language_code _only

kernheimer:

Dear mc12345678

What are the settings for:
1. Import/Export Primary Key. - Products_model
2. Import Language Override. _ Language_code _only
Ok, I tried to ask for information about the settings that are involved with this, I missed one setting.... Although I did not ask for one setting, I received information that is/was helpful.

The settings I was really interested in were:

1. Import/Export Primary Key
2. Import Language Override
3. Export Language Identifier

The default settings for these in 4.0.37.13 were:

1. products_model
2. language_code_only
3. id

The problem with some of those defaults (as discussed several times before) is that when importing data that is language dependent, trying to import only fields with the language_code means that the field likely does not exist. Because the specific field does not exist then the data does not exist for the program. Why would the field not exist? Because when exporting the data, the language dependent field(s) only ended with the language_id.

So, to more accurately state the necessary conditions:

If 3. (Export Language Identifier) is id when exporting the file, then 2. (Import Language Override) should NOT be language_code_only.
If 3. (Export Language Identifier) is code when exporting the file, then 2. (Import Language Override) should NOT be language_id_only.

The above is for the problem described. The problem described has been, when importing a product file created by export, the system response is "No category provided for this new product". Why? because, if the code is looking for categories_name_en, but it only contains categories_name_1, then it is true that the category is considered to be missing.

So, in summary, may I suggest changing the setting for Import Language Override to ANYTHING other than language_code_only. Generally speaking, I would suggest changing it to language_id_only to keep things "simple". :)

Hi,

I tried updating the image and description of existing categories using the CategoriesMeta-EP file. The file import was complete and the image and description were inserted. However, there is an error:
"An SQL error has occured. Please check your input data for tabs within fields and delete these. If this error continues, please forward your error log to the Easy Populate maintainer"

The debug error log shows:
MySQLi error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '
categories_id = 10387,
language_id ...' at line 1
When executing:
INSERT INTO zen_meta_tags_categories_description SET ,
categories_id = 10387,
language_id = 1

How can this error be fixed please?
Thanks

waterbender:

Hi,

I tried updating the image and description of existing categories using the CategoriesMeta-EP file. The file import was complete and the image and description were inserted. However, there is an error:
"An SQL error has occured. Please check your input data for tabs within fields and delete these. If this error continues, please forward your error log to the Easy Populate maintainer"

The debug error log shows:
MySQLi error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '
categories_id = 10387,
language_id ...' at line 1
When executing:
INSERT INTO zen_meta_tags_categories_description SET ,
categories_id = 10387,
language_id = 1

How can this error be fixed please?
Thanks
Either of three ways:

1. Since appears that only updating v_categories_name_ related fields, then name the file with the prefix format of "category-ep", it appears though that to update categories_description that for now it should use the "categorymeta-ep" file format and one of the below would apply. I am not 100% sure why the "category-ep" file format never included the categories_description field into the "category-ep" file format. Trying to review to see what if any issues might be expected by including the description in the general category file or a similar file.

2. Add any (or all) of the following three fields to the import file that has the filename format of categorymeta-ep:
   a. v_metatags_title_
   b. v_metatags_keywords_
   c. v_metatags_description_

3. Modify the file admin/includes/modules/easypopulate_4_import_categorymeta_ep.php to incorporate the following code change:

changing lines 165-166 https://github.com/mc12345678/EasyPopulate-4.0/blob/ff24fa7c7d1d4e3d185b3ffa4beb613acd9488bb/admin/includes/modules/easypopulate_4_import_categorymeta_ep.php#L165-L166 from:

              $sql .= ",
                ";

to

              if ($update_count) {
                $sql .= ",
                  ";
              }

Though this last "fix" should also include a notification that effectively nothing is being done as there is no meta data that has been provided, just category name information...

If there remain issues, then it may also be helpful to identify the admin->configuration->Easy Populate 4 settings for:
1. Import Language Override
2. Export Language Identifier

It looks like the problem is resolved when I changed these column titles:
v_categories_name_1
v_category_path_1
v_categories_description_1
v_metatags_title_1
v_metatags_keywords_1
v_metatags_description_1

to

v_categories_name_en
v_category_path_en
v_categories_description_en
v_metatags_title_en
v_metatags_keywords_en
v_metatags_description_en

Another question on importing attributes. Currently, I first use the Attrib-Basic-EP file to import attributes, then use the Attrib-Detailed-EP file to revise attributes that cost extra. However this seems tedious as I have to manually find and match the corresponding products_attributes_id one by one. Is this the only way to it?

waterbender:

It looks like the problem is resolved when I changed these column titles:
v_categories_name_1
v_category_path_1
v_categories_description_1
v_metatags_title_1
v_metatags_keywords_1
v_metatags_description_1

to

v_categories_name_en
v_category_path_en
v_categories_description_en
v_metatags_title_en
v_metatags_keywords_en
v_metatags_description_en
First of all, glad that were able to perform the import.
Second, simply changing the title of the fields became a "one time" correction because it is obvious that 1) the export file was generated using the language_id for export but import was ignoring the language_id and instead was using the language_code... So, please, please, please, review almost any number of pages back about changing the import/export settings so that they match... Please....
Third this appears to have revealed a condition that was to be prevented. Basically when it would not be possible to import content it was to provide a message. The current check for doing that simply validated that at least some field exists even if the current settings wouldn't have imported the content. So thank you for identifying that issue.

waterbender:

Another question on importing attributes. Currently, I first use the Attrib-Basic-EP file to import attributes, then use the Attrib-Detailed-EP file to revise attributes that cost extra. However this seems tedious as I have to manually find and match the corresponding products_attributes_id one by one. Is this the only way to it?

So population of attributes to a product requires at least two "levels" of data population: existence to a product, and detail about that existence.
Perhaps some of that could be done all in one file, but it certainly wouldn't be pretty.

The current process, also described in the instructions, basically has the basic file get populated with product, option name type, option name, and option values associated to that option name.
Export a detailed report.
Update the details of the detailed report
Import the detailed report to refresh the values.

So, I'm not entirely sure what "manual matching" issue is... I say that to get some further information, just because I don't understand doesn't mean it isn't an issue.

On the export of the detailed file, the products_model is included along with some other "human readable" content that should make it easier to locate the row to update. Certainly other data could be added for inclusion such as products_name, though that also shouldn't be necessary if "default settings" are used for operation as products_model should refer to a unique product... (Yes a little tongue in cheek there too, because there are additional settings that allow more freedom in using the software.)

Hi,

What I meant was to find out the corresponding attribute id of the imported product which has an additional cost.
There are many products on the site. When attributes are exported, can we be selective? If not, the file will be very big as the attributes of every single product will get exported.

waterbender:

Hi,

What I meant was to find out the corresponding attribute id of the imported product which has an additional cost.
There are many products on the site. When attributes are exported, can we be selective? If not, the file will be very big as the attributes of every single product will get exported.
Much more understandable. :)

Correct, at the moment it is a full or no basic/detailed attribute export. Incorporating the use of the dropdowns across the top is possible; however, would require some additional code in:
admin/easypopulate_4.php
admin/includes/languages/english/easypopulate_4.php
admin/includes/modules/easypopulate_4_filelayout.php
and
admin/includes/easypopulate_4_export.php

The thought is that in the filelayout, would incorporate an additional "string" to substitute before the "ORDER BY" statement.
In the language file(s), would incorporate the word(s) for the option(s) that would be in the dropdown (considering adding just the two "features" of basic/detailed attributes).
In the base file, adding the new language defines to support dropdown expansion.
In the export file adding the "control" to substitute the string added before the "ORDER BY" statement either with the desired filtering or to just make it blank as would be expected for the "default" export.

So, yes, doable and I'll look to initiate the change/addition of the feature/option. Thank you for clarifying.

Now, last part to that, the above does not add/offer the ability to specifically target just a single product nor the ability to enter in a list of specific/designated product, but would allow selection through existing selection options (category, manufacturer, enabled/disabled, etc...). So a "closer" range to select product, but perhaps not as unique as desired(?).

I'm getting an import error upon exporting the entire catalog and cannot import any changes to the csv file.

Version Easy Populate 4.0.37.13 - 05-03-2021 On 1.5.7
php7.4

wtfbbq:

I'm getting an import error upon exporting the entire catalog and cannot import any changes to the csv file.

Version Easy Populate 4.0.37.13 - 05-03-2021 On 1.5.7
php7.4
Please read over the last 10-15 posts. Feel free to answer the questions asked about settings of some of the specific configuration options.

mc12345678:

Much more understandable. :)

Correct, at the moment it is a full or no basic/detailed attribute export. Incorporating the use of the dropdowns across the top is possible; however, would require some additional code in:
admin/easypopulate_4.php
admin/includes/languages/english/easypopulate_4.php
admin/includes/modules/easypopulate_4_filelayout.php
and
admin/includes/easypopulate_4_export.php

The thought is that in the filelayout, would incorporate an additional "string" to substitute before the "ORDER BY" statement.
In the language file(s), would incorporate the word(s) for the option(s) that would be in the dropdown (considering adding just the two "features" of basic/detailed attributes).
In the base file, adding the new language defines to support dropdown expansion.
In the export file adding the "control" to substitute the string added before the "ORDER BY" statement either with the desired filtering or to just make it blank as would be expected for the "default" export.

So, yes, doable and I'll look to initiate the change/addition of the feature/option. Thank you for clarifying.

Now, last part to that, the above does not add/offer the ability to specifically target just a single product nor the ability to enter in a list of specific/designated product, but would allow selection through existing selection options (category, manufacturer, enabled/disabled, etc...). So a "closer" range to select product, but perhaps not as unique as desired(?).

Almost have it. Forgot that the attribute exports don't use the products to categories table so having to generate the product list based on products in categories. I plan to create two export options for each basic and detailed to support of linked product and the second to exclude linked product.