EasyPopulate 4.0 Support Thread

[mc12345678]

I'm getting an import error upon exporting the entire catalog and cannot import any changes to the csv file.

Version Easy Populate 4.0.37.13 - 05-03-2021 On 1.5.7
php7.4

Please read over the last 10-15 posts. Feel free to answer the questions asked about settings of some of the specific configuration options.

[mc12345678]

Much more understandable. :)

Correct, at the moment it is a full or no basic/detailed attribute export. Incorporating the use of the dropdowns across the top is possible; however, would require some additional code in:
admin/easypopulate_4.php
admin/includes/languages/english/easypopulate_4.php
admin/includes/modules/easypopulate_4_filelayout.php
and
admin/includes/easypopulate_4_export.php

The thought is that in the filelayout, would incorporate an additional "string" to substitute before the "ORDER BY" statement.
In the language file(s), would incorporate the word(s) for the option(s) that would be in the dropdown (considering adding just the two "features" of basic/detailed attributes).
In the base file, adding the new language defines to support dropdown expansion.
In the export file adding the "control" to substitute the string added before the "ORDER BY" statement either with the desired filtering or to just make it blank as would be expected for the "default" export.

So, yes, doable and I'll look to initiate the change/addition of the feature/option. Thank you for clarifying.

Now, last part to that, the above does not add/offer the ability to specifically target just a single product nor the ability to enter in a list of specific/designated product, but would allow selection through existing selection options (category, manufacturer, enabled/disabled, etc...). So a "closer" range to select product, but perhaps not as unique as desired(?).

Almost have it. Forgot that the attribute exports don't use the products to categories table so having to generate the product list based on products in categories. I plan to create two export options for each basic and detailed to support of linked product and the second to exclude linked product.

[wtfbbq]

The solution to the import error in place of the import button was the configuration option to store the eptemp folder outside of admin is the cause of the problem.
This leads to another question. Why is setting the eptemp folder outside the admin folder causing this import error?

[mc12345678]

The solution to the import error in place of the import button was the configuration option to store the eptemp folder outside of admin is the cause of the problem.
This leads to another question. Why is setting the eptemp folder outside the admin folder causing this import error?

Understanding that English is not the stronger language, I will try to answer what I may understand.

A problem was created in a code change where placing the EP4 upload/download folder outside of the admin directory would cause the files identified in that folder to not be processed for import by EP4. The file would be identified as having a CSV delimiter error. Although a solution was only recently publicly posted, the solution had been determined a while ago awaiting someone else to confirm that there was an issue and request resolution (see ). That solution appears to have worked.

I am not sure if the question above is now answered by the above text, or if a more technical explanation is requested about why the code works the way it does, so I am going to step out and attempt an explanation.

To attempt to limit the ability of database settings from pointing to folder locations that are not expected to represent "safe" spaces in the Zen Cart software, a method is incorporated in the software to report/determine the file's location that is executing the code. That location is then compared to the path of the file to be accessed. If the two sets of information align then the path is used, if the requested path is not in the file's path or downstream from it (in a sub-folder or some sub-sub-folder), then the path is not trusted. But, a folder that is placed outside of the admin folder would need to be at least one folder to the left of the current folder. Therefore to allow that path, additional effort/code is required to permit the parent folder to be allowed.

So why not just have the folder always point to the parent folder? Not everyone sets up their admin folder to only be a sub-folder off of the catalog, there has been discussion of removing the admin folder, *and* I consider the "safer" option for all CSV data to be imported and exported from within a secure area such as the admin folder. Therefore, I wrote the code to default running from the admin directory instead of from one directory above the current folder and allow any folder off of the catalog root to be permitted.

Otherwise, because I made a mistake. :)

[mc12345678]

Understanding that English is not the stronger language, I will try to answer what I may understand.

A problem was created in a code change where placing the EP4 upload/download folder outside of the admin directory would cause the files identified in that folder to not be processed for import by EP4. The file would be identified as having a CSV delimiter error. Although a solution was only recently publicly posted, the solution had been determined a while ago awaiting someone else to confirm that there was an issue and request resolution (see https://www.zen-cart.com/showthread....80#post1385080). That solution appears to have worked.

I am not sure if the question above is now answered by the above text, or if a more technical explanation is requested about why the code works the way it does, so I am going to step out and attempt an explanation.

To attempt to limit the ability of database settings from pointing to folder locations that are not expected to represent "safe" spaces in the Zen Cart software, a method is incorporated in the software to report/determine the file's location that is executing the code. That location is then compared to the path of the file to be accessed. If the two sets of information align then the path is used, if the requested path is not in the file's path or downstream from it (in a sub-folder or some sub-sub-folder), then the path is not trusted. But, a folder that is placed outside of the admin folder would need to be at least one folder to the left of the current folder. Therefore to allow that path, additional effort/code is required to permit the parent folder to be allowed.

So why not just have the folder always point to the parent folder? Not everyone sets up their admin folder to only be a sub-folder off of the catalog, there has been discussion of removing the admin folder, *and* I consider the "safer" option for all CSV data to be imported and exported from within a secure area such as the admin folder. Therefore, I wrote the code to default running from the admin directory instead of from one directory above the current folder and allow any folder off of the catalog root to be permitted.

Otherwise, because I made a mistake. :)

The basic idea of the process is: if the database is modified by a bad actor, the worst that they can do/access is a file within the ZC catalog but not above it. In so accessing they would have to modify the file that is performing execution in order to affect something else with the idea that the user has placed their files in some sub-folder. Further, the admin directory itself is not stored in the database so that it can not be discovered by database review/inspection. While there is not a requirement right now for csv files to be in a sub-folder, I likely will incorporate such a requirement in a future release based on this evaluation.

[waterbender]

Almost have it. Forgot that the attribute exports don't use the products to categories table so having to generate the product list based on products in categories. I plan to create two export options for each basic and detailed to support of linked product and the second to exclude linked product.

Great thanks. Looking forward to the newer version with this feature that can selectively export a range of attributes (either by categories, manufacturers or date if possible).

[mc12345678]

Great thanks. Looking forward to the newer version with this feature that can selectively export a range of attributes (either by categories, manufacturers or date if possible).

Current plan is to support combination of categories and manufacturers through the existing dropdowns at the top of the EP4 admin screen. Not planning on addressing dates (newly added request) because of what I perceive as too many options to consider and dates to also address. Not to say that it couldn't be done. At the moment I see it as too much effort, likely too confusing for the user, with little in way of overall returns.

As to the category "issue" came across another wrinkle that am trying to ensure is addressed sufficiently. ZC 1.5.7 and older generated results in one format for a function, but upcoming ZC 1.5.8 will use a different response when working via the admin side. I want to "set it and forget it" by allowing a single body of code to function in both environments. So working through implementing the logic to address and for it to not add significant complexity.

[mc12345678]

Current plan is to support combination of categories and manufacturers through the existing dropdowns at the top of the EP4 admin screen. Not planning on addressing dates (newly added request) because of what I perceive as too many options to consider and dates to also address. Not to say that it couldn't be done. At the moment I see it as too much effort, likely too confusing for the user, with little in way of overall returns.

As to the category "issue" came across another wrinkle that am trying to ensure is addressed sufficiently. ZC 1.5.7 and older generated results in one format for a function, but upcoming ZC 1.5.8 will use a different response when working via the admin side. I want to "set it and forget it" by allowing a single body of code to function in both environments. So working through implementing the logic to address and for it to not add significant complexity.

Further status update. After further consideration of the "configuration" of the associated function across at least those two versions of Zen Cart, I decided to draw the function into EP4 instead of using the Zen Cart provided "versions". The reasoning is this:
in earlier versions, an array is generated where the key to the array represents the "numbered" item in the list (5 product, the last key would be 5) and the value of the array represented the products_id. In the upcoming version, the key to the array represents the products_id and the value of the array represents the category path. So I thought I had a good system developed to be able to evaluate the array results and make a determination off of that as to which result was being provided. In general this would work when evaluating the differences of the below to arrays:
$myArrayOld[1] = 7
$myArrayOld[2] = 5
versus
$myArrayNew[7] = 6
$myArrayNew[5] = 3

Using the above, I could sort the array by key, then evaluate the number of array items and if the key of the last array item did not equal the number of items, then I was using the "new" system and could "withdraw" the products_id for processing. If the last key was equal to the number of array items, then I would use the values for products_id.

But... In the boundary condition where the products_id of the first few products (1 and 2 for example) were being returned, then:
$myArrayOld[1] = 1
$myArrayOld[2] = 2
versus
$myArrayNew[1] = 8
$myArrayNew[2] = 9

Would mistakenly in a new system attempt to process the array with consideration of the right side (value) being the products_id, although that value side (the result) is actually the category path of category 8 followed by category 9 instead of products_id 1 followed by 2.

A separate consideration was to attempt to identify where the associated function was "stored" with the old version being in the admin directory and the new version in the catalog. But, then modifications that one *may* have made to their site could increase the needed assistance (much like the issue generated by the current defaults for import/export of language related fields). So, I don't of course like that concept. Then there was the consideration of attempting to identify the number of parameters of the associated function, same issue of support *and* though I had a process developed to internally adjust for an earlier catalog side code discrepancy that is corrected for ZC 1.5.8, if one doesn't incorporate a specific change (missed it or didn't want to apply it for some reason), again additional support assistance would become necessary.

So... As a result, I have incorporated the ZC 1.5.8 version of the applicable function directly into the easypopulate_4_export.php file giving it a specific ep4 related name so that the code is otherwise written more simply. I considered adding it to the extra_functions file, but really I am looking to move away from that file and instead implement that code into an ep4 class instead so that the code would only load while using ep4 or at an on call basis.

I know, more than you wanted to know @waterbender, but I wanted to at least publish somewhere "public" why the changes I'm making were made the way they were. I would expect that as time wears on and the code requires significant change to support operation, that any consideration of using it on a version less than 1.5.8 will mean that could revert back to the Zen Cart core code, but as long as this code is supported on a ZC version before 1.5.8, it seems the best course of action for the code is to use the ep4 version of the function.

[chuckrey]

Hi, I apologize if this has been answered but I cannot locate the question in this thread. EP4 for 1.5.7 is working just great for me with ONE exception. When downloading the Basic Attributes, I am seeing only 1 line when I have many, many more lines of data. I attempted to split thinking the size of the file may have been an issue, but it's nowhere near that big. Is there a setting or known solution to basic attributes not displaying all lines of data? Thanks so much and once again, I apologize if this was addressed elsewhere. Thank you!

[mc12345678]

Hi, I apologize if this has been answered but I cannot locate the question in this thread. EP4 for 1.5.7 is working just great for me with ONE exception. When downloading the Basic Attributes, I am seeing only 1 line when I have many, many more lines of data. I attempted to split thinking the size of the file may have been an issue, but it's nowhere near that big. Is there a setting or known solution to basic attributes not displaying all lines of data? Thanks so much and once again, I apologize if this was addressed elsewhere. Thank you!

So, I've only seen a "single" line of data when either there are no attributes assigned or there is a possibility that the method of defining the attributes to the product doesn't ensure a record in each of the tables: products_attributes (table a), products (table p), products_options (table o), and products_options_values (table v) and further that those tables are related by: a.products_id = p.products_id AND a.options_id = o.products_options_id AND a.options_values_id = v.products_options_values_id AND o.language_id in = v.language_id.
Those tables could possibly instead be referenced by left joins, but there may also be undesirable data found when doing that on the export...

Basically, there may be an issue with the data in the database, but I have done a comparison of the v4.0.37.13 software to what I am currently working on as an upgrade where my upgrade works (and a similar issue hadn't previously been identified) and haven't found anything that specifically limits the number of responses/records. The query that generates the list to be exported is:

SELECT
      a.products_attributes_id            as v_products_attributes_id,
      a.products_id                       as v_products_id,
      a.options_id                        as v_options_id,
      a.options_values_id                 as v_options_values_id,
      p.products_model            as v_products_model,
      o.products_options_id               as v_products_options_id,
      o.products_options_name             as v_products_options_name,
      o.products_options_type             as v_products_options_type,
      v.products_options_values_id        as v_products_options_values_id,
      v.products_options_values_name      as v_products_options_values_name,
      v.language_id                       as v_language_id
      FROM products_attributes as a, products as p, products_options as o, products_options_values as v
      WHERE
      a.products_id       = p.products_id AND
      a.options_id        = o.products_options_id AND
      a.options_values_id = v.products_options_values_id AND
      o.language_id       = v.language_id ORDER BY a.products_id, a.options_id, v.language_id, v.products_options_values_id

If that is only resulting in a single row and there are multiple product with attributes, then I would think there is something "wrong" (different than expected) with the data. The data may be handled differently and therefore this sql may need to be adjusted to accommodate ideally with the same fields being returned.