EasyPopulate 4.0 Support Thread

[reetp]

Oh FFS ^&((*"$&()£&%*(^&"$*(^""$( swear bang crash.

Sorry - I have been using the same format for some time - probably 4 years - and just had not realised that it had changed. Thank you pointing it out.

I had read the 'README.txt' but must have glazed over the filename - when you look at it on github, the end of the line is missed off so I only saw :

"5) File names act as a switch inside the script for Importing. Namely: PriceBreaks-EP, CategoryMeta-EP, Featured-EP, SBA-S"

So I wrongly assumed that it was the same rule as before and that so long as the file name CONTAINED this it would be OK.

I even started debugging and saw that it picked up the file as an import file and didn't see any other errors whilst it was processing.

Can I make a suggestion that there should be some form of sanity checking on the filename ? As far as I could see it all went through OK and reported that it had succeeded in the import, which obviously is a complete lie.

If I can figure where to do it I'll try and do a patch myself.

Thanks for the rapid response.

B. Rgds
John

[mc12345678]

Yosameti Sam at his worst. :)

I pulled that from the bottom of the github screen.

What I see can be done, though the code structure doesn't necessarily support such a clear separation of function is to provide notification on import of which "filetype" is being processed.

Alternatively and probably more user friendly/understandable is to have in the admin panel different "windows" ssociated with each filetype by name... An example of the expected filenme prefix shown.

That I think would take things the step further that would help those that have not read the instructions.

Btw, if 5 waas the last one to see/read, then there is a lot to go. The file is included in the download so should be able to read it from desktop.

[reetp]

LOL :-)

I think I start to understand.

It all starts to go wrong at this point in the import file :

1407 if ( strtolower(substr($file['name'],0,14)) == "pricebreaks-ep") {

First, that is probably a change from the version I was using and does indeed only check the START of the filename string whereas before it parsed the whole string probably something like this :

$filename = "test-file-pricebreaks-EP-myname.csv";

if (stristr($filename,"pricebreaks-ep")) {
echo "We found it";
}

That is case insensitive and find the filename anywhere in the string.

Second is that that if it FAILS that test, it jumps through to 1459 and carries on regardless... arrrrggghhhh !!!!!!!! That means it misses some vital bits, but still appears to work.

That is really bad..... it ought to gracefully die and notify the user that 'Filename is incorrect" when it fails the file check (and that occurs in several places). I'm no php coder, but it ain't rocket science....

Not sure how you want to fix it, but it is badly broken :-)

Happy to test anything if you want to try and resolve it.

B. Rgds
John

[mc12345678]

So, in a way I beg to differ that it is broken... That is akin to renaming a zip file of a text document to have an extension of .txt, then trying to open the document in a standard text editor expecting to be able to read the text as originally written.

File opens fine, there is some content provided, but it all did not work as "expected"...

EP4 was written to offer some greater flexibility, assuming that the operator(s) followed the instructions...

Sure, the filenaming could check for presence of words, but then it will be that the chosen text is "wrong", then ever more users will have to change their filename convention after that change.

Okay, all that aside, with the current filenme convention, when a directory is sorted by filename, each file "type" remains together and actully is sorted by date, so "restoration" of a state is easily identifiable. Then there is the saving of a modified file, go to the end of the filename, and do what is desired with it. If the files are sorted by modified date, the the result similar to prefixing the file with a date is obtained, identifying what file was changed and when. Presumably shortly after the change, the file is uploaded to effect the change. An added benefit of the "loose" action within the code is that fields that are not routinely a part of one of the special cases can also be included and will be processed. There are though some things that are relatively specific to the special name mostly requiring different queries compared to the product/product description.

Lastly, the code is modifiable to suit unique desires/needs. The code suggested could be individually incorporated, but in the grand ease of use and instruction, to me I struggle to see how that would provide an overall improvement.

[reetp]

No, it is much simpler than that.

Your analogy is wrong - it is like opening a zip with a text editor, the editor allowing you to open the file, and edit it...... without throwing an error !

If a decision process runs and does not complete correctly for whatever reason then an error should be thrown advising the user. The assumption is the user is dumb (that's me !) and everything you can do to stop them making stupid errors should be encouraged - hence a million different error checks throughout Zencart.

In this instance the filename was incorrect which caused the program to carry on and report that everything was OK when clearly it wasn't.... (the data was right - just the filename being wrong)

That cost me a day of running around based on information reported by the process that was clearly wrong. It said it worked, when clearly it had not. That isn't right.

I'm not trying to be negative - I'd just like to see the code improved to the benefit of all as it is an indispensable tool for Zencart.

Here's a patch to check the filenames and log an error - it may not be perfect but it is about there - also miles quicker than running through all the code for a simple failure :-)

You can see my comment line - sometimes it checks for the filename, others it does a anything else check with <>

Either way, if the name is checked at the start then it gives a fast response to the problem. If you have new sections / filenames then just add the name to the array. Using stristr would give you more flexibility as per my note, but I have left as is for now.

It's also dead simple.....

--- easypopulate_4_import-orig.php 2015-06-18 11:18:05.000000000 +0200
+++ easypopulate_4_import.php 2015-06-18 14:00:21.000000000 +0200
@@ -8,6 +8,37 @@

$file = array('name' => $_GET['import']);
$display_output .= sprintf(EASYPOPULATE_4_DISPLAY_LOCAL_FILE_SPEC, $file['name']);
+
+
+
+/*
+ * These are the filenames that we look for
+ *
+strtolower(substr($file['name'],0,11)) == "featured-ep")
+strtolower(substr($file['name'],0,15)) == "attrib-basic-ep"
+strtolower(substr($file['name'],0,18)) == "attrib-detailed-ep")
+strtolower(substr($file['name'],0,15)) == "sba-detailed-ep")
+strtolower(substr($file['name'],0,12)) == "sba-stock-ep")
+strtolower(substr($file['name'],0,15)) <> "categorymeta-ep")
+strtolower(substr($file['name'],0,7)) <> "attrib-")
+strtolower(substr($file['name'],0,4)) <> "sba-")
+strtolower(substr($file['name'],0,14)) == "pricebreaks-ep")
+*/
+
+
+$fileNameArray = array ("featured-ep", "attrib-basic-ep", "attrib-detailed-ep", "sba-detailed-ep", "categorymeta-ep", "attrib-", "sba-", "pricebreaks-ep");
+$discoveredFile = false;
+
+foreach ($fileNameArray as $key) {
+
+ $length = strlen ($key);
+ // probably better to do stristr but all the lines then need changing further down
+ if (strtolower(substr($file['name'],0,$length)) == $key) {
+ $discoveredFile = true;
+ }
+}
+
+if ($discoveredFile == true) { // we'll proceed

$ep_update_count = 0; // product records updated
$ep_import_count = 0; // new products records imported
@@ -1637,5 +1668,17 @@
} else {
$messageStack->add("File Import Completed.", 'success');
}
+}
+else {
+ $display_output .= '<h3>Finished Processing Import File</h3>';
+ $display_output .= '<br /><h3 style="color:red">Filename is wrong</h3> ';
+ $display_output .= '<br />Please check that the filename <b>starts</b> with the correct term';
+ $display_output .= '<br />The following are the permissible names :';
+ foreach ($fileNameArray as $key) {
+ $display_output .= "<br />" . $key;
+ }
+ $messageStack->add("File Import Failed.", 'error');
+
+} // End file check loop
} // END FILE UPLOADS
-?>
\ No newline at end of file
+?>


And a small patch to easypopulate.php itself to clear a small warning that was annoying me :-)

--- easypopulate_4-orig.php 2015-06-17 19:03:42.000000000 +0200
+++ easypopulate_4.php 2015-06-18 14:47:28.000000000 +0200
@@ -28,9 +28,9 @@
$ep_music = (int)EASYPOPULATE_4_CONFIG_MUSIC_DATA; // 0-Disable, 1-Enable
$ep_uses_mysqli = (PROJECT_VERSION_MAJOR > '1' || PROJECT_VERSION_MINOR >= '5.3' ? true : false--- easypopulate_4-orig.php 2015-06-17 19:03:42.000000000 +0200
+++ easypopulate_4.php 2015-06-18 14:14:01.000000000 +0200
@@ -30,7 +30,7 @@

@set_time_limit($ep_execution); // executin limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit

-if (!$error) {
+if (!isset($error)) {
$upload_max_filesize=ini_get("upload_max_filesize");
if (preg_match("/([0-9]+)K/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024;
if (preg_match("/([0-9]+)M/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024*1024;);

-@set_time_limit($ep_execution); // executin limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit
+@set_time_limit($ep_execution); // execution limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit

-if (!$error) {
+if (!isset($error)) {
$upload_max_filesize=ini_get("upload_max_filesize");
if (preg_match("/([0-9]+)K/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024;
if (preg_match("/([0-9]+)M/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024*1024;


I'd have a go on github but not very experienced with it.... and I am not sure which repo to follow - is chaddro still working on this ?

B. Rgds
John

[ranger_lp]

Did anyone ever come up with a fix for this being displayed on the bottom of the page?


. www.evoluted.net Layout: Manny . www.tenka.co.uk REQUIREMENTS ============ This script requires PHP and GD2 if you wish to use the thumbnail functionality. INSTRUCTIONS ============ 1) Unzip all files 2) Edit this file, making sure everything is setup as required. 3) Upload to server 4) ?????? 5) Profit! CONFIGURATION ============= Edit the variables in this section to make the script work as you require. Start Directory - To list the files contained within the current directory enter '.', otherwise enter the path to the directory you wish to list. The path must be relative to the current directory. */ $startdir = DIR_FS_CATALOG . $tempdir; /* Show Thumbnails? - Set to true if you wish to use the scripts auto-thumbnail generation capabilities. This requires that GD2 is installed. */ $showthumbnails = true; /* Show Directories - Do you want to make subdirectories available? If not set this to false */ $showdirs = true; /* Force downloads - Do you want to force people to download the files rather than viewing them in their browser? */ $forcedownloads = false; /* Hide Files - If you wish to hide certain files or directories then enter their details here. The values entered are matched against the file/directory names. If any part of the name matches what is entered below then it is now shown. */ $hide = array( 'dlf', 'index.php', 'Thumbs', '.htaccess', '.htpasswd', 'fileList.php' ); /* Show index files - if an index file is found in a directory to you want to display that rather than the listing output from this script? */ $displayindex = false; /* Allow uploads? - If enabled users will be able to upload files to any viewable directory. You should really only enable this if the area this script is in is already password protected. */ $allowuploads = false; /* Overwrite files - If a user uploads a file with the same name as an existing file do you want the existing file to be overwritten? */ $overwrite = false; /* Index files - The follow array contains all the index files that will be used if $displayindex (above) is set to true. Feel free to add, delete or alter these */ $indexfiles = array ( 'index.html', 'index.htm', 'default.htm', 'default.html' ); /* File Icons - If you want to add your own special file icons use this section below. Each entry relates to the extension of the given file, in the form => . These files must be located within the dlf directory. */ $filetypes = array ( 'png' => 'jpg.gif', 'jpeg' => 'jpg.gif', 'bmp' => 'jpg.gif', 'jpg' => 'jpg.gif', 'gif' => 'gif.gif', 'zip' => 'archive.png', 'rar' => 'archive.png', 'exe' => 'exe.gif', 'setup' => 'setup.gif', 'txt' => 'text.png', 'htm' => 'html.gif', 'html' => 'html.gif', 'fla' => 'fla.gif', 'swf' => 'swf.gif', 'xls' => 'xls.gif', 'doc' => 'doc.gif', 'sig' => 'sig.gif', 'fh10' => 'fh10.gif', 'pdf' => 'pdf.gif', 'psd' => 'psd.gif', 'rm' => 'real.gif', 'mpg' => 'video.gif', 'mpeg' => 'video.gif', 'mov' => 'video2.gif', 'avi' => 'video.gif', 'eps' => 'eps.gif', 'gz' => 'archive.png', 'asc' => 'sig.gif', ); /* That's it! You are now ready to upload this script to the server. Only edit what is below this line if you are sure that you know what you are doing! */ error_reporting(0); if(!function_exists('imagecreatetruecolor')) $showthumbnails = false; $leadon = $startdir; if($leadon=='.') $leadon = ''; if((substr($leadon, -1, 1)!='/') && $leadon!='') $leadon = $leadon . '/'; $startdir = $leadon; if($_GET['dir']) { //check this is okay. if(substr($_GET['dir'], -1, 1)!='/') { $_GET['dir'] = $_GET['dir'] . '/'; } $dirok = true; $dirnames = split('/', $_GET['dir']); for($di=0; $di
File
Size
Last Modified
'.$files[$i].''; $thumb2 = ' onmouseover="o('.$i.', \''.urlencode($leadon . $files[$i]).'\');" onmouseout="f('.$i.');"'; } if($filetypes[$ext]) { $icon = $filetypes[$ext]; } $filename = $files[$i]; if(strlen($filename)>43) { $filename = substr($files[$i], 0, 40) . '...'; } $fileurl = $leadon . $files[$i]; if(!$forcedownloads) { $fileurl = $_SESSION['PHP_SELF'] . '?dir=' . urlencode($leadon) . '&download=' . urlencode($files[$i]); } $fileurl = "../".$tempdir.$files[$i]; ?>
0KB
Jan 01 1970 01:00:00 AM

Thanks in advance

Tony

[mc12345678]

Umm, unless can point to where this issue is addressed in this forum, nothing about that content has to do with this version of EP

[reetp]

...code

Slight mod - needed to allow the file 'full-ep'

--- easypopulate_4-orig.php 2015-06-17 19:03:42.000000000 +0200
+++ easypopulate_4.php 2015-06-18 14:47:28.000000000 +0200
@@ -28,9 +28,9 @@
$ep_music = (int)EASYPOPULATE_4_CONFIG_MUSIC_DATA; // 0-Disable, 1-Enable
$ep_uses_mysqli = (PROJECT_VERSION_MAJOR > '1' || PROJECT_VERSION_MINOR >= '5.3' ? true : false);

-@set_time_limit($ep_execution); // executin limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit
+@set_time_limit($ep_execution); // execution limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit

-if (!$error) {
+if (!isset($error)) {
$upload_max_filesize=ini_get("upload_max_filesize");
if (preg_match("/([0-9]+)K/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024;
if (preg_match("/([0-9]+)M/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024*1024;
[root@home myadmin]# diff -ruN easypopulate_4_import-orig.php easypopulate_4_import.php
--- easypopulate_4_import-orig.php 2015-06-18 11:18:05.000000000 +0200
+++ easypopulate_4_import.php 2015-06-26 15:19:07.000000000 +0200
@@ -8,6 +8,34 @@

$file = array('name' => $_GET['import']);
$display_output .= sprintf(EASYPOPULATE_4_DISPLAY_LOCAL_FILE_SPEC, $file['name']);
+
+/*
+ * These are the filenames that we look for
+ *
+strtolower(substr($file['name'],0,11)) == "featured-ep")
+strtolower(substr($file['name'],0,15)) == "attrib-basic-ep"
+strtolower(substr($file['name'],0,18)) == "attrib-detailed-ep")
+strtolower(substr($file['name'],0,15)) == "sba-detailed-ep")
+strtolower(substr($file['name'],0,12)) == "sba-stock-ep")
+strtolower(substr($file['name'],0,15)) <> "categorymeta-ep")
+strtolower(substr($file['name'],0,7)) <> "attrib-")
+strtolower(substr($file['name'],0,4)) <> "sba-")
+strtolower(substr($file['name'],0,14)) == "pricebreaks-ep")
+*/
+
+$fileNameArray = array ("full-ep", "featured-ep", "attrib-basic-ep", "attrib-detailed-ep", "sba-detailed-ep", "categorymeta-ep", "attrib-", "sba-", "pricebreaks-ep");
+$discoveredFile = false;
+
+foreach ($fileNameArray as $key) {
+
+ $length = strlen ($key);
+ // probably better to do stristr but all the lines then need changing further down
+ if (strtolower(substr($file['name'],0,$length)) == $key) {
+ $discoveredFile = true;
+ }
+}
+
+if ($discoveredFile == true) { // we'll proceed

$ep_update_count = 0; // product records updated
$ep_import_count = 0; // new products records imported
@@ -1637,5 +1665,17 @@
} else {
$messageStack->add("File Import Completed.", 'success');
}
+}
+else {
+ $display_output .= '<h3>Finished Processing Import File</h3>';
+ $display_output .= '<br /><h3 style="color:red">Filename is wrong</h3> ';
+ $display_output .= '<br />Please check that the filename <b>starts</b> with the correct term';
+ $display_output .= '<br />The following are the permissible names :';
+ foreach ($fileNameArray as $key) {
+ $display_output .= "<br />" . $key;
+ }
+ $messageStack->add("File Import Failed.", 'error');
+
+} // End file check loop
} // END FILE UPLOADS
-?>
\ No newline at end of file
+?>


And a small patch to easypopulate.php itself to clear a small warning that was annoying me :-)

--- easypopulate_4-orig.php 2015-06-17 19:03:42.000000000 +0200
+++ easypopulate_4.php 2015-06-18 14:47:28.000000000 +0200
@@ -28,9 +28,9 @@
$ep_music = (int)EASYPOPULATE_4_CONFIG_MUSIC_DATA; // 0-Disable, 1-Enable
$ep_uses_mysqli = (PROJECT_VERSION_MAJOR > '1' || PROJECT_VERSION_MINOR >= '5.3' ? true : false--- easypopulate_4-orig.php 2015-06-17 19:03:42.000000000 +0200
+++ easypopulate_4.php 2015-06-18 14:14:01.000000000 +0200
@@ -30,7 +30,7 @@

@set_time_limit($ep_execution); // executin limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit

-if (!$error) {
+if (!isset($error)) {
$upload_max_filesize=ini_get("upload_max_filesize");
if (preg_match("/([0-9]+)K/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024;
if (preg_match("/([0-9]+)M/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024*1024;);

-@set_time_limit($ep_execution); // executin limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit
+@set_time_limit($ep_execution); // execution limit in seconds. 300 = 5 minutes before timeout, 0 means no timelimit

-if (!$error) {
+if (!isset($error)) {
$upload_max_filesize=ini_get("upload_max_filesize");
if (preg_match("/([0-9]+)K/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024;
if (preg_match("/([0-9]+)M/i",$upload_max_filesize,$tempregs)) $upload_max_filesize=$tempregs[1]*1024*1024;

[mc12345678]

This entire week I've been trying to figure out how to provide comment to your code without running you away because these things don't get better without someone pointing out an issue. I have no control over your response and if it is to walk away or otherwise be upset, then so be it. Hopefully now you can look back at the suggestions of the last several posts, including the goal of being helpful to the newbie as well as not unnecessarily forcing existing users to make changes in how they operate and maybe see things a little differently.

In regards to the most recent post:
1) Wondered when you would figure that one out, and now 2) have added a restriction that didn't exist before and is truly unnecessary other than to suppport this error message. With that one single addition now all files must follow a specific naming convention not just the ones that require special/unique handling. I'll have my mod up in a bit, been trying to come up with decent terminology to display and maybe be acceptable to the eye. It doesn't restrict the ability to process files of any name, shows the naming convention expected to process that type of file, and provides the info on the first screen shot rather than after import "after the damage is done". The display of all file types on the first screen is switchable from all, to all in the designated directory to no help provided (ie, current view for those classic users :) ) with the default operation/install being all help. So those installing the update and wnting to not have this extra assistance will need to run a smidge of sql or store a constant somewhere to change the display of the min page away from the "full help".