EasyPopulate 4.0 Support Thread

Now attempting to purge my entire inventory and start from scratch by deleting categories, but it seemed to be too much for the server as the admin page became inaccessible and eventually my site went down altogether for a time, with 'WARNING: An Error occurred, please refresh the page and try again.' Tried to work around by creating extra database users in MySQL and editing /includes/configure.php to the rotating users tweak as on http://www.zen-cart.com/showthread.php?24089-Rotating-Db-Users&p=144728#post144728 but that didn't work either so I had to switch back. It seems my shop works fine so long as I never try to update the inventory. There has to be an easier way than this... :sighs:

double-happiness:

Now attempting to purge my entire inventory and start from scratch by deleting categories, but it seemed to be too much for the server as the admin page became inaccessible and eventually my site went down altogether for a time, with 'WARNING: An Error occurred, please refresh the page and try again.' Tried to work around by creating extra database users in MySQL and editing /includes/configure.php to the rotating users tweak as on http://www.zen-cart.com/showthread.php?24089-Rotating-Db-Users&p=144728#post144728 but that didn't work either so I had to switch back. It seems my shop works fine so long as I never try to update the inventory. There has to be an easier way than this... :sighs:

Hi,

If you want to clear out your inventory, but the admin delete is timing out, then you need to do a search for the SQL commands that you can run via the SQL patch feature to remove them.

Note that when you delete an item from the admin, it also deletes the associated photo. You may not want that.

Adding a custom field is a two part process... first you need to add the field to the data base via phpMyAdmin from within your cpanel, once you've done that, then you need to enter it in the EP4 configuration page.

Also note: Database field name CANNOT have spaces in them, which is what you look to have done.

EP4 DOES handle file upload and downloads. But it doesn't STREAM the file at run time, meaning the data is exported to a file in your temp directory, and then from there you can download it.

Also, if you have a large import file, and a low end shared server/hosting environment, then you should use the SPLIT feature after you upload the file to break your import file into manageable chunks. Much work when into improving the performance of EP4, and I have tested with with another user importing 900,000 records... in this case we broke the file into 50,000 chunks and those import in about 200 seconds. He had a dedicated server which is much more powerful that the typical shared hosting. So try for the default of 2,000 record breaks when you split.

Note that this version is not an "upgrade" to the other out there, but a rewrite with many core changes to the code.

Hope this helps...

k9dug:

Easy Populate 4 Documentation

Is there a definitive set of docs for this version of the addon?

The wiki page (http://www.zen-cart.com/wiki/index.php/Easy_Populate) does not apply to version 4, and as far as I can see, there is no equivalent documentation for version 4.

There are some docs in Github, and 102 pages of posts here in the forum, as well as older info in the wiki, but it would make life a lot easier if there was an actual set of up-to-date docs somewhere for getting started, testing and then easy reference.

I would really like to see a nice clean table listing the columns and showing, at least, for each:

• Column heading

• Data type (string, int, float etc)

• Required/optional

• Unique (Y/N)

• Notes (covering all the ways of using that column, restrictions etc)

Cheers,

gavin

Hey Gavin,

You sound like a software engineer :P

Yes, that would be good to have, and I do have intentions, but time has not been permitting. I have a major update to post, but time is the killer....

Hi Chad,

I'm a new Zenner and built a site using this mod on 1.51. I try to learn as much as possible before doing especially since I'm not a programmer. Most features and functions work and I thank you for building this mod. I would like to take advantage of the additional product fields and have gone through this entire thread searching for why my listed fields still come up "FALSE" under Tools/EP4. In reading post #'s 205, 378, 479, 621, 786, 789, etc., you mention creating those new fields in the SQL database. Unfortunately, when I go there with MyPHP admin in C-Panel and see the structure and field entries for all the existing databases, I'm completely lost with what to do. Any assistance would be appreciated.

chadderuski:

There are basic instruction on how to import attributes in the README.txt file that was included in the zip you downloaded from github.

Be sure to use Dr. Bytes MySQL Backup Tool... this way you can experiment without and roll back should something go awry.

Also, it is really recommended that you create a mirror of your live install for experimentation and testing. I always create LIVE, MIRROR and TESTING sites. This does take more time and effort initially, but it can really save your behind.

I also use the AutoMySQLBackup VER 2.5.1 to do daily MySQL backups. The script is quite good, and will rotate your old backups according to the schedule you want (daily, weekly, monthly).

-chadd

OK, now getting somewhere.

I did a fresh install of ZC 1.5.1, with all example products.
Then installed Backup MySQL Plugin, backed up the DB, installed EP4 (all ok).

Then I generated one of each of the 11 types of export file for a copies of the original product data.

As I am specifically interested in creating and updating attributes and attribute prices, I then ran several tests on the file "Attrib-Detailed-EP2013Jun18-134609.csv":

1. Deleted all except for 5 rows showing "MG200MMS" in v_products_model (ie all 5 varieties of a single product).
   Altered all the values for v_options_values_price.
   Uploaded and imported - all test values appeared correctly in the front end. (Great! - first time this far).

2. Starting with the same 5 rows, I then emptied all columns except
   v_products_model
   v_products_options_name
   v_products_options_values_name
   v_options_values_price
   These are the ones I am interested in.

This failed and generated the following errors:

Output page top:

Warning File Import Completed with issues.
Warning An SQL error has occured. Please check your input data for tabs within fields and delete these. 
If this error continues, please forward your error log to the Easy Populate maintainer

Output page bottom:

Import Results

Filename: Attrib-Detailed-EP2013-test2-a.csv

SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
Finished Processing Import File

Updated records: 0
New Imported records: 0
Errors Detected: 5
Warnings Detected: 0
Memory Usage: 6322296
Memory Peak: 6463696
Execution Time: 0.0096170902252197 seconds.

3. I then tried deleting out all columns except for the four I'm interested in.
   Same errors as above.

Can you tell me which columns are required and which are optional?

I want to minimise as much as possible the size of the spreadsheet.

Thanks,

gav

Hello Chad – I have created 2 user defined fields using the How to add new product fields? thread from
http://www.zen-cart.com/showthread.php?120523-How-To-Add-New-Product-Fields

The fields are created in a table called products_extra_stuff.

Even though I am adding the 2 field names under admin>Configuration>Easy Populate 4 User Defined Products Fields(separated with commas, no spaces), they are defined as **false **under admin>Tools>Easy Populate 4.

Is there something that I have missed? Or should I be adding something else to EP4 code or this is not possible with EP4?

Really need this to upload data.

Kevin205:

Hello Chad – I have created 2 user defined fields using the How to add new product fields? thread from
http://www.zen-cart.com/showthread.php?120523-How-To-Add-New-Product-Fields

The fields are created in a table called products_extra_stuff.

Even though I am adding the 2 field names under admin>Configuration>Easy Populate 4 User Defined Products Fields(separated with commas, no spaces), they are defined as **false **under admin>Tools>Easy Populate 4.

Is there something that I have missed? Or should I be adding something else to EP4 code or this is not possible with EP4?

Really need this to upload data.

Kevin,

User defined PRODUCTS fields only apply to the PRODUCTS table. Can you put those fields in the products table? If you do that, it will work.

NOW, if you brave and want to dive into the code, it shouldn't be TOO hard to follow the my lead. The next release of EP4 (crosses fingers) will have this more simplified. I moved all the custom stuff to user defined fields.

-chadd

k9dug:

OK, now getting somewhere.

I did a fresh install of ZC 1.5.1, with all example products.
Then installed Backup MySQL Plugin, backed up the DB, installed EP4 (all ok).

Then I generated one of each of the 11 types of export file for a copies of the original product data.

As I am specifically interested in creating and updating attributes and attribute prices, I then ran several tests on the file "Attrib-Detailed-EP2013Jun18-134609.csv":

1. Deleted all except for 5 rows showing "MG200MMS" in v_products_model (ie all 5 varieties of a single product).
   Altered all the values for v_options_values_price.
   Uploaded and imported - all test values appeared correctly in the front end. (Great! - first time this far).

2. Starting with the same 5 rows, I then emptied all columns except
   v_products_model
   v_products_options_name
   v_products_options_values_name
   v_options_values_price
   These are the ones I am interested in.

This failed and generated the following errors:

Output page top:

Warning File Import Completed with issues.
Warning An SQL error has occured. Please check your input data for tabs within fields and delete these.
If this error continues, please forward your error log to the Easy Populate maintainer

Output page bottom:

Import Results

Filename: Attrib-Detailed-EP2013-test2-a.csv

SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
SKIPPED! - Attribute Entry on Model: MG200MMS - Not Found!
Finished Processing Import File

Updated records: 0
New Imported records: 0
Errors Detected: 5
Warnings Detected: 0
Memory Usage: 6322296
Memory Peak: 6463696
Execution Time: 0.0096170902252197 seconds.

> 
> 3. I then tried deleting out all columns except for the four I'm interested in.
> Same errors as above.
> 
> Can you tell me which columns are required and which are optional?
> 
> I want to minimise as much as possible the size of the spreadsheet.
> 
> Thanks,
> 
> gav

Gav, 

You must first use the BASIC sheet to create your products options names and products options values names. Once you do that, you download the DETAILED sheet, make your edits and then upload to update.

-chadd

Thanks for the prompt reply.

When would the next version be out?

Any lead into where to start diving would be greatly appreciated.

ps. everyone says try to keep any additional fields away from the core tables. That's why I chose to create a new table!

chadderuski:

Hi,

If you want to clear out your inventory, but the admin delete is timing out, then you need to do a search for the SQL commands that you can run via the SQL patch feature to remove them.

Note that when you delete an item from the admin, it also deletes the associated photo. You may not want that.

Adding a custom field is a two part process... first you need to add the field to the data base via phpMyAdmin from within your cpanel, once you've done that, then you need to enter it in the EP4 configuration page.

Also note: Database field name CANNOT have spaces in them, which is what you look to have done.

EP4 DOES handle file upload and downloads. But it doesn't STREAM the file at run time, meaning the data is exported to a file in your temp directory, and then from there you can download it.

Also, if you have a large import file, and a low end shared server/hosting environment, then you should use the SPLIT feature after you upload the file to break your import file into manageable chunks. Much work when into improving the performance of EP4, and I have tested with with another user importing 900,000 records... in this case we broke the file into 50,000 chunks and those import in about 200 seconds. He had a dedicated server which is much more powerful that the typical shared hosting. So try for the default of 2,000 record breaks when you split.

Note that this version is not an "upgrade" to the other out there, but a rewrite with many core changes to the code.

Hope this helps...

Thanks for the reply.

I've learned the hard way about images being deleted when an item is deleted from admin. That is not something I want at all, not sure what to do about that though.

I've looked in phpMyAdmin to try to find where the User Defined Product Fields are supposed to to go, without success.

None of the database field names were entered with spaces.

Kevin205:

Thanks for the prompt reply.

When would the next version be out?

Any lead into where to start diving would be greatly appreciated.

ps. everyone says try to keep any additional fields away from the core tables. That's why I chose to create a new table!

Well... you don't HAVE to follow that advice. For coding changes, be sure to use the over rides, but if you are simply adding a couple fields that are very specific to the products data, put them in the PRODUCTS database. Things will be MUCH simpler, you'll have fewer SQL requests, and actually LESS core modifications.

Next version? Weeks... Months? Depends on my work load and if I can get some free time. Next version will be submitted to the zencart downloads, so I want to be sure it's solid and well documented.

-chadd

That's exactly what I did.

Thank you. Chad

Thanks for the great software. I used to use an old version of Easy Populate on an old store. I'm setting up a new store at the moment and this version of Easy Populate is Looking very promising! MUCH MUCH faster. A lot less memory usage on my VPS.

Will test it out tomorrow! Thanks already for ALL your work. I program a lot of stuff in my freetime and I know how much work a project like this is. You are a blessing for the community!

CEON URI mapping and Easy Populate
Im having a problem with these 2 mods. When I export all my products using easy populate, the URL column is blank. Its blank in phpmyadmin as well which is normal behavior I guess with CEON?

But how can I do an export so all products include a URL? Im doing this for a shopping comparison site feed.

Chargin:

CEON URI mapping and Easy Populate
Im having a problem with these 2 mods. When I export all my products using easy populate, the URL column is blank. Its blank in phpmyadmin as well which is normal behavior I guess with CEON?

But how can I do an export so all products include a URL? Im doing this for a shopping comparison site feed.

Read this about Product URL:

http://www.zen-cart.com/showthread.php?82957-What-is-Products-Url

Not what you'd expect.

I am working on an update that export a products cpath (what you normally see when you're on a product page), I will have to do some research to see what Conor did in his URI mapping mod, but I think he uses some of his own tables and core modifications....

-chadd

Wow! Amazing timing. I just came here looking for a way to get Ceon URI and EP4 to play together! :D

Yes, Conor added his own tables. I have manipulated Ceon URIs manually before, but it's been a long time. If memory serves, he links to the actual Prod ID. I think what I did was dump his Prod URI table (using phpMyAdmin), then dump the ZC Prod URI table, then line the two files up by Prod ID. Once you have the info, you can manipulate it, split up again, and import back.

I have been wondering for a long time why EP4 does not use the actual Prod IDs (or Cat IDs)? I really, really like to use (and occasionally manipulate) those bits of information. Extremely useful stuff. I guess it could too easily lead to screwing things up?

Anyway, I'm bummed to see there is no easy way to import/export Ceon URIs yet.

Chadd, the update you describe working on (export cpath) is super exciting! I would get a ton of use out of it, especially if it worked with Ceon URI! How far off in the future is your update?

Chargin, I'm guessing Chadd's update won't be ready in the next 24 hours, so I'm probably going to try to figure out how to do it manually. If you are comfortable with manipulating the database directly, I'll be happy to relay some instructions after I figure it out.

Feznizzle that sounds good, if you can give us an outline.
I looked at phpmyadmin and was thinking excel might work with an export but it all seemed too hard....

Id love to get chadd's mod when hes ready though.

Thanks guys.

Chadd (or anybody!),

I've been searching thru this thread, trying to find a way to get EP4 to export product_ID. But I can't seem to find a way to do it. Can you tell me an easy way to do it?

Next question... in EP4, what triggers the connection between product info (name, description, price, etc) and the controlling poduct_id? Is it product model? No, I don't think so. I'm thinking Product Name?

Man, I really need an easy way to access those ID's. Please help!

Thanks,
Mike

NM the question about what triggers the connection... I see model numbers control everything. Wow. That is rough, if you have thousands and thousands of products as I do. I don't control the model numbers, manufacturers do. A model # for a widget from Manufacturer #1 could easily conflict with Manufacturer #2.

Wow. This is a bummer! The only thing I am sure won't conflict is ZC ID.

For a moment there, I thought getting product id was going to be easy.

I went to: Configuration/Easy Populate 4

Clicked on: User Defined Products Fields

And added: v_products_id

But it showed up as FALSE on the Tools/Easy Populate 4 page. What am I doing wrong?