Multiple, critical problems - EZ POP

I have Zen Cart 1.3.9h installed with Easy Populate 1.2.5.4

I am just having nothing but problems with it.

When I did finally manage to get an upload of my inventory (3700 products) it came out quite badly wrong with almost every product linked to two categories, the correct one and another incorrect one. The way the upload has come out it seems to have incorrectly assigned the first few products to category A, the next few to category B, the next to category C, and so on. I am pretty sure of this as I always keep my uploads in alphabetical order and that is the way they have come out, alongside the correct category assignment. I am pretty experienced with spreadsheets and csv uploads so I don't think that is the problem, the file was in tab-delimited text, formatted correctly I think, with EOREOR at the end of each row.

The 'server response' (if I can call it that) to the upload was pretty horrible, with a dialogue box along the lines of 'You have decided to open blahblahblah.php what do you want to do with this file - open - save - cancel' this would then open a blank page in a new tab every time I clicked 'open', this would only stop when I clicked 'cancel'. I never got any kind of confirmation of an upload success.

So anyway I decided the best thing to do was to manually edit all the products to their right categories. This isn't such a problem in itself as I have worked out how to use the Multiple Categories Link Manager to do this in bulk. Problem is, Zen cart is just throwing up every kind of error, timing out, failing to connect to the server, it is just completely unusable. I sometimes get php error messages though not so often, it is usually just impossible to access admin at all. I do have some logs (I've altered the admin area name):

[18-May-2011 13:30:07] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:07] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:07] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:07] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:11] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:11] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:13] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:13] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:14] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:14] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:32] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:32] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:44] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:44] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:53] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:53] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:58] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:30:58] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:31:04] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:31:04] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:31:04] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241
[18-May-2011 13:31:04] PHP Warning: explode() [<a href='function.explode'>function.explode</a>]: Empty delimiter in /home/a2679801/public_html/catalog/mysecretadminarea/includes/functions/extra_functions/easypopulate_functions.php on line 241

[18-May-2011 13:50:48] PHP Warning: preg_replace() [<a href='function.preg-replace'>function.preg-replace</a>]: Unknown modifier 'a' in /home/a2679801/public_html/catalog/includes/modules/pages/popup_image/header_php.php on line 41

[18-May-2011 13:57:50] PHP Warning: preg_replace() [<a href='function.preg-replace'>function.preg-replace</a>]: Unknown modifier 'a' in /home/a2679801/public_html/catalog/includes/modules/pages/popup_image/header_php.php on line 41

[19-May-2011 01:43:19] PHP Warning: preg_replace() [<a href='function.preg-replace'>function.preg-replace</a>]: Unknown modifier 'a' in /home/a2679801/public_html/catalog/includes/modules/pages/popup_image/header_php.php on line 41

[19-May-2011 02:47:21] PHP Warning: file(/home/a2679801/public_html/catalog/temp/2011_05_18th) [<a href='function.file'>function.file</a>]: failed to open stream: No such file or directory in /home/a2679801/public_html/catalog/mysecretadminarea/easypopulate.php on line 1223
[19-May-2011 02:47:21] PHP Warning: Invalid argument supplied for foreach() in /home/a2679801/public_html/catalog/mysecretadminarea/easypopulate.php on line 1301

[19-May-2011 05:05:02] PHP Warning: preg_replace() [<a href='function.preg-replace'>function.preg-replace</a>]: Unknown modifier 'a' in /home/a2679801/public_html/catalog/includes/modules/pages/popup_image/header_php.php on line 41

To make matter worse, FTP connection to the site seems to time out after just a short amount of inactivity and I have to reconnect.

Can anyone help please? I'm really at my wit's end. Thanks.

Looks like everything is pointing to EP. If you are at the beginning, I'd start fresh, carefully load the latest EP, and use it to load about 20 items to make sure it's working before doing the entire 3700.

MOST problems with EP are a result of errors in the DATA file - not the program.

True... the program is very fussy, but you must get your TAB-DELIMITED TXT file absolutely correct before you load the product data.

See my WIKI tutorial: http://www.zen-cart.com/wiki/index.php/Easy_Populate

Thanks for the responses.

I did try to uninstall EP using /easypopulate.php?langer=remove but nothing happened.

Here's a sample of my file for what it's worth:

v_products_model v_products_image v_products_name_1 v_products_description_1 v_products_url_1 v_specials_price v_specials_date_avail v_specials_expires_date v_products_price v_products_weight v_date_avail v_date_added v_products_quantity v_manufacturers_name v_categories_name_1 v_categories_name_2 v_categories_name_3 v_categories_name_4 v_categories_name_5 v_categories_name_6 v_categories_name_7 v_tax_class_title v_status EOREOR

1 92544.jpg 666: D.E.V.I.L. 2x12" 666: D.E.V.I.L.<BR>2x12"<BR>House Nation DST 70713-12<BR>Germany 2000<BR><BR>D.E.V.I.L. (Break The Spell Mix)<br>D.E.V.I.L. (What The Hell Mix)<br>Alarma!<br>Diablo<br>Amokk<br>D.E.V.I.L.<br>Bomba!<br>I'm Your Nitemare<br>Paradox<BR><BR>Condition media / cover: Very Good Plus (VG+) / Generic<BR> 3.99 450 1 Trance --none-- 1 EOREOR

Maybe special characters in my category names are causing problems, I've used these:

& , - / '

So just use letters, nothing else? I don't know what is the exact defintion of a special character...

You are using " to indicate INCHES... Change these to In.

The " character can cause problems.

So can ' .

And as I said... the problems are most often data quality.

You should TEST one product (yes... just one product in a TXT file).

If this works then EP works... Any subsequent issues will be with DATA.

Wow, that is a bit of a suprise. :shocking:

I don't quite know what to make of that as my main sale items are 12" singles.

The galling thing is, all my stock is uploaded and my shop displays fine, it's just the admin area I'm having problems with. It would be quite a blow not to be able to use the primary search term my customers are looking for. It's too bad I don't sell CDs...! I am almost exclusively a vinyl trader! Hmmmn... :unsure:

OR invest $75 for the commercial version. Even though it hasn't been updated in 3 years and there are no support for it, It works every time even with special characters such as (", ').

I can understand Free is good, but the tool the allow you to import/export your store files is worth few bucks.

The Free version is limited and from what I read in it's support thread (Which is very lengthy) hard to add or change fields for the templates.

EP Advanced work on my stores with more than 200,000 products, I can upload 10000 product in one click.

I'll bear that in mind.

I guess at least with my inventory up (in spite of all these incorrect additional categories) and a working store (apparently) the best thing to do would be to bide my time and see how things go, at least until I next need to do an update on my stock.

When I had an OSCommerce store it got hacked and was deleted by my webhost, so there is still lots of time for things to go catastrophically wrong before I start tinkering with things!

At least I feel a bit calmer as I have a better idea what is probably causing the problems, so that is something. It really never occured to me it could be something as simple as a quotation mark.

Thanks all. Any other pointers are welcome of course.

If you are using MS Excel to work the initial spreadsheet, you will find that it forces unwanted formatting - even if you choose to save the file as a tab-delimited text file.

Annoyingly, when Excel sees " in the data, it assumes you also want to delimit text using inverted commas, and it will pepper the resultant txt file with inverted comma symbols. Sometimes, it places these after a tab space resulting in the next field being delimited by inverted commas.

Easy Populate gets confused with this. However, it CAN tolerate inverted commas within the field contents - so your 12" should be OK.

So... how does one circumvent excel's tendency to pepper the TXT file with " . ?

I have found a possible solution - but you will need a program like CRIMSON EDITOR or EMERALD EDITOR (both are free text editors - use Google to find the download).

Install Emerald Editor, open it at an empty file... then...

1. In your excel spreadsheet, HIGHLIGHT the range of cells containing your data (ie: all the columns across and all the rows down THAT CONTAIN DATA. (Do not highlight the entire sheet.

2. COPY that highlighted block.

3. Go over to Enmerald editor, and use CTRL+V (paste) the block.

4. It should behave nicely, and paste EXACTLY the data, and NOT put unwanted " marks directly ahead of, or behind TAB spaces.

5. SAVE the emerald editor file as your xxxxxxxxx**.txt** file.

6. Load that through easy Populate.

(You could delete ALL product data in your store - to remove duplicates - by grabbing a EP dump of your current store data, then using v_status = 9, then importing that into EP. Be careful here... you MUST backup your images folder first. I usually rename my images folder to images_bak, and then create a new, empty images folder. If you fail to do this, a v_status=9 will delete all your images too. Then, with a cleaned out product database, load your new file created via Emerald Editor, into your store.)

One other downside of this is that all your products will assume new ID's, so if you have attributes, you may lose the associations. Also, historical order data may be lost too, as the original id will be gone, and the product that owned it will have a new id. I have worked around this too, by getting EP to export the product ID field contents, then making sure to start the database's increment at 0 again - (requires phpMyAdmin knowledge).

Wow, this is great stuff, many thanks.

As soon as I read your post I went straight to the tab-delimited text file, opened it with Notepad++, and sure enough, what did I see?

6 1285073.jpg """Weird Al"" Yankovic: Eat It 7""" """Weird Al"" Yankovic: Eat It<BR>7""<BR>Scotti Bros. Records A 4257<BR>UK 1984<BR><BR>Eat It<br>That Boy Could Dance<BR><BR>Condition media / cover: Very Good (VG) / Generic<BR>The labels have some water damage and there is a small crack in the edge of the vinyl, the playing surface is in good shape however." 0.99 50 1 Synth-pop --none-- 1 EOREOR

On that one row alone I count no less than 10 quotes that shouldn't be there, it looks like you've hit on a possible major issue...

So I tried following your instructions using Notepad++, and here is the result for the same line:

6 1285073.jpg "Weird Al" Yankovic: Eat It 7" "Weird Al" Yankovic: Eat It<BR>7"<BR>Scotti Bros. Records A 4257<BR>UK 1984<BR><BR>Eat It<br>That Boy Could Dance<BR><BR>Condition media / cover: Very Good (VG) / Generic<BR>The labels have some water damage and there is a small crack in the edge of the vinyl, the playing surface is in good shape however. 0.99 50 1 Synth-pop --none-- 1 EOREOR

This is exactly right, yee ha! :clap:

Nice post schoolboy, it looks like you've managed to identify a major issue and solve it in one go.

What I don't understand is though, you say I could delete all product data in your store to remove duplicates, and all my products will assume new ID's, I don't get this, can't I just overwrite all my existing products using their old ID's 1 - 3700? Should I run tests using this method in any case?

NB I'm not sure if this will help but I can confidently say my spreadsheet product IDs are currently the same as they were when I uploaded, I have exactly 3665 products on my sheet and the order should still be the same, as I always sort by artist, then title, then cat # before I upload. They will certainly all have changed by the next time I want to upload though, and I will need to replace the entire stock with an up-to-date inventory by then.

Understand what I mean when I refer to ID.

This is NOT the "model number", which you insert.

A relational database (like the one ZC applies), uses dbase-generated ID numbers in many instances, to give a unique identification to a line of data.

A database usually uses a function called AUTO INCREMENT... a technique of assigning the next number up from the previous number, so that there is always a unique, single ID number to identify a particular piece of data.

You can see these id's appear in many instances in your admin panel. If you go to ADMIN >>> CATALOG >>> CATEGORIES/PRODUCTS, you will see the ID numbers appear in the left column.

These ID numbers are used to CROSS-REFERENCE (relate) one set of data with another. For example, if a customer buys a widget in your store, that widget is referenced by an ID number. Instead of re-writing the name and description of the widget in the table that stores purchase history, the association is referenced by the ID number - so the purchase history will just show a list of Product ID numbers. Sort of like saying:

" Joe Smith bought 112, 134, 154, 165, 225 ".

I see. But the issues that you mentioned before about losing attributes and historical order data won't apply to me as 1) I don't have any products atributes and 2) I don't have any historical order data because I've never had any orders to my Zen Cart store.

I'm still a bit unsure what to do next. Backup my database and reload my catalogue as you advised, I guess?

Yes... do a backup of the database.

Then, get EasyPop to CREATE a file of your current data to store in the TEMP folder.

You can also then use a COPY of this file to do a v_status = 9 "mass-delete" (see below).

Then, either manually delete all your product data (via your admin >>> catalog >>> categories/products ), or as i said, run an easypop file where v_status column is all equal to 9.

BUT... rename your images folder first. If you use EP to DELETE products, it will remove its associated image as well.

If your store is not yet trading, you can "afford" to try things out. But doing regular Dbase backups is critical.

schoolboy:

These ID numbers are used to CROSS-REFERENCE (relate) one set of data with another. For example, if a customer buys a widget in your store, that widget is referenced by an ID number. Instead of re-writing the name and description of the widget in the table that stores purchase history, the association is referenced by the ID number - so the purchase history will just show a list of Product ID numbers. Sort of like saying:

" Joe Smith bought 112, 134, 154, 165, 225 ".

Actually... this is a "bad example" because in zencart's case it DOES record the name and model number in the orders history table... Clever zencart... it knows that if the original product data is deleted, the order history must still maintain these essential bits of info!