EP 1.2.5.4 & 1.2.5.5: What's file format?

Hosted: Zen Cart v1.3.8a with Database Patch Level v1.3.8, Easy Populate v1.2.5.5.csv, PHP v5.2.5 (Zend: 2.2.0), and MySQL v5.0.67.d7-ourdelta-log. Local: Windows Vista Ultimate, Office 2007 Professional

I restored MySQL back to the database that came with new install of ZC with its sample store (before EP got installed). With EP already installed, I then created 1 of every type file in its tempEP folder. I then ftp’d the Full type .csv file to my local computer to look at with Excel 2007.

I then used EP to restore the FULL file back into ZX. I got two lines of red text:
No model field in record. This line was not imported | Test Docum | This is a |
No model field in record. This line was not importedsamples/3_ | Sample of |

My first question is: are those real errors EP detects and what makes them occur?

In the Specials Summary area I got 5 errors reported in red text:
SKIPPED! - Model: HPLJ1100XI - specials price higher than normal price...
SKIPPED! - Model: HPLJ1100XI - specials price higher than normal price...
SKIPPED! - Model: Test25-10AttrAll - specials price higher than normal price...
SKIPPED! - Model: HPLJ1100XI - specials price higher than normal price...
SKIPPED! - Model: Book - specials price higher than normal price...

I assume that’s normal filtering? The part that bothers me is the 1st two lines. My second question is if they should be appearing twice in that report, or not?

But that was all a test, and it established that EP could read the file it created and place it into ZC. It meant that the format of that FULL file was correct.

Then I open up Excel into a new session with just 1 workbook, and make sure the cursor is at the row/column location A1. Then I click on the Data tab. In the Get External Data section, I click on From Text button navigate to my copy of that Full type .csv file and highlight it, and then click the Import button. In step #1 of the import wizard I assure that Original data type is set for Delimited, and that the import will start from Row #1, and then I click the Next button. In Wizard step #1, I’ve noticed in the preview box that the row fields in the Full file are not tab delimited.

That was a surprise, so my 3rd question is: If EP is supposed to be using tab delimited files then why isn’t it creating tab delimited Full files?

In the Excel Wizard, now at step #2, I get another surprise. There is absolutely combination of selections for the tab delimiter, comma delimiter, and any text qualifier that will keep errors from showing up in the v_column_model column when examining fields in the data preview window provided there. If I just scroll down a ways, I see text that simply does not belong in that 1st column. I also notice that there is no EOREOR text at the end of each row (file line).

Curious as to why that was so, I looked at the Full data closer with a Hex Editor. The comma simply must be used as a column (field) delimiter, as that is how Full fields are delimited (not by tabs). I noticed that the line terminator used in the Full file was not $0D $0A (Return followed by a Line Feed), it was only $0A but no $0D character code.

The next thing I noticed was that some comma delimited fields that have text (model for example) do not have that text enclosed within quote marks, while others do (description_1 for example).

How I set the text qualifier (" ' or {none} are the available selections) impacts how Excel tries to determine what data goes in what column. If some column that is NOT enclosed with quote marks happens to have a comma in it then, with commas being truly required to delimit fields, Excel will get out of sync. It appears that the correct way would be to use the " character as a text qualifier, and then Excel will try to treat the cell as a numerical value unless it finds any text characters in that cell.
So my 4th question is, how should the text qualifier be set? I’m thinking a " setting.

I thought ALL fields were supposed to be set to type text when Excel goes to actually import the file (the next step, #3 of 3 in the Wizard). But now I’m thinking only columns that are really supposed to contain text and which, if they did, would be enclosed within quotes should be imported as type text, and the others left to type general in the Wizard. Which leads to my 5th question: Which columns should be set and how should they be set as relates to when using the Excel import Wizard in step #3?

I don’t think it’s possible to import the Full file into Excel 2007 that EP1.2.5.5.csv creates. It imports into ZC just fine (ignoring the above mentioned 2 errors). Which makes me think I must do programmatical parsing of the Full file before I can pass it into Excel. I guess it all depends on the final answers to my above 5 questions.

These are valid concerns impacting the ability to use ZC. Nice as it is, it’s not usable if hundreds or thousands of products have to be put into ZC by hand, of course.

After reading all the above, please tell me and the rest of the forum what the exact formatting of the EP Full file should be, so that I can programmatically change its content to be in a format that Excel can use. There’s nothing I can do otherwise to make it import the Full file without errors eventually popping up in the v_products_model field.

And my last question, #6. Will ZC (not talking about EP) silently fail with what EP is giving it if the format of a field is not as expected? Might that be why, despite my best programming efforts, EP might report no errors and categories actually get created but no product ever shows up from the import process?

:oops:
I was unable to edit my above post (more than 7 minutes had past). A typo needs correction. The text that reads in part "surprise. There is absolutely combination" should read "surprise. There is absolutely no combination".
:beta:
I am able to programmatically create a file for EP to give to ZC. EP reports no errors in doing so. The file consists of the header names for the "v_" variables, exactly the same format you see if you look at the first $0A (Lf) delimited line of any Full-*.csv file that EP creates, followed by 1 line for a product. When EP hands the data fields from that file off to ZC then the product category and subcategory is created, and neither EP nor ZC does any complaining at all. But ZC doesn’t create the product entry, it’s not there if you go look for it. But neither EP nor ZC did any complaining at all. The file used is located here: http://www.zen-cart.com/forum/attachment.php?attachmentid=6581&d=1255405128

There are 2 reasons I can think of that could result in a product not showing up when EP gives the file data to ZC. The least likely reason would be that EP simply isn’t giving any notice about it only handing off everything EXCEPT product specific data (name, price, etc.)

The most likely reason seems to me might be that all the file line fields, concerning the product that come AFTER the category data, must be in quote marks if ZC is expecting that specific field to be enclosed within quote marks, and vice versa for specific other fields that ZC does not want enclosed within quote marks. I think ZC will silently fail, after having created the categories, if the product data fields are not found to be formatted as expected – and so the product will be silently rejected and not imported. ZC might also silently fail if the file product row/record ends with an extra field delimiter.

Without being able to properly import an EP generated FULL-*.csv file into Excel 2007, without it eventually getting out of sync no matter what, makes it very difficult to determine which product (not category) related fields need to be enclosed within quote marks - and which must not be. It becomes near impossible to tell when fields that would use enclosing quote marks use no quote marks at all if there is no data for that field (i.e. the file field contains only a comma field delimiter but not 2 quote marks, with no character between them, followed by a comma.

I’m afraid PHP is not a language I program in. If I were I’d not have started this thread, I could have looked! But I’m very good using Visual Basic. When I construct the file to give to EP to hand off to ZC, I’m doing things like these few example lines:

If v_tax_class_title > "" Then _

MyInput = MyInput & Q & _

v_tax_class_title & Q 'Quotes enclose field data

If v_status > "" Then _

MyInput = MyInput & _

v_status 'No quotes enclose field data

'Strip last field delimiter if finished parsing.
If Right(MyInput,1) = Cm then _
MyInput = Left(MyInput,Len(MyInput)-1)

MyInput = MyInput & Chr(10) 'Lf end of row delimiter

where Q is a quote mark and Cm is a comma field delimiter character. Note that quote marks enclose only data when a field appears to me to require it, some do and some do not. Also note that nothing gets into a field except the delimiting character if there is no data for that field (in all cases, too).
:onfire:
What my 2 bottom line question (the others in my previous post are just investigative questions) then are:

v_* by v_* EP file field name, when creating a file to give to EP so products can be imported into ZC by EP, which of the field v_ names need to have their data enclosed within quote marks and which do not? Which field names should consist of nothing but the comma field delimiting character if a field has no data, and what is the needed formatting for exceptions to that rule?*:huh:

I got the trial version of MagneticOne Store Manager, which I will buy if they have solution to a couple of issues (transfer speed and image path formatting). Meanwhile I see how it creates a .csv file after importing from an Excel file, and in particular which fields require being enclosed within quote marks and which ones are not supposed to be enclosed in quote marks (SM shows default values for fields). But something about Zen Cart catalog structure confuses me if I also want to use EPcsv.

EP needs fields v_categories_name_1 through v_categories_name_7.

Say that I have products that should appear to the store visitor in a tree as:
Auto Parts
..Ford
...Electrical
....product1

Should v_categories_name_1 look something similar to this?

Auto Parts|Ford|Electrical

Does EP use that "|" character (ASCII #124 decimal) or something similar in creating the catalog tree within just one category, such as in v_categories_name_1?

If v_categories_name_2 has this exact content:

Wow!|Super Deals

Auto Parts
..Ford
...Electrical
....product1

Wow!
.Super Deals
..product1

Is that how one product gets "linked" to show in more than one catalog?

Still trying to get EPcsv to work for me when I programmatically create the tab delimited .csv file it wants. I understand that the order of the field names (left to right) in the .csv file I create for EP to hand off to ZC needs to be identical to the order they appear in when EPcsv is used to create a file that can be downloaded and looked at in a spreadsheet, and now I think I finally know which header data fields must or must not have content enclosed within quote marks, and also which fields are required to have content and which ones are not.