Little red question mark in red diamond - Easy Populate Backend Import Results - UTF8

Hi Guys

So I tried my best to research my issue and to be honest have tried everything everyone has mentioned on the subject and after breaking the site a few times and restoring it or changing code back have still gotten nowhere.

I installed a fresh Zencart v1.5.6c, made some changes here and there. Added Easy Populate 4.0.36.ZC - 07-05-2016 and started uploading CSV's.
I noticed after importing the CSV there are alot of red question marks inside a red diamond in the admin.

The front end cuts off any further characters of product name and specs after this symbol.

http://www.gotitall.co.za/index.php?...oducts_id=7860

I read it had to do with CHAR sets and changing database to UTF8 and I have tried all of this, without success.
I also change the code in various files as per different possible solutions.

I have copied the original specs from supplier pricelists in Notepad++ and can see that some of the original character is degrees, plus and minus ± etc etc.
For some reason I am unable to get Zencart/EasyPopulate/Database to see these characters.


Below is an example of the symbol that is replacing the special character.
Any help would be greatly appreciated.

Import Results

Filename: TarsusDistributionEpsonHardwareResellersPricelist02July2020.csv

WARNING! - Model: V11H921540 P390 - Products name: "EB-1480Fi, Projectors, Ultra short distance, Full HD, 5.000�lumen- 3.500�lumen�(economy), 5.000�lumen - 3.500�lumen�(economy), Over 2.500.000�: 1, USB 2.0 Type A (2x), Powered USB, RS-232C, Ethernet interface (100 Base-TX / 10 Base-T), Wireless LAN IEEE 802.11a/b/g/n/ac (WiFi 5), Wi-Fi Direct, VGA in (2x), VGA out, HDMI in (3x), HDMI out, Component in, Miracast, Stereo mini jack audio out, Stereo mini jack audio in (3x), Microphone input, Finger-touch interface, Multi-Touch interface, USB 2.0 Type B (2x), Sync. in, Sync. out, 9,8�kg, 16�W, Stereo:�Stereo, Linux, Mac OS 10.10.x, Mac OS 10.7.x, Mac OS 10.8.x, Mac OS 10.9.x, Ubuntu 14.10, Ubuntu 15.04, 2 x Pen tip(Teflon), 4 x Pen tip(Felt), CD Manual, Cable cover, Interactive pens, Lens cap, Pen Holder/Stand, Power cable, Remote control incl. batteries, Replacement pen tips" exceeds max. length: 64
UPDATED! - Model: V11H921540 P390 | 1 | | EB-1480Fi, | EB-1480Fi, | | | | | 71794.492 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H942940 P390 - Products name: "EB-L1050U, Projectors, Installation/Large venue, WUXGA, 1920 x 1200, 16:10, 4K enhancement, 7.000�lumen- 4.900�lumen�(economy) in accordance with IDMS15.4, 7.000�lumen - 4.900�lumen�(economy) in accordance with ISO 21118:2012, Over 2.500.000�: 1, USB 2.0 Type B (Service Only), RS-232C, WLAN (2x), VGA in, DVI in, HDMI in, BNC in, HDBaseT, Stereo mini jack audio out, Stereo mini jack audio in (3x), Ethernet interface (100 Base-TX / 10 Base-T), Wireless LAN IEEE 802.11a/b/g/n (optional), VGA out, HDMI (HDCP 2.2)" exceeds max. length: 64
UPDATED! - Model: V11H942940 P390 | 1 | | EB-L1050U, | EB-L1050U, | | | | | 73994.412 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H940840 P390 - Products name: "EB-L1075U, Projectors, Installation/Large venue, WUXGA, 1920 x 1200, 16:10, 4K enhancement, 7.000�lumen- 4.900�lumen�(economy) in accordance with IDMS15.4, 7.000�lumen - 4.900�lumen�(economy) in accordance with ISO 21118:2012, Over 2.500.000�: 1, Stereo mini jack audio out, VGA out, RS-232C, HDMI (HDCP 2.2), HDMI in, WLAN (2x), HDBaseT, VGA in, USB 2.0 Type B (Service Only), Stereo mini jack audio in (3x), DVI in, Ethernet interface (100 Base-TX / 10 Base-T), BNC in, Wireless LAN IEEE 802.11a/b/g/n (optional), 16,9�kg, CD Manual, Cable cover, Lens cap, Power cable, Quick Start Guide, Remote control incl. batteries" exceeds max. length: 64
UPDATED! - Model: V11H940840 P390 | 1 | | EB-L1075U, | EB-L1075U, | | | | | 101513.972 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H940940 P390 - Products name: "EB-L1070U, Projectors, Installation/Large venue, WUXGA, 1920 x 1200, 16:10, 4K enhancement, 7.000�lumen- 4.900�lumen�(economy) in accordance with IDMS15.4, 7.000�lumen - 4.900�lumen�(economy) in accordance with ISO 21118:2012, Over 2.500.000�: 1, HDBaseT, VGA in, USB 2.0 Type B (Service Only), Stereo mini jack audio in (3x), DVI in, Ethernet interface (100 Base-TX / 10 Base-T), BNC in, Wireless LAN IEEE 802.11a/b/g/n (optional), Stereo mini jack audio out, VGA out, RS-232C, HDMI (HDCP 2.2), HDMI in, WLAN (2x), 16,9�kg, CD Manual, Cable cover, Lens cap, Power cable, Quick Start Guide, Remote control incl. batteries" exceeds max. length: 64
UPDATED! - Model: V11H940940 P390 | 1 | | EB-L1070U, | EB-L1070U, | | | | | 101513.972 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11HA16140 P390 - Products name: "EB-L1495U Projectors, Installation/Large venue, WUXGA, 1920 x 1200, 16:10, 4K enhancement, 9.000�lumen- 6.300�lumen�(economy) In accordance with IDMS15.4, 9.000�lumen -6.300�lumen�(economy) In accordance with ISO 21118:2012, 2.500.000�: 1, Stereo mini jack audio in (3x), Stereo mini jack audio out, HDBaseT, BNC in, HDMI in, DVI in, VGA out, VGA in, Ethernet interface (100 Base-TX / 10 Base-T), RS-232C, USB 2.0 Type B (Service Only), HD-SDI, Wireless LAN b/g/n (2.4GHz) (optional), 23,8�kg, CD Manual, Cable cover, VGA cable, Power and signal cable (3 m), Quick Start Guide, Remote control incl. batteries" exceeds max. length: 64
UPDATED! - Model: V11HA16140 P390 | 1 | | EB-L1495U | EB-L1495U | | | | | 207058.732 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11HA16040 P390 - Products name: "EB-L1490U Projectors, Installation/Large venue, WUXGA, 1920 x 1200, 16:10, 4K enhancement, 9.000�lumen- 6.300�lumen�(economy) In accordance with IDMS15.4, 9.000 lumen - 6.300�lumen�(economy) In accordance with ISO 21118:2012, 2.500.000�: 1, Stereo mini jack audio in (3x), Stereo mini jack audio out, HDBaseT, BNC in, HDMI in, DVI in, VGA out, VGA in, Ethernet interface (100 Base-TX / 10 Base-T), RS-232C, USB 2.0 Type B (Service Only), HD-SDI, Wireless LAN b/g/n (2.4GHz) (optional), 23,8�kg, CD Manual, Cable cover, VGA cable, Power and signal cable (3 m), Quick Start Guide, Remote control incl. batteries" exceeds max. length: 64
UPDATED! - Model: V11HA16040 P390 | 1 | | EB-L1490U | EB-L1490U | | | | | 207058.732 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H981040 P390 - Products name: "EB-E20, Projectors, Mobile, XGA, 1024 x 768, 4:3, HD ready, 3.400�Lumen- 2.200�Lumen�(economy), 3.400�Lumen - 2.200�Lumen�(economy) In accordance with ISO 21118:2013, 15.000�: 1, USB 2.0 Type A, RS-232C, VGA in (2x), VGA out, HDMI in, Composite in, Stereo mini jack audio out, Stereo mini jack audio in (2x), Cinch audio in, USB 2.0 Type B, 3,1�kg, 5�Watt, Computer cable, Main unit, Power cable" exceeds max. length: 64
UPDATED! - Model: V11H981040 P390 | 1 | | EB-E20, Pr | EB-E20, Pr | | | | | 8849.36 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H988040 P390 - Products name: "EB-992F, Projectors, Mobile/Large venue, Full HD 1080p, 1920 x 1080, 16:9, Full HD, 4.000�Lumen- 2.400�Lumen�(economy), 4.000�Lumen - 2.400�Lumen�(economy) In accordance with ISO 21118:2013, 16.000�: 1, USB 2.0 Type A, USB 2.0 Type B, RS-232C, Ethernet interface (100 Base-TX / 10 Base-T), VGA in (2x), VGA out, HDMI in (2x), Composite in, Stereo mini jack audio out, Stereo mini jack audio in (2x), Cinch audio out, Microphone input, Wireless LAN IEEE 802.11b/g/n, Miracast, 3,1�kg, 16�Watt, Computer cable, Main unit, Power cable, Remote control incl. batteries, User manual, Warranty document, Wireless LAN unit" exceeds max. length: 64
UPDATED! - Model: V11H988040 P390 | 1 | | EB-992F, P | EB-992F, P | | | | | 19303.746 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H978040 P390 - Products name: "EB-FH52, Projectors, Mobile, Full HD 1080p, 1920 x 1080, 16:9, Full HD, 4.000�Lumen- 2.400�Lumen�(economy) In accordance with ISO 21118:2013, 4.000�Lumen - 2.400�Lumen�(economy) In accordance with ISO 21118:2013, 16.000�: 1, USB 2.0 Type A, USB 2.0 Type B, Wireless LAN IEEE 802.11b/g/n, VGA in, HDMI in (2x), Composite in, Cinch audio in, Miracast, 3,1�kg, 16�Watt, Carrying Case, Computer cable, Main unit, Power cable, Remote control incl. batteries, User's Manual Set, Warranty Documents" exceeds max. length: 64
UPDATED! - Model: V11H978040 P390 | 1 | | EB-FH52, P | EB-FH52, P | | | | | 15518.244 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H983040 P390 - Products name: "EB-W49, Projectors, Mobile, WXGA, 1280 x 800, 16:10, 3.800�Lumen, 3.800�Lumen In accordance with ISO 21118:2013, 16.000�: 1, USB 2.0 Type A, USB 2.0 Type B, RS-232C, Ethernet interface (100 Base-TX / 10 Base-T), Wireless LAN a/n (5GHz) (optional), VGA in (2x), VGA out, HDMI in (2x), Composite in, Stereo mini jack audio out, Stereo mini jack audio in (2x), Cinch audio in, 2,7�kg, 5�Watt, Computer cable, Power cable, Quick Setup Guide, Remote control incl. batteries, Warranty card" exceeds max. length: 64
UPDATED! - Model: V11H983040 P390 | 1 | | EB-W49, Pr | EB-W49, Pr | | | | | 13000.764 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H914040 P390 - Products name: "EF-100W Portable laser projector, white. HD ready, 0.59 inch LCD panel, Contrast: 2.500.000:1 Aspect ratio 16:10, Laser light source, Screen size 30-150 inches, USB 2.0 Type A, USB 2.0 Type B, HDMI in, Stereo mini jack audio out, Bluetooth, Power consumption: 172 Watt, 115 Watt (economy), 0,4 Watt (standby), Heat dissipation: 584,8 BTU/hour (max), Dimensions 210x230x92mm, Weight 2.7kg Room Type / Application: Gaming, Home Cinema, Home Office, Home Photo, Huddle space / Portable, Sports, Streaming, TV & Box sets." exceeds max. length: 64
UPDATED! - Model: V11H914040 P390 | 1 | | EF-100W Po | EF-100W Po | | | | | 10918.404 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: V11H914040 P390 - Products name: "EF-100W Portable laser projector, white. HD ready, 0.59 inch LCD panel, Contrast: 2.500.000:1 Aspect ratio 16:10, Laser light source, Screen size 30-150 inches, USB 2.0 Type A, USB 2.0 Type B, HDMI in, Stereo mini jack audio out, Bluetooth, Power consumption: 172 Watt, 115 Watt (economy), 0,4 Watt (standby), Heat dissipation: 584,8 BTU/hour (max), Dimensions 210x230x92mm, Weight 2.7kg Room Type / Application: Gaming, Home Cinema, Home Office, Home Photo, Huddle space / Portable, Sports, Streaming, TV & Box sets." exceeds max. length: 64
UPDATED! - Model: V11H914040 P390 | 1 | | EF-100W Po | EF-100W Po | | | | | 10918.404 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Projectors | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: C11CG07402SA P390 - Products name: "Epson WorkForce Pro WF-M5299DW, WorkForce Pro. Maximum duty cycle: 45000 pages per month, Maximum resolution: 1200 x 1200 DPI, Standard printer languages: ESC/P-R,PCL 5e,PCL 6,PDF 1.7,PostScript 3. Print speed (black, normal quality, A4/US Letter): 34 ppm, Time to first page (black, normal): 4.8 s, Duplex print speed (ISO/IEC 24734, A4) black: 15 ppm. Standard input capacity: 250 sheets, Maximum input capacity: 251 sheets. Standard output capacity: 150 sheets. Maximum ISO A-series paper size: A4, Standard tray media types: Letterhead,Plain paper,Postcard,Recycled paper, ISO A-series sizes (A0...A9): A4,A5,A6 12�months�Carry in" exceeds max. length: 64
UPDATED! - Model: C11CG07402SA P390 | 1 | | Epson Work | Epson Work | | | | | 5266.107 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Printers a | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |
WARNING! - Model: C11CH72403SA P390 - Products name: "Epson L15150 MEAF, Inkjet Printers, Consumer/Multi-fuction/Ink tank system, A3+, 4 Ink Cartridges, CMKY, Print, Scan, Copy, Fax, Yes (A4/A3, plain paper), Touchscreen, 4.800�x�2.400�dpi, 22�pages/min�Colour (plain paper), 32�pages/min�Monochrome (plain paper), 250�Sheets�Standard, 2.400�dpi x 1.200�dpi (Horizontal x Vertical), 50�pages, LCD screen: Type: Colour, Touchscreen, Diagonal:�10,9�cm, WiFi, Ethernet, USB, USB host, Wi-Fi Direct, Mac OS X 10.6.8 or later, Windows 10 (32/64 bit), Windows 7 (32/64 bit), Windows 8 (32/64 bit), Windows 8.1 (32/64 bit), Windows Server 2008 R2, Windows Server 2012 R2, Windows Server 2016, Windows Vista (32/64 bit), Windows XP Professional x64 Edition SP2 or later, Windows XP SP3 or later (32-bit), Windows Server 2003 R2, Windows Server 2003 SP2 or later, 1 set of ink (1x 127ml BK, 3x 70ml CMY), Main unit, Power cable, Quick Start Guide, User manual, 12�months�Carry in, 150.000�pages" exceeds max. length: 64
UPDATED! - Model: C11CH72403SA P390 | 1 | | Epson L151 | Epson L151 | | | | | 15736.446 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | | | 0 | Epson | Printers a | Taxable Go | 1 | 0 | 0 | 0 | 0 | 0 | | | |

Finished Processing Import File

Couple of things... first it appears that the csv file wasn't quite "made" right... The first record response appears to be an attempt to add a product with a *realllly* long product name... Is that really the intention? When going through the line that begins with WARNING: can see that at the end it identifies that the product's name is greater than 64 characters (length of the database field for products_name in the products_description and orders_products tables...)

As to the strange characters, that is typical when the original characters are not of the UTF8ish format and have been converted from the other character set to UTF8 but without an equivalent conversion result.

I tried to follow the link to the product and possibly view the page source; however, when attempting to see the page source, I'm presented with some other coding not representative of the page content... I haven't dug into why or what that is as it doesn't really relate to this issue.

While the data may be passed/entered in notepad++, the format of that file/data is important to be able to properly import the information. Not entirely sure (at the moment) why there are two rows of report for each such entry and not sure (yet) if the remaining data is truly processed for import after the name length issue occurs... it appears that it doesn't process further (contrary to the success message), but seems like seeing the headers and one of the "problem" lines may help clear that up. Note that if posted here (or in the EP4 forum), please click the # button in the reply message box toolbar before pasting that information...

Quote:

---

Originally Posted by mc12345678

Couple of things... first it appears that the csv file wasn't quite "made" right... The first record response appears to be an attempt to add a product with a *realllly* long product name... Is that really the intention? When going through the line that begins with WARNING: can see that at the end it identifies that the product's name is greater than 64 characters (length of the database field for products_name in the products_description and orders_products tables...)

As to the strange characters, that is typical when the original characters are not of the UTF8ish format and have been converted from the other character set to UTF8 but without an equivalent conversion result.

I tried to follow the link to the product and possibly view the page source; however, when attempting to see the page source, I'm presented with some other coding not representative of the page content... I haven't dug into why or what that is as it doesn't really relate to this issue.

While the data may be passed/entered in notepad++, the format of that file/data is important to be able to properly import the information. Not entirely sure (at the moment) why there are two rows of report for each such entry and not sure (yet) if the remaining data is truly processed for import after the name length issue occurs... it appears that it doesn't process further (contrary to the success message), but seems like seeing the headers and one of the "problem" lines may help clear that up. Note that if posted here (or in the EP4 forum), please click the # button in the reply message box toolbar before pasting that information...

---

Thanks for your response. The CSV list is created from our supplier price list. Excel format. We save the excel as CSV (comma delimited). Different suppliers price list behave differently. For example the HP price list from the same supplier loads with no issues. The length of the name exceeding 64 characters is not an issue and the reason I do this is because often the supplier do not have a name for the product just a code and description. We bulk upload CSV lists and then follow up by renaming the product in the format we require.

I don't know the format used by suppliers for their pricelist. We have 60 suppliers with about 5 price lists each and each listed is created by the relevant product managers for that brand. All the price lists are in excel format though and we convert to CSV using the save as function in excel.

I am not a coder and have very limited knowledge of what the page source is or is not supposed to contain. I do try understand it sometimes.

The only reason I copied the original specs into notepad++ was because of an article I read regarding this issue and I wanted to see for myself what the actual original character was before Zencart changed it to question mark in diamond.

I would like to be able to get suppliers pricelists save as CSV and upload into Zencart with all the characters that are contained in the price lists. To be honest there is really nothing special about these characters as I have seen many other websites advertising the same products with the same specs without the issues I'm facing.. and I really can't understand why Zencart can't read them properly and cuts of the rest of the specs as soon as it hits one of these "special" characters.

Quote:

---

Originally Posted by GameChanger69

Thanks for your response. The CSV list is created from our supplier price list. Excel format. We save the excel as CSV (comma delimited). Different suppliers price list behave differently. For example the HP price list from the same supplier loads with no issues. The length of the name exceeding 64 characters is not an issue and the reason I do this is because often the supplier do not have a name for the product just a code and description. We bulk upload CSV lists and then follow up by renaming the product in the format we require.

I don't know the format used by suppliers for their pricelist. We have 60 suppliers with about 5 price lists each and each listed is created by the relevant product managers for that brand. All the price lists are in excel format though and we convert to CSV using the save as function in excel.

I am not a coder and have very limited knowledge of what the page source is or is not supposed to contain. I do try understand it sometimes.

The only reason I copied the original specs into notepad++ was because of an article I read regarding this issue and I wanted to see for myself what the actual original character was before Zencart changed it to question mark in diamond.

I would like to be able to get suppliers pricelists save as CSV and upload into Zencart with all the characters that are contained in the price lists. To be honest there is really nothing special about these characters as I have seen many other websites advertising the same products with the same specs without the issues I'm facing.. and I really can't understand why Zencart can't read them properly and cuts of the rest of the specs as soon as it hits one of these "special" characters.

---

Understand. Apologies if I don't address all of the above in this post, ask if there is more desired... For the export from Excel, yes CSV (comma delimited), but also need to have it use the appropriate encoding. The encoding can be changed by then selecting the "settings" dropdown near the save/cancel buttons, selecting Web options. Then in I think it is called the encoding tab, change from "western" to Unicode UTF8... that should store the characters as uTF8 compatible characters....

View source: when looking at a Web page on say a PC, right clicking on the page should offer a dropdown of options. One of those "usually" is view source. Or could have an inspect option. The result of making such a choice is supposed to show the html code that the browser processes to show content. When I did that for the above page, there was a lot of "other" data not directly readable as I would normally expect to see.

Quote:

---

Originally Posted by mc12345678

Understand. Apologies if I don't address all of the above in this post, ask if there is more desired... For the export from Excel, yes CSV (comma delimited), but also need to have it use the appropriate encoding. The encoding can be changed by then selecting the "settings" dropdown near the save/cancel buttons, selecting Web options. Then in I think it is called the encoding tab, change from "western" to Unicode UTF8... that should store the characters as uTF8 compatible characters....

View source: when looking at a Web page on say a PC, right clicking on the page should offer a dropdown of options. One of those "usually" is view source. Or could have an inspect option. The result of making such a choice is supposed to show the html code that the browser processes to show content. When I did that for the above page, there was a lot of "other" data not directly readable as I would normally expect to see.

---

Note that Excel's UTF8 is actually UTF8-BOM, which can lead to import issues.

Quote:

---

Originally Posted by lat9

Note that Excel's UTF8 is actually UTF8-BOM, which can lead to import issues.

---

Yup, sure is as I now re-recall... was going to test it and see what other options were available for export, though getting directly from Excel to UTF8 has long been or raised an issue or required a lot of code and consistency of use/identifier...

Okay so I needed to first establish whether the issue was with the database or easy populate or excel or CSV. I manually uploaded a single product with all the special characters and guess what. It works perfectly.

http://www.gotitall.co.za/index.php?...oducts_id=7997

So now I am in 2 minds. Do I continue to try and pin point the problem and fix the issue or do I go ahead and install Database I/O Manager and hope not to mess up the installation.

https://vinosdefrutastropicales.com/...products_id=68

I have read the threads regarding this plugin and I must give credit where it is due to the author lat9 - Vinos de frutas tropicales as from the thread I can see you are super active and have dealt with all the teething problems of your plugin.

Do I try fix what I currently have or install a whole new database manager. Hmmm...

Now that I know my site can display those special characters no problem and the database handles them no problem. I can only assume the issue must lie with Easy Populate. Assumption is the mother of all ....... Can anyone shed any light please.

Quote:

---

Originally Posted by GameChanger69

Okay so I needed to first establish whether the issue was with the database or easy populate or excel or CSV. I manually uploaded a single product with all the special characters and guess what. It works perfectly.

http://www.gotitall.co.za/index.php?...oducts_id=7997

So now I am in 2 minds. Do I continue to try and pin point the problem and fix the issue or do I go ahead and install Database I/O Manager and hope not to mess up the installation.

https://vinosdefrutastropicales.com/...products_id=68

I have read the threads regarding this plugin and I must give credit where it is due to the author lat9 - Vinos de frutas tropicales as from the thread I can see you are super active and have dealt with all the teething problems of your plugin.

Do I try fix what I currently have or install a whole new database manager. Hmmm...

Now that I know my site can display those special characters no problem and the database handles them no problem. I can only assume the issue must lie with Easy Populate. Assumption is the mother of all ....... Can anyone shed any light please.

---

Because the issue appears to be with the contents of the CSV file (that wasn't working in its entirety) the only way to determine the cause is to further review operation with that file... that said, the version of EP4 used isn't the most recent as available at https://github.com/mc12345678/EasyPopulate-4.0. Otherwise, DBIO has been written to include the encoding conversion from Excel to ZC when the "user" knows that the file is from Excel (there's not really been any autodetection that I've seen to be able to let the software determine the source of the file and a reason EP4 hasn't been revised to include such a feature.) Basically EP4 says, make the CSV file a UTF8 file and it will handle the import. That is directly supportable by other spreadsheet programs such as Open Office. Issue though is that it depends on the ability to install such a program like if trying to install to a computer that is "locked down" by IT people at ones company.

As far as activity, yeah, couple of ways to look at it, if it needs to be fixed (and an issue has been identified as repeatable) then yes there will likely be a change made.

Is this the latest version ?

I thought I already had the latest version.
Easy Populate 4.0.36.ZC - 07-05-2016

Can I get the latest version here ?

https://www.zen-cart.com/downloads.php?do=file&id=2069

Quote:

---

Originally Posted by GameChanger69

Is this the latest version ?

I thought I already had the latest version.
Easy Populate 4.0.36.ZC - 07-05-2016

Can I get the latest version here ?

https://www.zen-cart.com/downloads.php?do=file&id=2069

---

Unfortunately, because I have known that there were still some issues (incomplete desired changes) with the code, I haven't pushed it to the forum yet... Ever since I first started using EP4 nearly 8 years ago, I have wanted it to use the language code instead of the language id; however, it had been deeply rooted in the database numeric designation. Anyways, most (but not all) of the import filters have been updated to handle using the "newer" language code (e.g. _en, _de) in place of the language id (e.g. _1, _2) with a current setting to allow both of the fields to be present but to use one as a priority over another to support those that still have whatever "process" that uses the language_id and/or to allow "conversion" to the language_code (more universal as it is not database dependent).

Anyways, the last identified version number is/was 4.0.37.10, though it appears that I didn't update the version number with other recent changes...

Haha that's okay. It been so long since I've added a product manually that I almost forgot how to do it :p
I'm downloading open office and going to create CSV's and repopulate.
Will post results.

Update

Opened Excel file from supplier with Open Office saved as text CSV.
Uploaded using Easy Populate and all is working perfectly.

Seems to be a problem with Excel/CSV comma delimited.
My excel version is 2007 as I am a die hard old school and refuse to upgrade.
Perhaps with newer version of Excel there are other CSV options that might work too.

Anyway hope this helps anyone that has had this issue.

Verdict.

No problems with Excel, database, files/code, character sets etc. etc..

Thanks for everyone's input.

Wow okay so my problems never seem to end...

So open office worked well for about 10 supplier excel price lists and then I noticed that when I opened an excel sheet, open office duplicated the specifications 3 or 4 times.. So back to the drawing board as I can't seem to remedy this open office/excel issue.

I did try converting the excel to CSV using MS and then opening the CSV with Open Office to save as text CSV but this comes with it's own set of issues from information in v columns being scrambled all over the place to Easy Populate not seeing and information in the product model column even though I could visually see it in Open Office.

Any suggestions ? Perhaps another alternative to Open Office ?

Quote:

---

Originally Posted by GameChanger69

Wow okay so my problems never seem to end...

So open office worked well for about 10 supplier excel price lists and then I noticed that when I opened an excel sheet, open office duplicated the specifications 3 or 4 times.. So back to the drawing board as I can't seem to remedy this open office/excel issue.

I did try converting the excel to CSV using MS and then opening the CSV with Open Office to save as text CSV but this comes with it's own set of issues from information in v columns being scrambled all over the place to Easy Populate not seeing and information in the product model column even though I could visually see it in Open Office.

Any suggestions ? Perhaps another alternative to Open Office ?

---

The issue of "opening, duplication" can you explain/clarify a little more? Origin of file being opened (was it an exported file from the database, a new file from supplier, etc...) duplication: columns being duplicated, rows, both? Does it happen only with this one supplier? Do they identify the encoding that they use to generate their file? (might need to specifically open the file to convert it as there may not be a way to auto-detect the source encoding).

There's open Libre I think as well... again, it's important to know what is being presented so can either handle it properly or figure out a process for that supplier. I've known some that have dropped a supplier because their file format is just simply too cumbersome to use regardless the import tool... I would of course suggest talking to the supplier to identify the issue(s). If they refuse to improve, then sure make a choice...

Okays thanks for response.

Yes it would duplicate the product specs.
So basically just repeat the specs 3 or 4 times.

I found people with the same issues here

https://bz.apache.org/ooo/show_bug.cgi?id=123022


The file is the original Excel file that we download from suppliers website or is sent to us via email.
All of our 50+suppliers use exactly this same format.

We then convert to CSV to populate.

I used Libre now and it works perfectly without these issues.

Thanks. Hopefully I won't have to come back to this thread again.