Database I/O Manager (DbIo) Plugin [Support Thread]

[lynbor]

@lynbor, thanks for a copy of that product-import CSV. I took the first line, stripping all others, and added the DbIo command ADD as the last column, so that the added product won't overwrite my test setup.

I did receive:

✗ products.products_date_added, line #2: Value (0000-00-00 00:00:00) is not a recognized date value; the default value for the field will be used.
✗ Product not inserted at line number 2; category id (0) has categories.
✗ products.products_date_added, line #3: Value (0000-00-00 00:00:00) is not a recognized date value; the default value for the field will be used.
✗ Product not inserted at line number 3; category id (0) has categories.
✗ products.products_date_added, line #4: Value (0000-00-00 00:00:00) is not a recognized date value; the default value for the field will be used.
✗ Product not inserted at line number 4, no match found for categories_name (Irontite).

Where the category-related issues are due to differences between my categories and yours, but I'm not seeing the "Value is not an integer". Are you using the same version of DbIo and its parameters (set in Configuration->DbIo Manager Settings) for both the store from which you're exporting and the store to which you're importing?

I installed DbIo from the same file to all three shopping cart programs. The only setting change I made was to the version I'm importing into I turned on the ability for products to be able to create catagories. Otherwise they are all from the same install files. Also, i'm importing into a brand new fresh full install without sample data in the database. (not sure if that matters) It's just weird that it thinks those numbers are not integers. How is that even possible?

[lat9]

@lynbor, I still don't have an answer regarding the results you're seeing and have requested that a full DbIo debug be generated and sent to me directly.

[lat9]

Correction found for the issue reported by @lynbor; see https://github.com/lat9/dbio/commit/...7bb295aebedbea for details.

[OopsIbrickedIT]

Hi,

I installed DbIo on ZC 1.5.0 which runs non english language. Copied the language files to my language folders and DBIo seemed to work. Then I started to make around 1700 product import preparations and spent all most a day of doing so. Now I am little frustrated because I get this error:

The current import-file is missing these (v_products_id) required columns; the import is disallowed.

I use ; as a delimeter but changed that on the settings to be correct first thing. I also converted to , delimeter but no change. v_products_id is first column in my csv file. This is very frustrating because I have a tight time frame regarding these products and now I have 0 of them for sale and selling season goes by hour by hour... Any help would be greatly appreciated.

[OopsIbrickedIT]

I turned out debugging but it revealed nothing I could see, then I made an export from the folder where I was going to import products and it revealed something very peculiar:



Actually that is all most funny but I am so pissed of I am not laughing. See Wikipedia and find out what an Integer is

[OopsIbrickedIT]

Okay, perhaps the problem lies here, because we use umlauts the db collation is the Real deal aka UTF-8 not utf8, no they are not the same thing: https://www.iana.org/assignments/cha...ter-sets.xhtml Preferred name is UTF-8!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[OopsIbrickedIT]

So this reveals something: https://medium.com/@adamhooper/in-my...4-11761243e434

So is your dbio using 3 bytes encoding?

[OopsIbrickedIT]

There is a ready made library for utf-8 support: https://github.com/tchwork/utf8/blob...Shim/Iconv.php

[OopsIbrickedIT]

Sorry if I was very snappy earlier, haven't slept for ages and totally lost it.

There is a ready made library for iconv utf-8 support: https://github.com/tchwork/utf8/blob...Shim/Iconv.php

Seems it could possibly be incorporated to support other encodings. this is from forceutf8 at encoding.php:

* It may fail to convert characters to UTF-8 if they fall into one of these scenarios:
   *
   * 1) when any of these characters:   ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß
   *    are followed by any of these:  ("group B")
   *                                    ¡¢£¤¥¦§¨©ª«¬##®¯°±²³´µ¶•¸¹º»¼½¾¿
   * For example:   %ABREPRESENT%C9%BB. «REPRESENTÉ»
   * The "«" (%AB) character will be converted, but the "É" followed by "»" (%C9%BB)
   * is also a valid unicode character, and will be left unchanged.
   *
   * 2) when any of these: àáâãäåæçèéêëìíîï  are followed by TWO chars from group B,
   * 3) when any of these: ðñòó  are followed by THREE chars from group B.

The database in question is ancient and has been around since early 2000 and db-tables use utf-8(so it supports newly named utf8mb4 if I have understood right) and must accomodate all sorts of peculiar strings because related to technical products and language is not english to begin with. DB is ancient, converted possibly many times to support those and I suspect it contains åö䩪«¬##® etc chars. So the db connection is utf-8 in configure.php
and I am not sure if you try to import öä宩ª strings to it and have mysql db connection of utf8 and db tables utf-8 with myisam. What will happen? mysql utf8 connection means 3-byte character support right? So I am in terrible trouble if I try to squeeze 3 byte stuff which requires special chars in 4 byte environment?

I guess myisam is a good enough storage engine still for lots of reads and less writes. I have still no proper testing environment so can not simply test things out. Thanks to our very sluggish hosting operator.

Db currently anyways works and shows special characters right. Reason I must understood this thorough is that we are planning (or my boss is planning) to accomodate extra 30-70K products into that db. This tool would be a great help say for transferring few thousand products at a time. Next part is perhaps more complex but I tried to start with this.

Sorry for lengthy post.

[lat9]

Hi,

I installed DbIo on ZC 1.5.0 which runs non english language. Copied the language files to my language folders and DBIo seemed to work. Then I started to make around 1700 product import preparations and spent all most a day of doing so. Now I am little frustrated because I get this error:

The current import-file is missing these (v_products_id) required columns; the import is disallowed.

I use ; as a delimeter but changed that on the settings to be correct first thing. I also converted to , delimeter but no change. v_products_id is first column in my csv file. This is very frustrating because I have a tight time frame regarding these products and now I have 0 of them for sale and selling season goes by hour by hour... Any help would be greatly appreciated.

I turned out debugging but it revealed nothing I could see, then I made an export from the folder where I was going to import products and it revealed something very peculiar:



Actually that is all most funny but I am so pissed of I am not laughing. See Wikipedia and find out what an Integer is

It sounds like (from the first referenced post) that you're importing those products. You'll need to add a column named v_dbio_command and fill each column with the DbIo command ADD to force the import of those products.

For the second referenced post, you've run into the same issue as @lynbor; see post #213 for the correction.