SGS-020003 Invalid XML error in FirstData / Linkpoint module since upgrade to 1.51

I have a client that has a lot of special characters in their product names (TM, (R), 1/2, 1/4, etc.). It seems that since upgrading from Zen Cart 1.50 to 1.51, a large percentage of orders are failing with the error "SGS-020003 Invalid XML". All of these failing orders have a special character in them, while the ones that go through do not.

Looking at previous threads, I thought this issue was fixed at some point - I have replaced linkpoint_api.php with an earlier version (@version $Id: linkpoint_api.php 8559 2008-05-29 12:31:24Z drbyte), but I'll have to wait for another order of that type to see if the problem has been fixed.

Any other ideas?

Thanks

UPDATE: Switching to the old linkpoint_api.php code did not solve the problem. We just had one order go through successfully, and one order fail that had a '®' in the product name.

There shouldn't be any significant difference in operation between 1.5.0 and 1.5.1

Did you also upgrade from v1.3.x earlier?
What character-set are you using? http://www.zen-cart.com/content.php?313-should-i-convert-my-store-from-iso-8859-1-to-utf8
What DB_CHARSET are you using?
What's your site URL? (I want to look at the browser's "view source" screen and see what charset you're pushing to your customers). It's okay to break the URL up with spaces when posting if you don't want search engines to pick it up.
What PHP version are you using?
What plugins/addons does the site have installed?

Yes, I upgraded from 1.3.x earlier

I have define('CHARSET', 'utf-8') set - same as before the site upgrade.

The site's HTML header also has <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

The DB_CHARSET is set to 'utf8'
Now that I think about it, I had to add the file my_db_charset.php - it was not there previously. It was the only way I could get the new theme I was testing to display special characters correctly in the HTML output.

The collation in the database itself is latin1_swedish_ci for all tables except the admin_* tables, which are utf8_general_ci - I'm not sure how it ended up this way.

PHP version is 5.3.16

Plugins/addons: aside from theme-specific stuff (lightbox, etc.), I am using Image Manager.

Looking at your suggestions, I'm guessing the added DB_CHARSET has something to do with the problem, as that is the only significant thing that has changed. As a temporary workaround, I have commented out lines 322-357 in linkpoint_api.php (the part dealing with building the order line items that get sent to LinkPoint). We have had 4 successful orders since then.

d9000:

As a temporary workaround, I have commented out lines 322-357 in linkpoint_api.php (the part dealing with building the order line items that get sent to LinkPoint). We have had 4 successful orders since then.Ya, that's a temporary bandage for a much bigger deeper problem which you'll need to sort out separately.

d9000:

The collation in the database itself is latin1_swedish_ci for all tables except the admin_* tables, which are utf8_general_ci - I'm not sure how it ended up this way.The latin1 stuff is how your old database was originally created, and thus the content inside your database will really be latin1, not utf8.
So, it appears as though you've got an invalid hybrid of latin1 and utf8 in your database (ie: you're really using latin1, but you're telling the system to treat it as utf8).

And, because you're incorrectly "tricking" it into thinking it's really already using utf8, the conversion-to-utf8 logic in the linkpoint module is saying "well, since I'm being told that the data I'm receiving is already utf8, I won't bother converting it to utf8 before submitting to linkpoint".

Hi Dr Byte,
I have the same problem.
One order did not go through with this error: SGS-020003: Invalid XML
I checked the database and it is the same built as you described above: all data files are built as latin1_swedish_ci except the admin files which are utf8_general_ci.
How did I made this mistake and how I can fix it?
Please let me know,

I am using
Zencart 1.5.0
PHP 5.2.17
MySQL 5.0.96-community

D9000:
Can you advise the path to that particular linkpoint_api.php file? I have similar issues and would like to try your method (commenting lines) to confirm if my problem is related to the charsets.

Your response will be greatly appreciated.

Ron

d9000:

Yes, I upgraded from 1.3.x earlier

I have define('CHARSET', 'utf-8') set - same as before the site upgrade.

The site's HTML header also has <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

The DB_CHARSET is set to 'utf8'
Now that I think about it, I had to add the file my_db_charset.php - it was not there previously. It was the only way I could get the new theme I was testing to display special characters correctly in the HTML output.

The collation in the database itself is latin1_swedish_ci for all tables except the admin_* tables, which are utf8_general_ci - I'm not sure how it ended up this way.

PHP version is 5.3.16

Plugins/addons: aside from theme-specific stuff (lightbox, etc.), I am using Image Manager.

Looking at your suggestions, I'm guessing the added DB_CHARSET has something to do with the problem, as that is the only significant thing that has changed. As a temporary workaround, I have commented out lines 322-357 in linkpoint_api.php (the part dealing with building the order line items that get sent to LinkPoint). We have had 4 successful orders since then.