Page 1 of 2 12 LastLast
Results 1 to 10 of 15
  1. #1
    Join Date
    Apr 2008
    Posts
    133
    Plugin Contributions
    1

    Default How to deal with users registering in Cyrillic (unicode)?

    I am running 1.56c.

    Occasionally I have customers in Russia that create new accounts and type in their information in Cyrillic. What I end up seeing on my end is a bunch of question marks, i.e. User Name ????????? ????????.

    Is there any way to deal with this, such as preventing them from using Cyrillic, or changing some settings on my side so that the Cyrillic is properly displayed?

  2. #2
    Join Date
    Jul 2012
    Posts
    16,718
    Plugin Contributions
    17

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    What is the history of the store/database? Has it been fully set to handle utf8mb4 type entries?
    ZC Installation/Maintenance Support <- Site
    Contribution for contributions welcome...

  3. #3
    Join Date
    Apr 2008
    Posts
    133
    Plugin Contributions
    1

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    I would guess that the answer to "Has it been fully set to handle utf8mb4 type entries?" is NO.

    The store/database has been around for many many years. I'm not sure what was the original first version I installed. 1.3.x. For a long while, it was stuck on 1.3.8a. I then, finally in the last year or so, did the entire update procedure (followed all of the recommended steps in an upgrade post here), which involved (I think) going to 1.3.9, then 1.53, then 1.5.5, then 1.5.6 etc. So all of the updating and database updating seems to have worked, the store has been functioning fine on the latest version other than this issue; but I don't specifically recall anything about UTF8MB4 upgrades.

    So what would I need to do to fix this? Is there a setting somewhere? A database script that needs to be run?

  4. #4
    Join Date
    Sep 2009
    Location
    Stuart, FL
    Posts
    12,401
    Plugin Contributions
    87

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by karma-lab View Post
    I would guess that the answer to "Has it been fully set to handle utf8mb4 type entries?" is NO.

    The store/database has been around for many many years. I'm not sure what was the original first version I installed. 1.3.x. For a long while, it was stuck on 1.3.8a. I then, finally in the last year or so, did the entire update procedure (followed all of the recommended steps in an upgrade post here), which involved (I think) going to 1.3.9, then 1.53, then 1.5.5, then 1.5.6 etc. So all of the updating and database updating seems to have worked, the store has been functioning fine on the latest version other than this issue; but I don't specifically recall anything about UTF8MB4 upgrades.

    So what would I need to do to fix this? Is there a setting somewhere? A database script that needs to be run?
    Check out Convert db2utf8, with a database that old, I'm guessing that you've either got "mixed database collations" or the 'collation' (aka language) used by the database is different from the value identified by your /admin/includes/configure.php and /includes/configure.php files' DB_CHARSET setting.

  5. #5
    Join Date
    Apr 2008
    Posts
    133
    Plugin Contributions
    1

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by lat9 View Post
    Check out Convert db2utf8, with a database that old, I'm guessing that you've either got "mixed database collations" or the 'collation' (aka language) used by the database is different from the value identified by your /admin/includes/configure.php and /includes/configure.php files' DB_CHARSET setting.
    Thanks!

    My configure.php has:

    define('DB_CHARSET', 'utf8'); // 'utf8' or 'latin1' are most common

    While using phpMyAdmin to look at the database shows:

    Collation: latin1_swedish_ci

    ...which it turns out is the default setting for databases in earlier versions of mySQL, I guess.

    I'm a little worried about this comment in the Convert db2utf8 link:


    CAUTION: If your database is NOT in UTF8 format but contains UTF8 characters, you may end up with unpredictable results. ALWAYS MAKE A DATABASE BACKUP FIRST!!!!

    I have had people use UTF8 characters in registrations; I hope it doesn't blow up.... I will backup the database of course.

  6. #6
    Join Date
    Sep 2009
    Location
    Stuart, FL
    Posts
    12,401
    Plugin Contributions
    87

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by karma-lab View Post
    Thanks!

    My configure.php has:
    define('DB_CHARSET', 'utf8'); // 'utf8' or 'latin1' are most common

    While using phpMyAdmin to look at the database shows:
    Collation: latin1_swedish_ci

    ...which it turns out is the default setting for databases in earlier versions of mySQL, I guess.

    I'm a little worried about this comment in the Convert db2utf8 link:

    CAUTION: If your database is NOT in UTF8 format but contains UTF8 characters, you may end up with unpredictable results. ALWAYS MAKE A DATABASE BACKUP FIRST!!!!

    I have had people use UTF8 characters in registrations; I hope it doesn't blow up.... I will backup the database of course.
    See also this article that I've posted on my website.

  7. #7
    Join Date
    Apr 2008
    Posts
    133
    Plugin Contributions
    1

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Thanks, it seems that both of you (lat9, mc12345678) have been involved in the latest releases of this converter. Is there any more info you can give me about the comment "CAUTION: If your database is NOT in UTF8 format but contains UTF8 characters, you may end up with unpredictable results."?

  8. #8
    Join Date
    Sep 2009
    Location
    Stuart, FL
    Posts
    12,401
    Plugin Contributions
    87

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by karma-lab View Post
    Thanks, it seems that both of you (lat9, mc12345678) have been involved in the latest releases of this converter. Is there any more info you can give me about the comment "CAUTION: If your database is NOT in UTF8 format but contains UTF8 characters, you may end up with unpredictable results."?
    Besides, as @mc12345678 indicated, making a backup of the database as the first step, I'd get screenshots of the phpMyAdmin display so that I 'know' what tables have potentially invalid data.

    After running the db2utf8 tool, I'd then either visually inspect those tables to see if any characters got mangled. Even better, I'd keep a copy of the old database so that I cold export the affected tables one-by-one and then use a file-comparison tool to see if any of those characters were changed.

  9. #9
    Join Date
    Apr 2008
    Posts
    133
    Plugin Contributions
    1

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by lat9 View Post
    Besides, as @mc12345678 indicated, making a backup of the database as the first step, I'd get screenshots of the phpMyAdmin display so that I 'know' what tables have potentially invalid data.
    Thanks, but what do you mean by "tables that have potentially invalid data."? How do I know which tables potentially have invalid data?

  10. #10
    Join Date
    Sep 2009
    Location
    Stuart, FL
    Posts
    12,401
    Plugin Contributions
    87

    Default Re: How to deal with users registering in Cyrillic (unicode)?

    Quote Originally Posted by karma-lab View Post
    Thanks, but what do you mean by "tables that have potentially invalid data."? How do I know which tables potentially have invalid data?
    You'll need to use your webhost's phpMyAdmin to visually inspect (that'll take a while on large databases) the various tables that are currently using a latin1_* type collation.

    This will also be potentially time-consuming, but I found this sweet little SQL query
    Code:
    SELECT * FROM tableName WHERE columnToCheck NOT REGEXP '[A-Za-z0-9.,- ]';
    ... that will identify any entries where a character field is not ASCII characters (alphanumerics), a blank or one of .,-. You'll need to replace tableName with the Zen Cart table name and the columnToCheck with the desired character field.

    Note, too, that the query could be extended using the OR constructor to check for multiple columnToCheck values.

 

 
Page 1 of 2 12 LastLast

Similar Threads

  1. v151 How to deal with this problem?
    By andis02 in forum General Questions
    Replies: 8
    Last Post: 9 Dec 2014, 01:03 PM
  2. v139h blocking new users from registering with certain email providers
    By JoseWilliams in forum Basic Configuration
    Replies: 4
    Last Post: 7 Nov 2013, 05:37 PM
  3. How to deal with a weekly rate
    By afukada in forum General Questions
    Replies: 0
    Last Post: 17 Jan 2011, 01:14 AM
  4. how to deal with shoes size?
    By summerhere in forum Setting Up Categories, Products, Attributes
    Replies: 2
    Last Post: 28 Aug 2009, 01:55 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
disjunctive-egg
Zen-Cart, Internet Selling Services, Klamath Falls, OR