Paginate by Letter issue of Countries list in Multilingual Admin

ZC 1.5.8
Upgrading from 1.5.7d and testing on local server PHP Version: 8.1.10
Multilingual site - two languages one of them English. Two different alphabets.
DB Table "countries" includes country names in both languages.

Problem/Bug:
When trying to access Location/Taxes -> Countries from the menu only the firs page appears without the new alphabetical list on the bottom. The following is the error log:

PHP Fatal error: Uncaught ArgumentCountError: 4 arguments are required, 3 given in ..\admin\includes\classes\split_page_results.php:360
Stack trace:
#0 \admin\includes\classes\split_page_results.php(360): sprintf('\xD0\x9F\xD0\xBE\xD0\xBA\xD0\xB0\xD0\xB7\xD0\xB2\xD0\xB0 ...', 15, 474)
#1 ..\admin\countries.php(217): splitPageResults->display_count(15, 20, 'A', '\xD0\x9F\xD0\xBE\xD0\xBA\xD0\xB0\xD0\xB7\xD0\xB2\xD0\xB0 ...')
#2 ..\admin\index.php(11): require('..\\htdocs...')
#3 {main}
thrown in ..\admin\includes\classes\split_page_results.php on line 360

Not an issue in ZC 1.5.7d.

I have dual language, php8.2 ok.
Does this happen with both languages?
Check your language files are utf-8.

All pages and the DB are UTF-8.
When the language is English, the alphabetical list at the bottom is there and all URLs of the English alphabet lead to the corresponding places in the countries list. But when I click on any of the letters in the other language they open the A in the English countries list.
In the alternative language the alphabetical list at the bottom is entirely missing.

I didn't read your first post correctly, you have multiple fields for country names/a customised table and I assume a customised countries.php to deal with it.
It is impossible for anyone here to guess at the problem/a solution without seeing the code.

Whoever coded that should be fixing it.

The "countries" table is not altered. A single country has two rows in the table - one for each language This is done as there is no other way of localizing country names.
The alphabetical list with links at the bottom is shown correctly in both languages. The problem is that the URLs of this list work only for the Latin characters. When I click on a character that is from the other alphabet it takes me to the "A" section of the english country names. There is no error message in the logs file.

Quote:

---

Originally Posted by vasilt

The "countries" table is not altered. A single country has two rows in the table - one for each language.

---

No it doesn't, not in the vanilla code:

Code:

---

CREATE TABLE countries (
  countries_id int(11) NOT NULL auto_increment,
  countries_name varchar(64) NOT NULL default '',
  countries_iso_code_2 char(2) NOT NULL default '',
  countries_iso_code_3 char(3) NOT NULL default '',
  address_format_id int(11) NOT NULL default '0',
  status tinyint(1) default 1,
  PRIMARY KEY  (countries_id),
  KEY idx_countries_name_zen (countries_name),
  KEY idx_address_format_id_zen (address_format_id),
  KEY idx_iso_2_zen (countries_iso_code_2),
  KEY idx_iso_3_zen (countries_iso_code_3)
) ENGINE=MyISAM;

---

No column for language_id.

Quote:

---

Originally Posted by torvista

No it doesn't, not in the vanilla code:

Code:

---

CREATE TABLE countries (
  countries_id int(11) NOT NULL auto_increment,
  countries_name varchar(64) NOT NULL default '',
  countries_iso_code_2 char(2) NOT NULL default '',
  countries_iso_code_3 char(3) NOT NULL default '',
  address_format_id int(11) NOT NULL default '0',
  status tinyint(1) default 1,
  PRIMARY KEY  (countries_id),
  KEY idx_countries_name_zen (countries_name),
  KEY idx_address_format_id_zen (address_format_id),
  KEY idx_iso_2_zen (countries_iso_code_2),
  KEY idx_iso_3_zen (countries_iso_code_3)
) ENGINE=MyISAM;

---

No column for language_id.

---

It can, but then the same country, listed in two different languages has two different countries_id identifiers. As identified elsewhere in this forum or in Github, this is not supposed to be a problem as the intent is for each unique country to have its own countries_iso_code_3 value. The result here is that two listings would have the same other identifying information to be identified by some other method. (Hopefully not by countries_id as any newly added country may get miscategorized.)

As to the original issue/log and the encoding, it looks like perhaps the alternate country names need to be updated by a sql statement to populate with utf-8ish names. They probably were populated in a previous encoding. Not yet sure why though only three of the associated field are fed for processing when English does through fine unless the encoding of the foreign language is introducing a character that causes a problem.

mc12345678's explanation fits my situation. The whole database is in utf-8. Disregard the original error log.
The alphabetical list at the bottom of the Location/Taxes -> Countries lists all english characters followed by all characters of the other langugage. Every character's associated URL is like: "localhost/store/admin/index.php?cmd=countries&page=F" for the countries starting with "F" for example. The characters associated links are built correctly for both languages. However the URLs of this list work only for the Latin characters. When I click on a character that is from the other alphabet it takes me to the "A" section of the english country names.
What I noticed so far is that the $_GET['page'] value in customers.php, line 11 should be set to the character in the link but this is only the case with latin letters. In the other language it is an empty string.

Quote:

---

Originally Posted by vasilt

mc12345678's explanation fits my situation. The whole database is in utf-8. Disregard the original error log.
The alphabetical list at the bottom of the Location/Taxes -> Countries lists all english characters followed by all characters of the other langugage. Every character's associated URL is like: "localhost/store/admin/index.php?cmd=countries&page=F" for the countries starting with "F" for example. The characters associated links are built correctly for both languages. However the URLs of this list work only for the Latin characters. When I click on a character that is from the other alphabet it takes me to the "A" section of the english country names.
What I noticed so far is that the $_GET['page'] value in customers.php, line 11 should be set to the character in the link but this is only the case with latin letters. In the other language it is an empty string.

---

Based on the above I tried to investigate why customers.php was involved or what was wrong with it. I just realized the line in question is actually in countries.php, line 11.

OK, so now that have the correct line of the correct file. The comment was that at line 11, the current page is getting set to an empty string, specifically the statement is that $_GET['page'] is set to an empty string. I'm curious about the value of $_GET['page'] along the processing of that file before line 11.

One thing that is unfortunate is that the code in line 11 is attempting to evaluate the provided content against the current locale which is generally set differently for each language (I mean sure some countries have multiple languages), so if those values in the database have a different "flavor" they will be filtered out at line 11 because they don't or might not match the current locale.

While that is perhaps interesting and may become an issue for someone, I still really think that it is the database values for the specific countries in question and that they need to be replaced via sql update using utf-8 or utf-8mb4 versions of the names.

In it is indeed the file coutries.php. Sorry about the confusion.

It is very confusing that the value of $_GET['page'] is correct before the execution of the initial line#8

Code:

---

require 'includes/application_top.php';

---

and it is an empty string only for the alternative alphabet after that.

Quote:

---

Originally Posted by vasilt

In it is indeed the file coutries.php. Sorry about the confusion.

It is very confusing that the value of $_GET['page'] is correct before the execution of the initial line#8

Code:

---

require 'includes/application_top.php';

---

and it is an empty string only for the alternative alphabet after that.

---

Again, because the data for those other country names in the database are not utf-8ish. By that I mean they are neither utf-8 nor utf-8mb4, which are the two encodings typically used now by zen Cart.

As previously stated, these could be updated through one or more sql statements. Either the errant entries get converted or a separate query is run to update the data in the database with utf-8ish entry. This can be done via tools, install sql patches.

Quote:

---

Originally Posted by mc12345678

Again, because the data for those other country names in the database are not utf-8ish. By that I mean they are neither utf-8 nor utf-8mb4, which are the two encodings typically used now by zen Cart.

As previously stated, these could be updated through one or more sql statements. Either the errant entries get converted or a separate query is run to update the data in the database with utf-8ish entry. This can be done via tools, install sql patches.

---

personally i was never a fan of this letter pagination idea.

that said, props to @mc12345678 for figuring this bug out.

i would also look at the zones page as i think if the encoding is wrong, that page may not work as well.

best.

Quote:

---

Originally Posted by mc12345678

Again, because the data for those other country names in the database are not utf-8ish. By that I mean they are neither utf-8 nor utf-8mb4, which are the two encodings typically used now by zen Cart.

As previously stated, these could be updated through one or more sql statements. Either the errant entries get converted or a separate query is run to update the data in the database with utf-8ish entry. This can be done via tools, install sql patches.

---

And I'm willing to offer more specific assistance; however, need more details about what capabilities you have available, what data is known about the problem country names, what language is involved, etc...

My thought on the "replacement" style query, in absence of detailed database data and if there is only one other language, then the query could do a substitution of the countries_name record for the country that has the same countries_iso_code_3 but is not the english equivalent. E.g.:

Code:

---

UPDATE countries SET countries_name = 'FOREIGN COUNTRY NEW NAME' WHERE countries_iso_code_3 = 'XYZ' AND countries_name != 'ENGLISH NAME OF COUNTRY';

---

If the consensus is that countries.php line 11 is a bug (or needs additional logic such as a notifier), please PR. Thanks!

Quote:

---

Originally Posted by vasilt

In it is indeed the file coutries.php. Sorry about the confusion.

It is very confusing that the value of $_GET['page'] is correct before the execution of the initial line#8

Code:

---

require 'includes/application_top.php';

---

and it is an empty string only for the alternative alphabet after that.

---

Quote:

---

Originally Posted by swguy

If the consensus is that countries.php line 11 is a bug (or needs additional logic such as a notifier), please PR. Thanks!

---

Based on the report of the above investigation, line 11 in admin/countries.php does not appear to be the issue but rather sanitization executed within admin/includes/application_top.php.

As pointed out by mc12345678, the problem is in the sanitization within admin/includes/application_top.php. When I revert the value of $_GET['page'] back to its original value after the sanitization, everithing works as expected. The table "countries" is utf8 and the column "countries_name" is utf8. I converted the column to BLOB and checked the hex values. They correspond to utf-8. For example the hex code for Zimbabwe in English is: 5A 69 6D 62 61 62 77 65 and in Bulgarian is: D0 97 D0 B8 D0 BC D0 B1 D0 B0 D0 B1 D0 B2 D0 B5. This is exactly the the value of the BLOB field.

Quote:

---

Originally Posted by vasilt

As pointed out by mc12345678, the problem is in the sanitization within admin/includes/application_top.php. When I revert the value of $_GET['page'] back to its original value after the sanitization, everithing works as expected. The table "countries" is utf8 and the column "countries_name" is utf8. I converted the column to BLOB and checked the hex values. They correspond to utf-8. For example the hex code for Zimbabwe in English is: 5A 69 6D 62 61 62 77 65 and in Bulgarian is: D0 97 D0 B8 D0 BC D0 B1 D0 B0 D0 B1 D0 B2 D0 B5. This is exactly the the value of the BLOB field.

---

Actually, you've just done the first step of what may resolve your issue. You've converted a nonbinary datatype to a binary datatype. Next would be to convert that column back to the utf-8ish datatype expected for the column... would expect the issue to be resolved for this problem.

I explicitly changed the encoding of the "countries_name" field of the "countries" table by applying the following two statements in phpMyadmin:
ALTER TABLE countries MODIFY countries_name BLOB
ALTER TABLE countries MODIFY countries_name VARCHAR(64) CHARACTER SET utf8
and this didn't solve the problem

I found a solution though I am not sure if it compromises site security. The file is admin/includes/classes/AdminRequestSanitizer.php and the line is 343. If the regular expression is modified to include the characters of the alternative alphabet, letter pagination works as expected.