Future Japanese Language Pack Status, version 1.5.5 and beyond...

[gernot]

Note to self: from above list, set of files that cannot be overridden at present as I understand it:

A. includes/
1. includes/application_top.php
2. includes/classes/order.php
3. includes/modules/order_total/ot_cod_fee.php
4. includes/modules/pages/account_edit/header_php.php
5. includes/modules/pages/address_book_process/header_php.php
6. includes/modules/pages/contact_us/header_php.php

B. admin/
1. admin/customers.php
2. admin/includes/functions/localization.php
3. admin/includes/menu.css
4. admin/includes/modules/document_general/preview_info_meta_tags.php
5. admin/includes/modules/document_general/preview_info.php
6. admin/includes/modules/document_product/preview_info_meta_tags.php
7. admin/includes/modules/document_product/preview_info.php
8. admin/includes/stylesheet.css

So language pack will be perhaps in three parts:
1. usual langauge translation additions which will enable users to view pages (no changes to core files required, and adds Japanese shipping modules). Anyone wishing to have this support can easily install and uninstall this part.
2. additions that will allow Japanese persons to register with Furigana support (requires changes to core files, several admin files, and database. Adds one Japanese payment module). Anyone using this will have to reconcile the modified files provided (based on standard 1.5.5e) with whatever customizations they have made in the corresponding files. Furthermore, removing the additions will require review of the database (for example, I don't know if it is feasible to remove an address format once some customer information is using that format).
3. further additions that translate the admin section into Japanese (even adding Furigana support requires changes to core and admin files though). Again, anyone using this will have to reconcile the modified files provided with whatever customization they have made in the corresponding files. However, there should not be any database changes here, so removal would be straightforward reversion to the old files (and reconciling any additional customizations made in the meantime which the additions were active).

I hope that is more or less logical.

[mc12345678]

With regards to some of the "overrides". It depends on what is needed to be done, obtained or modified, but the following files have notifiers that can be used by an observer class to modify the data that is present without modifying the files specifically:

2. includes/classes/order.php
4. includes/modules/pages/account_edit/header_php.php
5. includes/modules/pages/address_book_process/header_php.php
6. includes/modules/pages/contact_us/header_php.php


While I haven't looked at what changes may be considered necessary to includes/application_top.php it seems that there would likely be some other way to make the changes necessary to support the additional software/configurations.

[gernot]

Hello mc12345678,
Many thanks for the feedback.
I will have a look at the code in more detail later this week, and study how I could use observer classes to manipulate the data.
As for what functionality in application_top, it is the addition of furigana support.
Here the diff between 1.5.1-jp and 1.5.1 (something similar is what I expected I would need to add to the corresponding 1.5.5e file):

diff -wu -Nr zen-cart-v1.5.1-jp-1/includes/application_top.php zen-cart-v1.5.1/includes/application_top.php
--- zen-cart-v1.5.1-jp-1/includes/application_top.php	2013-01-17 21:46:58.000000000 +0900
+++ zen-cart-v1.5.1/includes/application_top.php	2012-09-18 21:36:42.000000000 +0900
@@ -159,12 +159,3 @@
 if (!isset($_SESSION['customers_ip_address'])) {
   $_SESSION['customers_ip_address'] = $customers_ip_address;
 }
-
-/**
-  * is Furikana nesessary?
-**/
-if (defined('FURIKANA_NECESSARY_COUNTRIES') &&
-  !is_bool(strpos(strtolower(FURIKANA_NECESSARY_COUNTRIES), strtolower($_SESSION['language']))))
-  define('FURIKANA_NESESSARY', true);
-else
-  define('FURIKANA_NESESSARY', false);

I've been going through related Japanese language module threads while trying to figure out the best way to handle the zone for prefectures: English or Japanese or both. Crystal Koi Designs is no longer around it seems, so I cannot access any nifty pull-down jscript that was made available back in the day to enable choice between either language when selecting a prefecture.
I figure it will be logical to have 2 zone files available, one for English (default, maybe check for existing Japanese zones and update them if they exist...), and another that updates this to Japanese if desired (or installs new if no English or Japanese zones found).

The following thread has some (now old) modules for shipping that I will look at. They would need updating, but it seems they might contain size as well as weight breakdown, which is something quite necessary but missing (as far as I can tell) in the 1.5.1-jp shipping modules.
https://www.zen-cart.com/showthread....anguage+module

[gernot]

I've created two zones files for Japan.
The attached ZIP file contains two SQL files for adding zones:

1. An English version of the prefecture names
2. A Japanese version of the prefecture names

For the code, I have chosen the ISO 3166-2:JP code (JP-01 through JP-47), which I discovered while trying to decide on what to put for code.
The great thing is that the default sort order will result in the normal sequence of prefectures seen on Japanese sites, namely from North to South. I haven't seen this in the Japanese localized version, I think the English version would be a great addition for the default settings in ZenCart.
If someone can check that there is nothing wrong with these zones file (I have looked at some add-ons and this seems to be close to what others have uploaded) I will put this in the Zones Add-Ons sub-forum.

[mc12345678]

Hello mc12345678,
Many thanks for the feedback.
I will have a look at the code in more detail later this week, and study how I could use observer classes to manipulate the data.
As for what functionality in application_top, it is the addition of furigana support.
Here the diff between 1.5.1-jp and 1.5.1 (something similar is what I expected I would need to add to the corresponding 1.5.5e file):

diff -wu -Nr zen-cart-v1.5.1-jp-1/includes/application_top.php zen-cart-v1.5.1/includes/application_top.php
--- zen-cart-v1.5.1-jp-1/includes/application_top.php	2013-01-17 21:46:58.000000000 +0900
+++ zen-cart-v1.5.1/includes/application_top.php	2012-09-18 21:36:42.000000000 +0900
@@ -159,12 +159,3 @@
 if (!isset($_SESSION['customers_ip_address'])) {
   $_SESSION['customers_ip_address'] = $customers_ip_address;
 }
-
-/**
-  * is Furikana nesessary?
-**/
-if (defined('FURIKANA_NECESSARY_COUNTRIES') &&
-  !is_bool(strpos(strtolower(FURIKANA_NECESSARY_COUNTRIES), strtolower($_SESSION['language']))))
-  define('FURIKANA_NESESSARY', true);
-else
-  define('FURIKANA_NESESSARY', false);

I've been going through related Japanese language module threads while trying to figure out the best way to handle the zone for prefectures: English or Japanese or both. Crystal Koi Designs is no longer around it seems, so I cannot access any nifty pull-down jscript that was made available back in the day to enable choice between either language when selecting a prefecture.
I figure it will be logical to have 2 zone files available, one for English (default, maybe check for existing Japanese zones and update them if they exist...), and another that updates this to Japanese if desired (or installs new if no English or Japanese zones found).

The following thread has some (now old) modules for shipping that I will look at. They would need updating, but it seems they might contain size as well as weight breakdown, which is something quite necessary but missing (as far as I can tell) in the 1.5.1-jp shipping modules.
https://www.zen-cart.com/showthread....anguage+module

Haven't looked to see where FURIKANA_NECESSARY_COUNTRIES is defined, but if it is defined after the session has started, then the above modification could be incorporated there or in another file loaded after that point. Then application_top would not need to be modified.

[gernot]

Hi,
I get at what you are saying, especially in this case where it is merely a conditional (and extra) define.
For what it is worth, the define is made in the localization SQL file zc_install/sql/mysql_utf8_japanese_localize.sql:

INSERT INTO configuration (configuration_title, configuration_key, configuration_value, configuration_description, configuration_group_id, sort_order, set_function, date_added) VALUES
('ふりがなが必要な国', 'FURIKANA_NECESSARY_COUNTRIES', 'Japanese', 'ふりがなが必要な国名をカンマで区切って入力してください', '5', '100', '', now());

So this adds an admin configuration element (presumably some template files are modified to realize this) into which one can put the list of countries for which this should be true. The code I quoted in the previous comment seems to only compare the session language chosen by the user, not really a good way to decide whether to make kana necessary in my view (unless I am missing something).
OK, so clearly this condition check happens once a session is in progress, and not before. Whether that is a good way to do that is another issue, since in my view the use of kana should be separate from the language, it is a user property after all. Making it a site-wise optional element would be realistic also I suppose, controlled from the admin console.

[gernot]

I've made my own address format addition SQL statement, to insert the new format and set for use by country Japan (countires_id=107), checking for existing ones with a procedure, without modifying the address_format table with unique keys.

# Japan address format addition if not already set

# (1) simple method based on Japanese localization in 1.5.1 assuming 8 will be the next available record in address_format (7 existing formats in 1.5.5e by default)

INSERT INTO address_format VALUES (8, '$firstname $lastname$cr$postcode$cr$state$city$cr$streets$cr$country$cr$telephone$cr$fax','$statename $city');
UPDATE countries SET address_format_id=8 WHERE countries_id=107;

# (2) more complex method that uses a precedure to insert a new address format for Japan only if the values of the two fields do not yet exist
#     Note this does not modify the address_format table to use unique keys (as would be required by DUPLICATE KEYS or IGNORE statements)
#     The update to the coutries table is done at the same time, using the values of the address format

DELIMITER $$
DROP PROCEDURE IF EXISTS insert_jp_address_format $$
CREATE PROCEDURE insert_jp_address_format()
BEGIN
IF NOT EXISTS (SELECT * FROM address_format WHERE address_format = '$firstname $lastname$cr$postcode$cr$state$city$cr$streets$cr$country$cr$telephone$cr$fax' AND address_summary = '$statename $city') THEN
    INSERT INTO address_format (address_format,address_summary) VALUES ('$firstname $lastname$cr$postcode$cr$state$city$cr$streets$cr$country$cr$telephone$cr$fax','$statename $city');
    UPDATE countries SET address_format_id=(SELECT address_format_id FROM address_format WHERE address_format = '$firstname $lastname$cr$postcode$cr$state$city$cr$streets$cr$country$cr$telephone$cr$fax' AND address_summary = '$statename $city') WHERE countries_id=107;
END IF;
END $$
DELIMITER ;

CALL insert_jp_address_format();

# Utility/Reference:
#
# Direct update:
# update countries set address_format_id=(select address_format_id from address_format where address_format = '$firstname $lastname$cr$postcode$cr$state$city$cr$streets$cr$country$cr$telephone$cr$fax' and address_summary = '$statename $city') where countries_id=107;
#
# To get back to original (address_format_id=1):
# update countries set address_format_id=(select address_format_id from address_format where address_format = '$firstname $lastname$cr$streets$cr$city, $postcode$cr$statecomma$country' and address_summary = '$city / $country') where countries_id=107;

[gernot]

After reading up on notifiers/observers, I tried to find a notifier to attach in include/application_top.php. However, there does not appear to be one.
On the other hand, it seems I could use includes/extra_configures for exactly this purpose?
So I made a new file as follows, using the (slightly corrected spelling) logic from the Japanese localization for now. I don't know what to put in the header for the package, constants does not seem right, so I made up "langauges" until I learn what would be best:

<?php
/** 
 * define for use with Japanese: furigana support based on language chosen
 *
 * @package languages
 * @copyright Copyright 2003-2005 Zen Cart Development Team
 * @license http://www.zen-cart.com/license/2_0.txt GNU Public License V2.0
 * @version $Id: media_manager.php 2842 2006-01-13 06:21:11Z drbyte $
 */
/**
  * is Furigana necessary?
**/
if (defined('FURIGANA_NECESSARY_COUNTRIES') &&
  !is_bool(strpos(strtolower(FURIGANA_NECESSARY_COUNTRIES), strtolower($_SESSION['language']))))
  define('FURIGANA_NECESSARY', true);
else
  define('FURIGANA_NECESSARY', false);

If this is correct, then this extra file would be added to the Japanese language pack now in preparation, and application_top.php has no changes.
For the code where FURIGANA_NECESSARY is used, I don't know how to handle that yet, but if it can be done by notifiers that would of course be great.

As an aside, the Japanese localization mades some changes to the database fields too for kana support obviously, but also for privacy of customer's telephone number:

# add telephone number to address, but remove from private information
ALTER TABLE address_book ADD COLUMN entry_telephone varchar(32) NOT NULL;
ALTER TABLE address_book ADD COLUMN entry_fax varchar(32);
ALTER TABLE orders ADD COLUMN delivery_telephone varchar(32);
ALTER TABLE orders ADD COLUMN delivery_fax varchar(32);
ALTER TABLE orders ADD COLUMN billing_telephone varchar(32);
ALTER TABLE orders ADD COLUMN billing_fax varchar(32);
ALTER TABLE orders ADD COLUMN customers_fax varchar(32);
ALTER TABLE customers CHANGE customers_telephone customers_telephone VARCHAR(32);
ALTER TABLE orders CHANGE customers_telephone customers_telephone VARCHAR(32);

# Furigana support added
ALTER TABLE address_book ADD entry_firstname_kana     varchar(32) NOT NULL default '';
ALTER TABLE address_book ADD entry_lastname_kana      varchar(32) NOT NULL default '';
ALTER TABLE customers    ADD customers_firstname_kana varchar(32) NOT NULL default '';
ALTER TABLE customers    ADD customers_lastname_kana  varchar(32) NOT NULL default '';
ALTER TABLE orders       ADD customers_name_kana      varchar(64) NOT NULL default '';
ALTER TABLE orders       ADD delivery_name_kana       varchar(64) NOT NULL default '';
ALTER TABLE orders       ADD billing_name_kana        varchar(64) NOT NULL default '';

Is this revertable at all, if one decides to use Japanese language with kana support (and Japan-centric privacy setting of telephone number)? Once data is entered, I presume this is not removable again if one uninstalls the language pack. It would be up to the user to decide how to dispose of the data first?

[gernot]

Further work, nothing with notifiers although I think part of the changes for furigana and address book change support could be done that way.

1. Custom files created for following files - to be placed in override directories (either generic common override directories, or custom template directories, as permitted)
   
   1. admin/customers.php: furigana support/address changes. Not overridable?
   2. includes/classes/order.php: since the order deals with addresses and mysql queries, I am unable to understand at present if changes can be make later with notifiers here. Overridable?
   3. includes/modules/checkout_new_address.php: furigana support/address changes. Overridable?
   4. includes/modules/create_account.php: furigana support/address changes. Overridable?
   5. includes/modules/order_total/ot_cod_fee.php: addition of Japan shipping modules. Overridable?
   6. includes/modules/pages/account_edit/header_php.php: furigana support/address changes. Overridable?
   7. includes/modules/pages/address_book_process/header_php.php: furigana support/address changes. Overridable?
   8. includes/modules/pages/contact_us/header_php.php: for privacy policy. Overridable?
   9. includes/templates/template_default/templates/tpl_account_edit_default.php: furigana support/address changes. Overridable OK.
   10. includes/templates/template_default/templates/tpl_contact_us_default.php: privay policy. Overridable OK.
   11. includes/templates/template_default/templates/tpl_modules_address_book_details.php: furigana support/address changes. Overridable OK.
   12. includes/templates/template_default/templates/tpl_modules_checkout_new_address.php: furigana support/address changes. Overridable OK.
   13. includes/templates/template_default/templates/tpl_modules_create_account.php: furigana support/address changes. Overridable OK.
2. Privacy policy for Japan has an addition to the admin configuration. I don't know if I can put that in an override somewhere, so for now I will make the definition in the same SQL file as the zones and address book, kana definition and address format additions.
   

   # privacy conditions
   INSERT INTO configuration (configuration_title, configuration_key, configuration_value, configuration_description, configuration_group_id, sort_order, set_function, date_added) VALUES ('Privacy policy confirmation for contact', 'DISPLAY_CONTACT_US_PRIVACY_CONDITIONS', 'true', 'In contact us page, display confirmation for privacy policy. <div style="color: red;">The guidelines for protection of personal information are displayed, in compliance with the Act on the Protection of Personal Information effective since April 1, 2005 and the Amended Act effective from May 30, 2017.</div>', '11', '3', 'zen_cfg_select_option(array(\'true\', \'false\'), ', now());

3. Privacy policy extended to contact_us page in Japanese as part of custom files above. But since this also should apply to the English page, I added the definitions from login.php to contact_us.pp in includes/languages/english/. The modified file can go into a custom template directory.

So now the work is just about done at a basic level:

• pure language translations without customizing for Japan
• addition of address format, zones, shipping modules for Japan
• addition of address book changes, furigana (kana) support, privacy policy support additions

No changes made to stylesheet or CSS (yet), something which is also done in the Japanese localization (something with EZ pages also, but I haven't looked at that yet, or if it is even desirable).

I hope to be able to test on my own site in the next few days, debugging as required (still trying to figure out definitively what can be overridden, and in which way - extension via generic override directory, using custom template, or replacement of core files, as the case may be).
Then I will put up the combined files for others to experiment on. I am pretty sure some people with experience will give feedback. And possible some thing, like the privacy policy addition to contact_us.php, might be desirable as a core feature in Zen Cart.

[gernot]

Previous post appears to be still in moderation - this post is testing the basic language module that was completed up till now.
Two things of note:

1. Charset in japanese.php (normal and admin): I had to put specific UTF-8 before generic ja_JP, else a couple of variables would not display correctly. These were things like %a for the short dayname, or %s for the month name (in Japanese). Despite looking at various threads, and adding files in extra_configures to control the charset of the DB even more, no changes were achieved. I don't know what causes this, but setting ja_JP.UTF-8 as the first item in the array solves it (all files are saved in UTF-8 format, and since this is 1.5.5e upgraded from 1.5.5d the DB and system has only ever used UTF-8 as far as I can tell).
   

   $locales = array('ja_JP.UTF-8', 'ja_JP', 'ja', 'Japanese_Japan.932');

2. I cannot figure out how to control which language displays in the browser: Initially, when I had the default language to be English, and the Store default language setting also at default, then I only got English displaying in the browsers on (English) linux and (Japanese) Windows, even though the browser default in Japanese Windows is Japanese. Setting the default language in the store to Japanese immediately changed the browser display on linux and Windows to Japanese. Setting default store language back to English, but setting the store default language setting to "browser" had no effect, still Japanese on linux and English on Japanese windows. Very odd. I read that browser language can only be explicitly set on Windows; on linux and MacOS systems the default OS language is used. What should I expect to be able to control? Ideally, I would like to be able for a user to switch between languages - maybe that is would require a separate plugin?

Some sideboxes and other defines are apparently not yet translated - there were many more in the 1.5.1-jp japanese.php, but I removed everything that did not have a corresponding entry in 1.5.5e english.php, so now I guess I need to add them back one by one, checking that the names have not changed (or been removed).
Other than that, I am pretty happy that there are no PHP errors or debugs to do (strict error display on-screen enabled for now).
I also haven't put in the database changes and custom files for working in Japan, or activated the Japanse shipping modules.