Future Japanese Language Pack Status, version 1.5.5 and beyond...

[gernot]

I'm also looking at adding Japanese language support to my (experimental) online shop here in Tokyo, which runs 1.5.5d perfectly well in English.
I've looked at the Japanese ZenCart 1.5.1 language files, extra modules for shipping and payment, and the localization script.
Since the Japanese Zencart is---I think---a pure localization, for support of multiple languages I surmise that parts of the localization script (and parts of other files that simply replace English terminology with Japanese) should be removed and put into language files and templates.
Is that the correct approach to take?

[mc12345678]

Welcome to Zen Cart. This FAQ somewhat describes the LANGUAGE system, though does not heavily go into the "override" portion of the system: https://www.zen-cart.com/content.php...-are-they-used
The override system is a little more explained here: https://www.zen-cart.com/content.php...verride-system
And there is a file that can be downloaded from the plugins area to be used/viewed on your local computer to better understand the override system: https://www.zen-cart.com/downloads.php?do=file&id=1393

Basically, it is recommended that words not be incorporated into the code, instead that defined variables be used and other tools to support language related display such as in addresses that the house number appear on the "correct" side of the street name, or that a quantity appears in the correct location of the "sentence".

[gernot]

Thank you mc12345678, I will check out the cheatsheet, though I notice it only is applicable correctly up to 1.5.5b. Since I began at 1.5.5c before upgrading to 1.5.5d, I will need to some to check what differences there might be.
I spent about 8 months looking into Zencart before deciding on the international 1.5.5c (at that time the latest) rather than the old Japanese 1.5.1(something), and run only in English at the moment.
I contacted the Facebook page of the Zencart Japan team last year, no reply at all despite my offer to work with them to update the Japanese version to 1.5.5c (and now 1.5.5d). The website also appears dead, and the forums do not function (at least search does not) when looking for modules.
Since the main changes appear to be not that difficult, in terms of name and address changes, my main problems are conceptual: does making localization changes to name and address formats mean that all users in all languages will be forced to use a format, or can one easily separate sets of fields to use per language (not that that is a major problem if one can only use a single type of format for names and addresses).
The only modules added to the Japanese 1.5.1 Zencart version appear to be addition of credit card and convenience store payments using Remise (a payment brand in Japan), and the addition of two express mail delivery services (Sagawa and Kuroneko Yamato) in the shipping modules. I am not sure if cod_table (pay on delivery) is a Japanese addition or not, and need to check the modules in 1.5.5d actually to compare.
So as far as I am concerned, since I am quite fluent in Japanese, with some cooperation with like-minded people here on this forum we could prepare a Japanese language and module update for 1.5.5d, assuming programming changes in the modules for updating from 1.5.1 to 1.5.5d is not too difficult (or necessary even).
I'll get spend some time on the cheatsheet this weekend and see if I understand whether all or some of the Japanese localization can be handled by the language (no extension of functionality, just translations) and override system (additions/modifications of functionality - but it seems language files also fall under override). One thing I am really eager to understand is whether localization falls within the ambit of the override system.
Many thanks in advance,
Gernot Hassenpflug

[gernot]

After revisiting several related threads on Japanese language packs (a misnomer I find) dating back to 2006 (version 1.3.8 or so), and the contributions therein, I conclude that there are three major areas included in what could be referred to a Japanese language pack-age.
I don't know if I got the breakdown correct, but here goes.

1. Japanese language definitions for site users/visitors/customers.
   This would be standard language-package territory.
2. Japanese language and other related definitions for site administrators.
   I guess this is standard language pack territory also, but perhaps not.
3. Core and module changes required to operate in Japan.
   This includes email templates (which if I understand correctly do not fall under teh template hierarchy), name definitions, addition of Japan-centric modules, and changes to some modules (such as seemed to be the case with Paypal back in 2006) to work in Japan. If I understand correctly, this impacts the entire Zencart system, regardless of language chosen by users and/or admins.

I'll spend some time checking if the 1.5.1 Japanese distribution modules have changes in them compared to the 1.5.1 and 1.5.5d standard versions. More time needed...

[gernot]

After looking at the Japanese 1.5.1 paypal files and the current 1.5.5e files, I don't see anything customized in the Japanese ones that hasn't been already put into the 1.5.5e files. There has certainly been a heck of a lot of work in the time since 1.5.1 by Dr. Byte!
In particular, I noticed that the differences do not show up any language-specific text, which is great.
On the other hand, the 3 shipping modules (nittsu.php, sagawaex.php, yamato.php) (and their related language and class files) do have specific Japanese-language strings in them. Presumably, for Anglicization, the best would be to make new defines for these texts, and override them with language-specific overrides; or perhaps one could just override the entire module and edit the texts in-situ as needed. Will have to test that.
In any case, I am not sure if the shipping modules will work out of the box in 1.5.5e, nor whether they are still current against the latest Nittsu, Sagawa and Yamato APIs. Again, will need to test that.

A related question would be what happens when Paypal, Nittsu, Sagawa, Yamato or whoever makes changes to their interface that breaks the modules. Fixing such issues could be quite a bit of work, but I assume any to modules would be released individually here on this site (for example I am happy to share any work I do for my site with the community).

[gernot]

I have split work into four parts:
(1) DB changes: leave till later (address format definition(s), name order, hiragana/katakana name field support, etc.)
(2) module changes: addition of Japanese shipping modules to 1.5.5e. This seems to require minimal changes to the declaration function and use of zone_country_id statements (apart from language-specifics).
(3) language addition of Japanese.
(4) template additions/changes (email, etc., as required).

I have started work on making the language files.
First, the differences to includes/languages/english.php between 1.5.1-jp-1 and 1.5.5.e:

4c4
<  * @copyright Copyright 2003-2011 Zen Cart Development Team
---
>  * @copyright Copyright 2003-2016 Zen Cart Development Team
7c7
<  * @version $Id: english.php 19690 2011-10-04 16:41:45Z drbyte $
---
>  * @version $Id: Author: DrByte  Tue Jan 5 15:06:15 2016 -0500 Modified in v1.5.5 $
19c19,20
<   @setlocale(LC_TIME, 'en_US');
---
>   $locales = array('en_US', 'en_US.utf8', 'en', 'English_United States.1252');
>   @setlocale(LC_TIME, $locales);
139d139
<   define('BOX_BBINDEX', 'Forum');
271,274c271,274
<   define('PREVNEXT_BUTTON_FIRST', '<<FIRST');
<   define('PREVNEXT_BUTTON_PREV', '[<< Prev]');
<   define('PREVNEXT_BUTTON_NEXT', '[Next >>]');
<   define('PREVNEXT_BUTTON_LAST', 'LAST>>');
---
>   define('PREVNEXT_BUTTON_FIRST', '«FIRST');
>   define('PREVNEXT_BUTTON_PREV', '[« Prev]');
>   define('PREVNEXT_BUTTON_NEXT', '[Next »]');
>   define('PREVNEXT_BUTTON_LAST', 'LAST»');
479a480
>   define('WARNING_PRODUCT_QUANTITY_ADJUSTED', 'Quantity has been adjusted to what is in stock. ');
509a511
>   define('TEXT_AUTHORIZATION_PENDING_CHECKOUT', 'Checkout Unavailable - Approval Pending');
586a589
>   define('PAYMENT_JAVASCRIPT_DISABLED', 'We could not continue with checkout as Javascript is disabled. You must enable it to continue');
602a606,607
>   define('ERROR_TEXT_COUNTRY_DISABLED_PLEASE_CHANGE', 'Sorry, but we no longer accept billing or shipping addresses in "%s".  Please update this address to continue.');
> 
608a614
>   define('ERROR_DATABASE_MAINTENANCE_NEEDED', '<a href="http://www.zen-cart.com/content.php?334-ERROR-0071-There-appears-to-be-a-problem-with-the-database-Maintenance-is-required" target="_blank">ERROR 0071: There appears to be a problem with the database. Maintenance is required.</a>');

I therefore modified/added corresponding fields to the includes/languages/japanese.php, so that the differences to the 1.5.1-jp-1 module are as follows:

20c20,21
<   @setlocale(LC_TIME, 'ja_JP.UTF-8');
---
>   $locales = array('ja_JP', 'ja_JP.UTF-8', 'ja', 'Japanese_Japan.932');
>   @setlocale(LC_TIME, $locales);
292,295c293,296
< define('PREVNEXT_BUTTON_FIRST', '<<最初');
< define('PREVNEXT_BUTTON_PREV', '[<< 前へ]');
< define('PREVNEXT_BUTTON_NEXT', '[次へ >>]');
< define('PREVNEXT_BUTTON_LAST', '最後>>');
---
> define('PREVNEXT_BUTTON_FIRST', '«最初');
> define('PREVNEXT_BUTTON_PREV', '[« 前へ]');
> define('PREVNEXT_BUTTON_NEXT', '[次へ »]');
> define('PREVNEXT_BUTTON_LAST', '最後»');
498a500
>   define('WARNING_PRODUCT_QUANTITY_ADJUSTED', 'ご注文個数を在庫中数量に変えました。 ');
528a531
>   define('TEXT_AUTHORIZATION_PENDING_CHECKOUT', '現在チェックアウトを利用できません - 承認手続き中');
605a609
>   define('PAYMENT_JAVASCRIPT_DISABLED', 'Javascriptが無効になっているためチェックアウトの手続きを進めません。Javascriptを有効にしてください。');
621a626,627
>   define('ERROR_TEXT_COUNTRY_DISABLED_PLEASE_CHANGE', 'お手数ですが、"%s"の支払い先または郵送先を受け入れなくなりました。この住所を更新してください。');
> 
627a634
>   define('ERROR_DATABASE_MAINTENANCE_NEEDED', '<a href="http://www.zen-cart.com/content.php?334-ERROR-0071-There-appears-to-be-a-problem-with-the-database-Maintenance-is-required" target="_blank">ERROR 0071: データベース問題が発生しているようです。メンテナンスが必要です。</a>');

Some contents of japanese.php are still in English in 1.5.1-jp-1, I may translate those later, but first I wanted to update to 1.5.5e standard, based on english.php.

Next, I'll work on the contents of includes/languages/Japanese.

Note: I keep getting token expired on my submissions even after Ctrl-R, pressing the reload icon in the URL bar of Friefox. Ctrl-F5 does not help either. I tried logging out and loggin in again now.

[gernot]

Files that changes in includes/languages/english top-level directory between 1.5.1-jp-1 and 1.5.5e. These will need updating in the languages/japanese:

1. button_names.php
2. checkout_payment.php
3. checkout_success.php
4. create_account_success.php
5. credit_cards.php
6. discount_coupon.php
7. download_time_out.php
8. email_extras.php
9. gv_faq.php
10. gv_redeem.php
11. header.php
12. meta_tags.php
13. password_forgotten.php
14. popup_coupon_help.php
15. reviews.php
16. shopping_cart.php
17. specials.php

Furthermore, a file that is only in 1.5.5e and not in 1.5.1-jp-1:
page.php

After updating the Japanese files I should put them somewhere accessible, even though a work in progress.

I will tackle the lower level directories after that, here the breakdown (no idea of differences yet):

1. classic
2. extra_definitions
3. html_includes
4. images
5. modules

Furthermore, a new directory only in 1.5.5e and not in 1.5.1-jp-1:
responsive_classic

[gernot]

Translations
=========

Translation of language files is completed.
First, the 18 files in the previous post were handled.
Note: in password_forgotten.php Japanese-specific definition EMAIL_GREET is translated ("様", read as "sama", to follow the name of the person addressed).
This is used in checkout_process.php as well (no update needed there).
In any case, definition additions and other changes to the core code I will tackle later.

Next, I took on the sub-directories in languages/japanese/.

A. Classic

One file updated:
1. header.php

B. extra_definitions

No files updated.
New in 1.5.5e is contents of responsive_classic/
Contains one file to translate:
1. product_free_shipping.php

C. html_includes

1. define_main_page.php
3. define_site_map.php

In classic/
3. define_site_map.php

New in 1.5.5e is contents of responsive_classic/
Most of those 12 files are placeholders at present, so I only translated a handful of items.

D. Images

No files to update.

E. modules

E.1 order_total/

One file updated:
1. ot_cod_fee.php

E2. payment/

Following files updated:
1. kuroneko_at_payment.php
2. authorizenet_aim.php
3. authorizenet.php
4. paypaldp.php
5. paypalwpp.php
Note the Paypal files are in English still.

Following file changed:
6. linkpoint_api.php -> firstdata_hco.php
Translation update was minimal.

Following files brought forward to 1.5.5e:
7. remisecps_zen.php
8. remise_zen.php
Note that the above 2 files also exist und languages/english/modules/payment in 1.5.1-jp,
so I brought those forward too and translated them, as they had been left completely in Japanese.

Following file is only in 1.5.5e:
9. payeezyjszc.php
So I copied it to lanauges/japanese/modules/payment
Note that this is untranslated for now, but also none of the Paypal modules were translated in 1.5.1-jp either,
so this is something to do later (having done the remise modules I am confident I can handle that).

H.3 shipping/

The following files were updated:
1. nittsu.php
2. sagawaex.php
3. yamato.php
All three need to be in the english version too, so I copied those from 1.5.1-jp (sawagaex.php is sagawa.php in the english subdir).
While I updated these language-wise, there are bigger problems:
(1) Nittsu no longer offers this service (since 2010 handled by Japanese Post Office "Yupack" service).
(2) all three need updates to the pricing, time service, and region definitions in the corresponding claculator code:
./includes/classes/_nittsu.php
./includes/classes/_sagawaex.php
./includes/classes/_yamato.php

Based on the above modules, a new module for Yupack service should be fairly simple to code later.

F. New in 1.5.5e: responsive_classic

Three files to translate, based on those in the other (classic and default) templates.
1. icon_names.php
2. index.php
3. other_images_names.php

Module Changes
============

I updated the Japanese-specific payment and shipping modules in two places:
- constructor statement changed from "function modulename() {" to "function __construct() {"
- (int) in front of the $order->... statement

1. includes/modules/payment/remisecsp_zen.php
2. includes/modules/payment/remise_zen.php
3. includes/modules/shipping/nittsu.php
4. includes/modules/shipping/sagawaez.php
5. includes/modules/shipping/yamato.php

Button images
===========

I copied across the entire button directory of Japanese images:
includes/templates/template_default/buttons/japanese
There do not appear to be any button changes between 1.5.1-jp and 1.5.5e.

[gernot]

Sumamry so far:
The above work was purely on the user side of the site, comparing changes between the English 1.5.1 and 1.5.5e, and then updating the Japanese language files 1.5.1 to 1.5.5e standard (since they do not exist in 1.5.5e).
The exception here is the Japanese-version module and class files, which were updated to 1.5.5e standard by inspection of other 1.5.5e modules (and note that the shipping modules need updating, and the remise payment module is untested for present use).

To actually make use of all of this, the following changes are required:

(1) updates to the admin/ folder. If not interested in localization, probably the admin/includes/language/japanese/ and admin/includes/languages/japanese/images/buttons/ could be left out. But a lot of customization has been done in 1.5.1-jp compared to 1.5.1, to support use in Japan. A diff between 1.5.1-jp and 1.5.1 showed up 97 files with changes. Of these 80 were in admin/includes/languages/japanese/ and likely can be skipped for now. Some naviagation sidebox has been added, so japanese.php and english.php include some changes for that.
The following files have changes, but do not appear to be directly related to Japanese:
- admin/includes/menu.css
- admin/includes/modules/document_general/preview_info_meta_tags.php
- admin/includes/modules/document_general/preview_info.php
- admin/includes/modules/document_product/preview_info_meta_tags.php
- admin/includes/modules/document_product/preview_info.php
- admin/includes/stylesheet.css
The remaining files with large changes are:
- admin/customers.php: furigana support among other things
- admin/includes/functions/localization.php: seems to be general updates/corrections, not related to Japanese use

(2) updates to the includes/ folder. Some work on modules was handled earlier and is not repeated here.

- includes/application_top.php: furigana support
- includes/functions/functions_email.php: something to do with charset (maybe general upgrade)
- includes/modules/checkout_new_address.php: furigana support
- includes/modules/create_account.php: furigana support
- includes/modules/order_total/ot_cod_fee.php: new shipping modules support
- includes/modules/pages/account_edit/header_php.php: furigana support
- includes/modules/pages/address_book_process/header_php.php: furigana support
- includes/modules/pages/contact_us/header_php.php: add display of privacy conditions (general upgrade?)
- includes/templates/classic/css/stylesheet.css: use of EZpages added
- includes/templates/template_default/templates/tpl_account_edit_default.php: furigana support
- includes/templates/template_default/templates/tpl_contact_us_default.php: add display of privacy conditions (general upgrade?)
- includes/templates/template_default/templates/tpl_modules_address_book_details.php: furigana support
- includes/templates/template_default/templates/tpl_modules_checkout_new_address.php: furigana support
- includes/templates/template_default/templates/tpl_modules_create_account.php: furigana support

(3) updates to the zc_install/ folder and actual database changes. The zc_install/includes/languages/japanese/ files (11 of them) are probably not needed at this stage. english.php and japanese.php again have the customized navigation sidebox additions, as does zc_install/includes/templates/template_default/sideboxes/navigation.php.
Other small changes are in:
- install.txt: info only
- zc_install/includes/templates/template_default/css/stylesheet.css
- zc_install/includes/templates/template_default/templates/about_zencart_ja.html
- zc_install/includes/templates/template_default/templates/gpl_license.japanese.html
- zc_install/includes/templates/template_default/templates/index_default.php: Japanese version
- zc_install/includes/templates/template_default/templates/license_default.php: Japanese version
Japanese demo data (not needed at thsi stage) is installed with:
- zc_install/demo/mysql_demo.sql
While the meat of the Japanese installation is in the following files:
- zc_install/includes/application_top.php: language setting (for installer only?)
- zc_install/includes/classes/installer.php: read localized sql (see below)
- zc_install/includes/modules/pages/store_setup/header_php.php: store country and zone set to Japan
- zc_install/sql/mysql_utf8_japanese_localize.sql: carries out the actions below
1. set display of price with tax
2. set default country Japan (107)
3. set privacy statement/conditions with Japanese law in mind
4. add a newe address format for use in Japan
5. set currency conversion defaults for USD, Euro, GBP, CAD, AUD and add JPY as standard
6. set language as Japanese
7. set use of new address format when country is Japan
8. add prefectures as zone(s?)
9. set prefecture zone(s) in same geo_zone Japan (107)
10. set tax_rate (update needed)
11. set geo_zone name/desc as tax zone
12. set tax class
13. set status words for new orders (addition, it seems, maybe use these in Japanese orders separate from English ones?)
14. add Telephone address to address book, remove from personal information (customer, orders) for privacy reasons
15. add kana (furigana) support to address book, customer, orders
16. set furigana required for coutnry Japan
17. translate general admin setting groups
18. translate mail sending (admin)
19. translate product_type layout (admin)
20. add feature for optional text to be added to product (not sure if this is needed, but it would be in some product-related templates then)

The job now is two-fold, as I see it:

First. in the above files, the logic of adding Japan-specific support in various files and in the database (with the localized sql file) needs to be added to the corresponding 1.5.5e files. At the same time, and non-Japanese-specific upgrades should be taken into account (not all changes may need to be carried forward). In particular, for now localization is not my goal, so only addition of Japan-centric information should be carried out.
This would involve comparing 1.5.1 international and 1.5.5e files, quite a job since there are many changes.

Second, any language-specific files would need to be added in 1.5.5e. diff does not work so great comparing translations, so probably the best (though tedious) way would be to to a file-by-file comparison between 1.5.1-jp and the 1.5.5e english files. No doulbt some files are no longer there, others might be new, or the functionality moved elsewhere.

I am not sure if overrides of definitions are required in any of the above, as opposed to only Japanese language additions in the language sub-folder. Even templates seem to be in this sub-folder.
However, for admin use, where I seem to recall overrides to not work, the support need for furigana would mean some changes to existing admin files. How to do this I am not quite sure, confused after too much reading of different topics...

[gernot]

Comments to self:

1. After reading more on the Wiki about Admin functions, it is clear (as I suspected) that a large part of the mysql_utf8_japanese_localize.sql is automating things that can be done from the admin console. Thus this can be skipped in a language pack.
2. According to overrides documentation, includes/* files can be extended in includes/extra_datafiles/*, but not sure if that means additions, or entire overrides. Also, this does not seem to be an override directory, no language or template custom name given. For Japanese we need changes to application_top.php.
3. No idea how to handle admin/customers.php, where furigana support addition is required. admin/includes seems OK for language specifics.