bug#28705: 25.3; [PATCH] Change prefered coding system for Japanese language environment

bug#28705: 25.3; [PATCH] Change prefered coding system for Japanese language environment

[Yasuhiro KIMURA]

[-- Attachment #1: Type: Text/Plain, Size: 380 bytes --]

In current implementation, 'japanese-iso-8bit' is used as prefered
coding system when you select 'Japanese' as language environment on
Unix system. But nowaday a lot of Japanese Unix users adopt
'ja_JP.UTF-8' as locale setting. So I propose to change prefered
coding system from 'japanese-iso-8bit' to 'utf-8'. And while I'm on
it, comment is fixed to fit current implementation.

[-- Attachment #2: 0001-Change-prefered-coding-system-for-Japanese-language-.patch --]
[-- Type: Text/X-Patch, Size: 1288 bytes --]

From 2c80312989659375f82d7eefd333e5ac77663f4b Mon Sep 17 00:00:00 2001
From: Yasuhiro KIMURA <yasu@utahime.org>
Date: Wed, 4 Oct 2017 19:09:21 +0900
Subject: [PATCH] Change prefered coding system for Japanese language
 environment.

* lisp/language/japan-util.el (setup-japanese-environment-internal):
  Use utf-8 as prefered coding system instead of japanese-iso-8bit
  when system is not windows. And while I'm at it, fix comment to fit
  current implementation.
---
 lisp/language/japan-util.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/language/japan-util.el b/lisp/language/japan-util.el
index 988b925409..fb4b408cd3 100644
--- a/lisp/language/japan-util.el
+++ b/lisp/language/japan-util.el
@@ -29,11 +29,11 @@
 
 ;;;###autoload
 (defun setup-japanese-environment-internal ()
-  ;; By default, we use 'japanese-iso-8bit for file names.  But, the
+  ;; By default, we use 'iso-2022-jp for default coding system.  But, the
   ;; following prefer-coding-system will override it.
   (if (memq system-type '(windows-nt ms-dos cygwin))
       (prefer-coding-system 'japanese-shift-jis)
-    (prefer-coding-system 'japanese-iso-8bit))
+    (prefer-coding-system 'utf-8))
   (use-cjk-char-width-table 'ja_JP))
 
 (defconst japanese-kana-table
-- 
2.14.2

bug#28705: 25.3; [PATCH] Change prefered coding system for Japanese language environment

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

Thanks for the bug report. Would you please discuss the practical 
advantages of the proposed patch?

Is the patch is for Japanese users who do not specify a locale via 
environment variables like LC_ALL? If so, how do they specify their 
preference to Emacs now, and how would this change affect behavior for 
them? And how do users specify their coding preference to other 
programs, such as "cat" and "diff"?

I have the impression that Emacs users in Japan are gradually switching 
from EUC-JP to UTF-8. Is there some way to verify this? I know of 
website surveys like 
<https://w3techs.com/technologies/history_overview/character_encoding/ms/y>, 
which says that since 2010 UTF-8 has grown from 51% to 90% of websites 
whereas EUC-JP has shrunk from 0.7% to 0.3% worldwide. However, this 
issue is more about what encodings are used by keyboards and terminal 
displays in Japan (typically these are programs in windowing systems 
these days, as GNU/Linux consoles cannot handle enough characters to 
support Japanese). Is there some way to estimate this?

A minor point: a change like this should be noted in the NEWS file, and 
the commit message should use the usual Emacs style. I'm attaching a 
copy of your patch with a revised commit message, along with another 
patch to mention this change in NEWS and to change affected 
documentation. I will ask for comment about this proposed change on the 
emacs-devel mailing list.


[-- Attachment #2: 0001-Change-preferred-Japanese-coding-system-to-UTF-8.patch --]
[-- Type: text/x-patch, Size: 1319 bytes --]

From 8484dc66eee6b78771fa3332e63ddec9d5022cd4 Mon Sep 17 00:00:00 2001
From: Yasuhiro KIMURA <yasu@utahime.org>
Date: Fri, 6 Oct 2017 13:07:41 -0700
Subject: [PATCH 1/2] Change preferred Japanese coding system to UTF-8

* lisp/language/japan-util.el (setup-japanese-environment-internal):
Use utf-8 as preferred coding system instead of japanese-iso-8bit when
system is not MS-Windows. And while I'm at it, fix comment to fit current
implementation.  (Bug#28705)
Copyright-paperwork-exempt: yes.
---
 lisp/language/japan-util.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/language/japan-util.el b/lisp/language/japan-util.el
index 988b925409..fb4b408cd3 100644
--- a/lisp/language/japan-util.el
+++ b/lisp/language/japan-util.el
@@ -29,11 +29,11 @@
 
 ;;;###autoload
 (defun setup-japanese-environment-internal ()
-  ;; By default, we use 'japanese-iso-8bit for file names.  But, the
+  ;; By default, we use 'iso-2022-jp for default coding system.  But, the
   ;; following prefer-coding-system will override it.
   (if (memq system-type '(windows-nt ms-dos cygwin))
       (prefer-coding-system 'japanese-shift-jis)
-    (prefer-coding-system 'japanese-iso-8bit))
+    (prefer-coding-system 'utf-8))
   (use-cjk-char-width-table 'ja_JP))
 
 (defconst japanese-kana-table
-- 
2.13.6


[-- Attachment #3: 0002-Mention-Japanese-change-in-documentation.patch --]
[-- Type: text/x-patch, Size: 1398 bytes --]

From fb3e73de90a7eb90ac979deca1f8bd32871bb608 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 6 Oct 2017 13:48:08 -0700
Subject: [PATCH 2/2] Mention Japanese change in documentation

---
 doc/emacs/mule.texi | 2 +-
 etc/NEWS            | 4 ++++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/doc/emacs/mule.texi b/doc/emacs/mule.texi
index 8a4d91f80b..d3f54f65c9 100644
--- a/doc/emacs/mule.texi
+++ b/doc/emacs/mule.texi
@@ -405,7 +405,7 @@ Language Environments
 coding system instead.  For example, if the locale @samp{ja_JP.PCK}
 matches @code{japanese-shift-jis} in
 @code{locale-preferred-coding-systems}, Emacs uses that encoding even
-though it might normally use @code{japanese-iso-8bit}.
+though it might normally use @code{utf-8}.
 
   You can override the language environment chosen at startup with
 explicit use of the command @code{set-language-environment}, or with
diff --git a/etc/NEWS b/etc/NEWS
index 15661808c7..2a1218a4a8 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -31,6 +31,10 @@ When you add a new item, use the appropriate mark if you are sure it applies,
 \f
 * Changes in Emacs 27.1
 
+** In Japanese environments that do not specify encodings and are not
+based on MS-Windows, the default encoding is now utf-8 instead of
+japanese-iso-8bit.
+
 +++
 ** New function 'logcount' calculates an integer's Hamming weight.
 
-- 
2.13.6

bug#28705: 25.3; [PATCH] Change prefered coding system for Japanese language environment

[Glenn Morris]

Paul Eggert wrote:

> I will ask for comment about this proposed change on the emacs-devel
> mailing list.

AFAICS nobody commented.

To my uninformed eye, this looks like it might be somewhat related to
issues discussed in https://debbugs.gnu.org/33796 , so I wonder if it
might be time to revisit this? (I have nothing to contribute.)

bug#28705: 25.3; [PATCH] Change prefered coding system for Japanese language environment

[Paul Eggert]

Since nobody commented on it and we have a bug report about this, I 
installed the patch into the master branch and am closing the bug 
report. Thanks again for reporting it, and sorry about the delay.