bug#34215: 27.0.50; Provide elisp access to Chinese pinyin-to-character mapping

[Eric Abrahamsen <eric@ericabrahamsen.net>]

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Sun, 27 Jan 2019 11:18:29 -0800

I've attached a diff adding the conversion function itself, but I'm not
familiar with makefiles and so far haven't been able to figure out how
to call it. It looks like the invocation I want will look like:

$(AM_V_GEN)${RUN_EMACS} -l titdic-cnv -f pinyin-convert \
  ${srcdir}/MISC-DIC/pinyin.map ${srcdir}/../lisp/language/pinyin.el

Where ${srcdir} is the leim directory, but I don't actually know how to
get this code called by make...

Additionally, I could factor the common code in py-converter and
pinyin-convert out into a separate defsubst.

>> >> PS: pinyin.map is ancient and is missing a lot of good correspondences.
>> >> Google's pinyin input method uses a much larger map, licensed with
>> >> Apache v2.0. This[1] seems to indicate that Apache 2.0 is okay for Gnu
>> >> projects, maybe we could consider switching to that map?
>> >
>> > Maybe.  Unfortunately, I don't know enough about these input methods
>> > to tell whether replacing the file is a good idea.  I wonder who can
>> > we ask about this.
>> 
>> It's more or less a drop-in replacement -- the format of the data would
>> be the same, only a bit more of it.
>
> I understand, but I wonder if someone could try that for a while and
> see if it makes better input method(s), before we decide to import it.

FWIW, that mapping is used by the pyim package, which I believe is the
most popular pinyin-based Chinese input method out there. I also use it
via the system-wide input framework fcitx, and it works very well.

>> I'm not sure who is "in charge" of these files, though.
>
> No one, I'm afraid.  Not these days.

That's too bad.

Eric


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: pinyinconvert.diff --]
[-- Type: text/x-patch, Size: 1484 bytes --]

diff --git a/lisp/international/titdic-cnv.el b/lisp/international/titdic-cnv.el
index 2ce2c527b9..54d9fc6211 100644
--- a/lisp/international/titdic-cnv.el
+++ b/lisp/international/titdic-cnv.el
@@ -1203,6 +1203,37 @@ batch-miscdic-convert
 	(miscdic-convert filename dir))))
   (kill-emacs 0))
 
+(defun pinyin-convert ()
+  "Convert text file pinyin.map into an elisp library.
+The library is named pinyin.el, and contains the constant
+`pinyin-character-map'."
+  (let ((src-file (car command-line-args-left))
+        (dst-file (cadr command-line-args-left)))
+    (with-temp-file dst-file
+      (insert ";; This file is automatically generated from pinyin.map,\
+ by the function pinyin-convert.")
+      (insert "(defconst pinyin-character-map\n(")
+      (let ((pos (point)))
+        (insert-file-contents src-file)
+        (goto-char pos)
+        (re-search-forward "^[a-z]")
+        (beginning-of-line)
+        (delete-region pos (point))
+        (while (not (eobp))
+          (insert "(\"")
+          (skip-chars-forward "a-z")
+          (insert "\" \"")
+          (delete-char 1)
+          (end-of-line)
+          (while (= (preceding-char) ?\r)
+	    (delete-char -1))
+          (insert "\")")
+          (forward-line 1)))
+      (insert ")\n\"An alist holding correspondences between pinyin syllables\
+ and Chinese characters.\")\n")
+      (insert "(provide 'pinyin)\n"))
+    (kill-emacs 0)))
+
 ;; Prevent "Local Variables" above confusing Emacs.
 \f