Re: ja-dic.el and SKK-JYSYO.L

[Kenichi Handa <handa@m17n.org>]

In article <m3d4094dsx.fsf@patsy.makina-nantes.net>, Ivan Kanis <expire-by-2010-02-18@kanis.fr> writes:

> I put the new SKK-JYSHO.L in my tmp directory and run the command
> skkdic-convert on the dictionary. It created ja-dic.el in my home
> directory.

> However when I eval ja-dic.el I get an error:

> Debugger entered--Lisp error: (args-out-of-range ">\x308fk \x5206" 6)
>   string-match("[^ ]+" ">\x308fk \x5206" 6)
>   (while (string-match "[^ ]+" entry i) (setq candidates (cons ... candidates)) (setq i (match-end 0)))
>   (let ((kana ...) (i ...) candidates) (while (string-match "[^ ]+" entry i) (setq candidates ...) (setq i ...)) (cons (skkdic-get-kana-compact-codes kana) candidates))
>   skkdic-extract-conversion-data(">\x308fk \x5206")

I found that the latest SKK-JISYO.L contains entries that
can't be handled by the current ja-dic-cnv.el.  So, I
installed the fix in addition to new SKK-JISYO.L and
re-generated ja-dic.el.  Please try the latest one, or try
the attached patch.

> Another issue with the new SKK-JYSHO.L is that it has some comments, for
> example :

> "あおt 煽;火を煽る 呷;酒を呷る"

> My guess is that the block after ; should be ignored. I think 呷 should
> be added as a candidate. I think I can fix that.

Instead, I downloaded SKK-JISYO.L.unannotated and renamed it
to SKK-JISYO.L.

>>> I don't think SKK-JYSYO.L needs to be included in the source code, it's
>>> 2.7M and doesn't need to be there if ja-dic.el exists. It would make the
>>> tar ball a bit smaller.
> >
> >  new SKK-JISYO.L much bigger than the current one?

> The new SKK-JISYO.L is 4.3M.

But the compressed one is just 400k bigger than the old one.
It is surely big but not that disastrous.  So, ...

> > Perhaps we should consider moving SKK-JISYO.L (and the other
> > big files) to `admin' directory which is not included in the
> > tarball.

> It sounds like a good idea, smaller tar ball saves bandwith.

I'll do that after 23.2.

---
Kenichi Handa
handa@m17n.org

=== modified file 'lisp/international/ja-dic-cnv.el'
--- lisp/international/ja-dic-cnv.el	2010-01-13 08:35:10 +0000
+++ lisp/international/ja-dic-cnv.el	2010-02-16 06:47:31 +0000
@@ -45,15 +45,6 @@
 ;; Name of a file to generate from SKK dictionary.
 (defvar ja-dic-filename "ja-dic.el")
 
-;; To make a generated ja-dic.el smaller.
-(define-coding-system 'iso-2022-7bit-short
- "Like `iso-2022-7bit' but no ASCII designation before SPC."
-  :coding-type 'iso-2022
-  :mnemonic ?J
-  :charset-list 'iso-2022
-  :designation [(ascii t) nil nil nil]
-  :flags '(short 7-bit designation))
-
 (defun skkdic-convert-okuri-ari (skkbuf buf)
   (message "Processing OKURI-ARI entries ...")
   (goto-char (point-min))
@@ -61,24 +52,22 @@
     (insert ";; Setting okuri-ari entries.\n"
 	    "(skkdic-set-okuri-ari\n"))
   (while (not (eobp))
-    (let ((from (point))
-	  to)
-      (end-of-line)
-      (setq to (point))
-
-      (with-current-buffer buf
-	(insert-buffer-substring skkbuf from to)
-	(beginning-of-line)
-	(insert "\"")
-	(search-forward " ")
-	(delete-char 1)			; delete the first '/'
-	(let ((p (point)))
-	  (end-of-line)
-	  (delete-char -1)		; delete the last '/'
-	  (subst-char-in-region p (point) ?/ ? 'noundo))
-	(insert "\"\n"))
+    (if (/= (following-char) ?>)
+	(let ((from (point))
+	      (to (line-end-position)))
+	  (with-current-buffer buf
+	    (insert-buffer-substring skkbuf from to)
+	    (beginning-of-line)
+	    (insert "\"")
+	    (search-forward " ")
+	    (delete-char 1)		; delete the first '/'
+	    (let ((p (point)))
+	      (end-of-line)
+	      (delete-char -1)		; delete the last '/'
+	      (subst-char-in-region p (point) ?/ ? 'noundo))
+	    (insert "\"\n"))))
 
-      (forward-line 1)))
+    (forward-line 1))
   (with-current-buffer buf
     (insert ")\n\n")))
 
@@ -348,7 +337,7 @@
       (erase-buffer)
       (buffer-disable-undo)
       (insert ";;; ja-dic.el --- dictionary for Japanese input method"
-	      " -*-coding: iso-2022-jp; byte-compile-disable-print-circle:t; -*-\n"
+	      " -*-coding: euc-japan; byte-compile-disable-print-circle:t; -*-\n"
 	      ";;\tGenerated by the command `skkdic-convert'\n"
 	      ";;\tDate: " (current-time-string) "\n"
 	      ";;\tOriginal SKK dictionary file: "
@@ -410,7 +399,7 @@
       ;; Save the working buffer.
       (set-buffer buf)
       (set-visited-file-name (expand-file-name ja-dic-filename dirname) t)
-      (set-buffer-file-coding-system 'iso-2022-7bit-short)
+      (set-buffer-file-coding-system 'euc-japan)
       (save-buffer 0))
     (kill-buffer skkbuf)
     (switch-to-buffer buf)))