Re: Ispell and unibyte characters

[Agustin Martin <agustin.martin@hispalinux.es>]

[-- Attachment #1: Type: text/plain, Size: 1655 bytes --]

On Wed, Mar 28, 2012 at 09:18:21PM +0200, Agustin Martin wrote:
> On Mon, Mar 26, 2012 at 04:08:06PM -0400, Eli Zaretskii wrote:
> > > Date: Mon, 26 Mar 2012 19:39:12 +0200
> > > From: Agustin Martin <agustin.martin@hispalinux.es>
> > > 
> > > Hi Eli,
> > 
> > Thanks for responding, I was beginning to think that no one is
> > interested.  In general, I find that ispell.el is in sore need of
> > modernization; at least that's my conclusion so far from playing with
> > hunspell (with which I want to replace my aging collection of Ispell
> > and its dictionaries that I use for many years).
> > 
> > > At least for aspell ispell.el already uses utf8 as default communication
> > > encoding and [:alpha:] as CASECHARS (and ^[:alpha:] as NOT-CASECHARS). 
> > > OTHERCHARS is guessed from aspell .dat file for given dictionary.
> > 
> > The question is, why isn't this done for any modern speller.  The only
> > one I know of that cannot handle UTF-8 is Ispell.
> 
> I think the only real remaining reason is for XEmacs compatibility. AFAIK 
> XEmacs does not support [:alpha:].
> 
> I thought about filtering ispell-dictionary-base-alist when used from FSF
> Emacs, so it uses [:alpha:] and still keeps compatibility. I am currently a
> bit busy, but at some time I may try this for Debian and see what happens.

For the records, I am attaching what I am currently trying, post-processing
global dictionary list while leaving local definitions at ~/.emacs
unmodified. This should also deal with [#11200: ispell.el sets incorrect
encoding for the default dictionary]. I would like to test this a bit more
and commit if there are no problems.

-- 
Agustin

[-- Attachment #2: ispell.el_alpha-regexp.2.diff --]
[-- Type: text/x-diff, Size: 2073 bytes --]

--- ispell.el.orig	2012-04-10 20:02:51.422092761 +0200
+++ ispell.el	2012-04-10 20:18:27.464680054 +0200
@@ -783,6 +783,12 @@
 (make-obsolete-variable 'ispell-aspell-supports-utf8
                         'ispell-encoding8-command "23.1")
 
+(defvar ispell-emacs-alpha-regexp
+  (if (string-match "^[[:alpha:]]+$" "abcde")
+      "[[:alpha:]]"
+    nil)
+  "[[:alpha:]] if Emacs supports [:alpha:] regexp, nil
+otherwise (current XEmacs does not support it).")
 
 ;;; **********************************************************************
 ;;; The following are used by ispell, and should not be changed.
@@ -1179,8 +1185,7 @@
 	       (error nil))
 	     ispell-really-aspell
 	     ispell-encoding8-command
-	     ;; XEmacs does not like [:alpha:] regexps.
-	     (string-match "^[[:alpha:]]+$" "abcde"))
+	     ispell-emacs-alpha-regexp)
 	(unless ispell-aspell-dictionary-alist
 	  (ispell-find-aspell-dictionaries)))
 
@@ -1204,8 +1209,27 @@
 			    ispell-dictionary-base-alist))
 	(unless (assoc (car dict) all-dicts-alist)
 	  (add-to-list 'all-dicts-alist dict)))
-      (setq ispell-dictionary-alist all-dicts-alist))))
+      (setq ispell-dictionary-alist all-dicts-alist))
 
+    ;; If Emacs flavor supports [:alpha:] use it for global dicts.  If
+    ;; spellchecker also supports UTF-8 via command-line option use it
+    ;; in communication.  This does not affect definitions in ~/.emacs.
+    (if ispell-emacs-alpha-regexp
+     	(let (tmp-dicts-alist)
+    	  (dolist (adict ispell-dictionary-alist)
+  	    (add-to-list 'tmp-dicts-alist
+   			 (list
+   			  (nth 0 adict)  ; dict name
+    			  "[[:alpha:]]"  ; casechars
+    			  "[^[:alpha:]]" ; not-casechars
+   			  (nth 3 adict)  ; otherchars
+    			  (nth 4 adict)  ; many-otherchars-p
+   			  (nth 5 adict)  ; ispell-args
+   			  (nth 6 adict)  ; extended-character-mode
+			  (if ispell-encoding8-command
+			      'utf-8
+			    (nth 7 adict)))))
+    	  (setq ispell-dictionary-alist tmp-dicts-alist)))))
 
 (defun ispell-valid-dictionary-list ()
   "Return a list of valid dictionaries.