bug#17742: Support for enchant?

bug#17742: Support for enchant?

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

It would be good if Emacs supported the Enchant multi-back-end spelling
engine wrapper, as this allows users to configure spelling preferences more
flexibly than Emacs (in particular, to use different spell-checking engines
for different languages), and the configuration works for programs other
than Emacs.

It seems that it would be pretty trivial to add, since Emacs already works
fine (since 2007) if you set ispell-program-name to /usr/bin/enchant.

Enchant supports UTF-8 (and only UTF-8) as an encoding, so for optimum
functionality, I guess support would need to be added to
ispell-encoding8-command (though since this is enchant's only mode of
operation, it's unclear that anything needs to be done).

In a wider GNU context, the GNU aspell developer wrote a few years ago: "I
believe Aspell future depends on making Enchant the system spell
checker."[1] Enchant itself seems to be stable, portable, and support a
wide range of spelling checker engines, including all those that Emacs
already supports; it would be great longer-term to be able to simplify
ispell.el by requiring Enchant, and thereby being able to rely on its UTF-8
support and single set of simple options.

[1] http://aspell.net/

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1459 bytes --]

bug#17742: Limitations of enchant

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

Since I didn't get any replies, I thought it might be worth noting what I'm
sure some developer must already know, namely that enchant doesn't
currently (v1.6.0) fully support the ispell pipe protocol. In particular,
it doesnt seem to understand either adding words to the dictionary, nor
saving the personal dictionary. In the former case it simply spell-checks
the line, a result which Emacs ignores; in the latter, it gives an empty
reply, and Emacs ignores the fact that it didn't get an "OK" result ("*").

There are still Emacs bugs here: I can't see in the code that Emacs ignores
these failures on purpose, so it should detect and report them. Enchant can
still be used to check spelling on a per-session basis (so maybe it's worth
making the failures warnings rather than errors) but the user shouldn't be
encouraged to believe that it's all working fine.

Obviously, it would be nice if upstream enchant could be made to add this
functionality in the pipe mode: since it's already implemented in
libenchant, that should be easy. Or, as I previously suggested, Emacs could
simply use libenchant and forget about the pipe interface.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1379 bytes --]

bug#17742: Limitations of enchant

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On 15 September 2014 at 12:06, Reuben Thomas <rrt@sc3d.org> wrote:

>
> Obviously, it would be nice if upstream enchant could be made to add this
> functionality in the pipe mode: since it's already implemented in
> libenchant, that should be easy.
>

​On further reflection, this seemed the best route to take, so anyone
interested in this bug may be interested in my patch to enchant that
implements this: https://github.com/AbiWord/enchant/pull/7

This has the advantages that it was simpler to program than libenchant
support in Emacs, allows other ispell-compatible programs to use enchant
for spelling, and will mean that most functionality should work in Emacs
without patching.

If the pull-request is accepted by the enchant maintainers, I will add
aspell's "replace for this session" command "$$ra"; this will require me to
add support to ispell to detect a new enough enchant and use this feature
as it would with aspell.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1840 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1.1: Type: text/plain, Size: 508 bytes --]

Attached, a patch to add initial enchant support.

This seems to work fine with one major limitation: at present there's no
dictionary auto-discovery, and as the built-in ispell-dictionary-alist is
all 8-bit encodings, while enchant is UTF-8, without some manual
configuration the dreaded "misalignment errors" occur.

I shall add dictionary listing support.

I have already added replace-for-session ("$$ra") to my enchant patch, so
enchant can use this functionality in ispell.el.

-- 
http://rrt.sc3d.org

[-- Attachment #1.2: Type: text/html, Size: 1126 bytes --]

[-- Attachment #2: 0023-Add-basic-Enchant-support-to-ispell.el.patch --]
[-- Type: text/x-patch, Size: 3600 bytes --]

From f3dce7c44241c0a17e6fb39f07a2cb32dc05a92a Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 4 Dec 2016 22:39:27 +0000
Subject: [PATCH 23/23] Add basic Enchant support to ispell.el
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/textmodes/ispell.el (ispell-program-name): Add “enchant”.
(ispell-really-enchant): Add variable.
(ispell-check-version): If using Enchant, check it’s new enough (at
least 1.6.1).  (Like the ispell check, this is absolute: cannot work
without.)
(ispell-send-replacement): Make it work with Enchant.
---
 lisp/textmodes/ispell.el | 40 ++++++++++++++++++++++++++++------------
 1 file changed, 28 insertions(+), 12 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 6733c75..1b81a5d 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -209,7 +209,8 @@ ispell-choices-win-default-height
   :group 'ispell)
 
 (defcustom ispell-program-name
-  (or (executable-find "aspell")
+  (or (executable-find "enchant")
+      (executable-find "aspell")
       (executable-find "ispell")
       (executable-find "hunspell")
       "ispell")
@@ -605,6 +606,8 @@ ispell-really-aspell
   "Non-nil if we can use Aspell extensions.")
 (defvar ispell-really-hunspell nil
   "Non-nil if we can use Hunspell extensions.")
+(defvar ispell-really-enchant nil
+  "Non-nil if we can use Enchant extensions.")
 (defvar ispell-encoding8-command nil
   "Command line option prefix to select encoding if supported, nil otherwise.
 If setting the encoding is supported by spellchecker and is selectable from
@@ -740,17 +743,29 @@ ispell-check-version
 		  (and (search-forward-regexp
 			"(but really Hunspell \\([0-9]+\\.[0-9\\.-]+\\)?)"
                         nil t)
+		       (match-string 1)))
+            (setq ispell-really-enchant
+		  (and (search-forward-regexp
+			"(but really Enchant \\([0-9]+\\.[0-9\\.-]+\\)?)"
+                        nil t)
 		       (match-string 1)))))
 
-      (let ((aspell-minver    "0.50")
-	    (aspell8-minver   "0.60")
-	    (ispell-minver    "3.1.12")
-	    (hunspell8-minver "1.1.6"))
-
-	(unless (version<= ispell-minver ispell-program-version)
-	  (error "%s release %s or greater is required"
+      (let* ((aspell-minver    "0.50")
+             (aspell8-minver   "0.60")
+             (ispell-minver    "3.1.12")
+             (hunspell8-minver "1.1.6")
+             (enchant-minver   "1.6.1")
+             (minver (cond
+                      ((not (version<= ispell-minver ispell-program-version))
+                       ispell-minver)
+                      ((and ispell-really-enchant
+                            (not (version<= enchant-minver ispell-really-enchant)))
+                       enchant-minver))))
+
+        (if minver
+	    (error "%s release %s or greater is required"
 		 ispell-program-name
-		 ispell-minver))
+		 minver))
 
 	(cond
 	 (ispell-really-aspell
@@ -1735,9 +1750,10 @@ ispell-accept-output
 	    (erase-buffer)))))))
 
 (defun ispell-send-replacement (misspelled replacement)
-  "Notify Aspell that MISSPELLED should be spelled REPLACEMENT.
-This allows improving the suggestion list based on actual misspellings."
-  (and ispell-really-aspell
+  "Notify spell checker that MISSPELLED should be spelled REPLACEMENT.
+This allows improving the suggestion list based on actual misspellings.
+Only works for Aspell and Enchant."
+  (and (or ispell-really-aspell ispell-really-enchant)
        (ispell-send-string (concat "$$ra " misspelled "," replacement "\n"))))
 
 
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Mon, 5 Dec 2016 21:41:06 +0000
> 
> Attached, a patch to add initial enchant support.

Thanks.

> This seems to work fine with one major limitation: at present there's no dictionary auto-discovery, and as the
> built-in ispell-dictionary-alist is all 8-bit encodings, while enchant is UTF-8, without some manual configuration
> the dreaded "misalignment errors" occur.

I wonder if it won't be better to delay pushing this until you solve
that problem.  The ispell-dictionary-alist is for Ispell speller only;
all the modern spellers should glean that information from the
installed dictionaries, and not bother users with tedious setups.  Do
you see any need to get this into master before you add this
functionality for enchant?

> I shall add dictionary listing support.

Thanks.  Note that this should not only list the installed
dictionaries, but also deduce the suitable values of CASECHARS and
NOT-CASECHARS for each dictionary.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On 6 December 2016 at 15:55, Eli Zaretskii <eliz@gnu.org> wrote:

>
> I wonder if it won't be better to delay pushing this until you solve
> that problem.  The ispell-dictionary-alist is for Ispell speller only;
> all the modern spellers should glean that information from the
> installed dictionaries, and not bother users with tedious setups.  Do
> you see any need to get this into master before you add this
> functionality for enchant?
>

​Sorry if I wasn't clear: I'm working on adding dictionary detection right
now.​ You're quite right that should be added before the patch is
considered installable.


> Thanks.  Note that this should not only list the installed
> dictionaries, but also deduce the suitable values of CASECHARS and
> NOT-CASECHARS for each dictionary.
>

​Thanks for the tip.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1643 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1.1: Type: text/plain, Size: 1513 bytes --]

On 6 December 2016 at 15:56, Reuben Thomas <rrt@sc3d.org> wrote:

> On 6 December 2016 at 15:55, Eli Zaretskii <eliz@gnu.org> wrote:
>
>>
>> I wonder if it won't be better to delay pushing this until you solve
>> that problem.  The ispell-dictionary-alist is for Ispell speller only;
>> all the modern spellers should glean that information from the
>> installed dictionaries, and not bother users with tedious setups.  Do
>> you see any need to get this into master before you add this
>> functionality for enchant?
>>
>
> ​Sorry if I wasn't clear: I'm working on adding dictionary detection right
> now.​ You're quite right that should be added before the patch is
> considered installable.
>
>
>> Thanks.  Note that this should not only list the installed
>> dictionaries, but also deduce the suitable values of CASECHARS and
>> NOT-CASECHARS for each dictionary.
>>
>
> ​Thanks for the tip.
>

​In fact, it proves to be impossible to deduce suitable values currently,
so I set defaults. Enchant will need changes to support this (unless one
finds the provider for each dictionary and uses ispell.el's existing
back-end-specific code, but this is a nasty solution). This simple solution
gets the correct encoding at least (Enchant is always utf8), and hence
fixes the misalignment errors.

However, I have updated my enchant patches to add dictionary listing, and
hence my ispell.el patches to automatically find the enchant dictionaries.

-- 
http://rrt.sc3d.org

[-- Attachment #1.2: Type: text/html, Size: 2784 bytes --]

[-- Attachment #2: 0025-Add-Enchant-support-to-ispell.el.patch --]
[-- Type: text/x-patch, Size: 8571 bytes --]

From b00f110f6c4e00bd6af6b9a1e12f0a2e25303998 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 4 Dec 2016 22:39:27 +0000
Subject: [PATCH 25/25] Add Enchant support to ispell.el
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/textmodes/ispell.el (ispell-program-name): Add “enchant”.
(ispell-really-enchant): Add variable.
(ispell-check-version): If using Enchant, check it’s new enough (at
least 1.6.1).  (Like the ispell check, this is absolute: cannot work
without.)
(ispell-enchant-dictionary-alist): Add variable.
(ispell-find-enchant-dictionaries): Add function, based on
ispell-find-aspell-dictionaries.
(ispell-set-spellchecker-params): Allow dictionary auto-detection for
Enchant, and call ispell-find-enchant-dictionaries to find them.  Use
old ispell name to locale mapping code for Enchant too.
(ispell-send-replacement): Make it work with Enchant.
---
 lisp/textmodes/ispell.el | 77 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 63 insertions(+), 14 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 37a6ffc..19a2e02 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -208,6 +208,10 @@ ispell-choices-win-default-height
   :type 'integer
   :group 'ispell)
 
+;; XXX Add enchant to this list once enchant >= 1.6.1 is widespread.
+;; Before that, adding it is useless, as if it is found, it will just
+;; cause an error; and one of the other spelling engines below is
+;; almost certainly installed in any case, for enchant to use.
 (defcustom ispell-program-name
   (or (executable-find "aspell")
       (executable-find "ispell")
@@ -605,6 +609,8 @@ ispell-really-aspell
   "Non-nil if we can use Aspell extensions.")
 (defvar ispell-really-hunspell nil
   "Non-nil if we can use Hunspell extensions.")
+(defvar ispell-really-enchant nil
+  "Non-nil if we can use Enchant extensions.")
 (defvar ispell-encoding8-command nil
   "Command line option prefix to select encoding if supported, nil otherwise.
 If setting the encoding is supported by spellchecker and is selectable from
@@ -739,17 +745,26 @@ ispell-check-version
 		  (and (search-forward-regexp
 			"(but really Hunspell \\([0-9]+\\.[0-9\\.-]+\\)?)"
                         nil t)
+		       (match-string 1)))
+            (setq ispell-really-enchant
+		  (and (search-forward-regexp
+			"(but really Enchant \\([0-9]+\\.[0-9\\.-]+\\)?)"
+                        nil t)
 		       (match-string 1)))))
 
       (let* ((aspell8-minver   "0.60")
              (ispell-minver    "3.1.12")
              (hunspell8-minver "1.1.6")
+             (enchant-minver   "1.6.1")
              (minver (cond
                       ((not (version<= ispell-minver ispell-program-version))
                        ispell-minver)
                       ((and ispell-really-aspell
                             (not (version<= aspell8-minver ispell-really-aspell)))
-                       aspell8-minver))))
+                       aspell8-minver)
+                      ((and ispell-really-enchant
+                            (not (version<= enchant-minver ispell-really-enchant)))
+                       enchant-minver))))
 
         (if minver
 	    (error "%s release %s or greater is required"
@@ -1173,6 +1188,34 @@ ispell-find-hunspell-dictionaries
                     (list dict))
                   ispell-hunspell-dictionary-alist :test #'equal))))
 
+;; Make ispell.el work better with enchant.
+
+(defvar ispell-enchant-dictionary-alist nil
+  "An alist of parsed Enchant dicts and associated parameters.
+Internal use.")
+
+(defun ispell-find-enchant-dictionaries ()
+  "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
+  (let* ((dictionaries
+	  (split-string
+	   (with-temp-buffer
+	     (ispell-call-process ispell-program-name nil t nil "-D")
+	     (buffer-string))))
+         (found
+          (mapcar #'(lambda (lang)
+                      `(,lang "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8))
+                  dictionaries)))
+    ;; Merge into FOUND any elements from the standard ispell-dictionary-base-alist
+    ;; which have no element in FOUND at all.
+    (dolist (dict ispell-dictionary-base-alist)
+      (unless (assoc (car dict) found)
+	(setq found (nconc found (list dict)))))
+    (setq ispell-enchant-dictionary-alist found)
+    ;; Add a default entry
+    (let ((default-dict
+           '(nil "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8)))
+      (push default-dict ispell-enchant-dictionary-alist))))
+
 ;; Set params according to the selected spellchecker
 
 (defvar ispell-last-program-name nil
@@ -1198,7 +1241,7 @@ ispell-set-spellchecker-params
 		   (setq ispell-library-directory (ispell-check-version))
 		   t)
 	       (error nil))
-	     ispell-encoding8-command)
+	     (or ispell-encoding8-command ispell-really-enchant))
 	;; auto-detection will only be used if spellchecker is not
 	;; ispell and supports a way to set communication to UTF-8.
 	(if ispell-really-aspell
@@ -1206,11 +1249,14 @@ ispell-set-spellchecker-params
 		(ispell-find-aspell-dictionaries))
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
-		  (ispell-find-hunspell-dictionaries)))))
+		  (ispell-find-hunspell-dictionaries))
+            (if ispell-really-enchant
+                (or ispell-enchant-dictionary-alist
+                    (ispell-find-enchant-dictionaries))))))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-    ;; If a recent aspell or hunspell, use the list of really
+    ;; With programs that support it, use the list of really
     ;; installed dictionaries and add to it elements of the original
     ;; list that are not present there. Allow distro info.
     (let ((found-dicts-alist
@@ -1219,17 +1265,19 @@ ispell-set-spellchecker-params
 		   ispell-aspell-dictionary-alist
 		 (if ispell-really-hunspell
 		     ispell-hunspell-dictionary-alist))
-	     nil))
+	     (if ispell-really-enchant
+                 ispell-enchant-dictionary-alist
+               nil)))
 	  (ispell-dictionary-base-alist ispell-dictionary-base-alist)
 	  ispell-base-dicts-override-alist ; Override only base-dicts-alist
 	  all-dicts-alist)
 
       ;; While ispell and aspell (through aliases) use the traditional
-      ;; dict naming originally expected by ispell.el, hunspell
-      ;; uses locale based names with no alias.  We need to map
+      ;; dict naming originally expected by ispell.el, hunspell & Enchant
+      ;; use locale-based names with no alias.  We need to map
       ;; standard names to locale based names to make default dict
-      ;; definitions available for hunspell.
-      (if ispell-really-hunspell
+      ;; definitions available to these programs.
+      (if (or ispell-really-hunspell ispell-really-enchant)
 	  (let (tmp-dicts-alist)
 	    (dolist (adict ispell-dictionary-base-alist)
 	      (let* ((dict-name (nth 0 adict))
@@ -1254,7 +1302,7 @@ ispell-set-spellchecker-params
 			(setq ispell-args
 			      (nconc ispell-args (list "-d" dict-equiv)))
 		      (message
-		       "ispell-set-spellchecker-params: Missing Hunspell equiv for \"%s\". Skipping."
+		       "ispell-set-spellchecker-params: Missing equivalent for \"%s\". Skipping."
 		       dict-name)
 		      (setq skip-dict t)))
 
@@ -1296,7 +1344,7 @@ ispell-set-spellchecker-params
                          (nth 4 adict)   ; many-otherchars-p
                          (nth 5 adict)   ; ispell-args
                          (nth 6 adict)   ; extended-character-mode
-                         (if ispell-encoding8-command
+                         (if (or ispell-encoding8-command ispell-really-enchant)
                              'utf-8
                            (nth 7 adict)))
                       adict)
@@ -1732,9 +1780,10 @@ ispell-accept-output
 	    (erase-buffer)))))))
 
 (defun ispell-send-replacement (misspelled replacement)
-  "Notify Aspell that MISSPELLED should be spelled REPLACEMENT.
-This allows improving the suggestion list based on actual misspellings."
-  (and ispell-really-aspell
+  "Notify spell checker that MISSPELLED should be spelled REPLACEMENT.
+This allows improving the suggestion list based on actual misspellings.
+Only works for Aspell and Enchant."
+  (and (or ispell-really-aspell ispell-really-enchant)
        (ispell-send-string (concat "$$ra " misspelled "," replacement "\n"))))
 
 
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Tue, 13 Dec 2016 00:53:47 +0000
> Cc: 17742@debbugs.gnu.org
> 
>  Thanks. Note that this should not only list the installed
>  dictionaries, but also deduce the suitable values of CASECHARS and
>  NOT-CASECHARS for each dictionary.
> 
>  ​Thanks for the tip.
> 
> ​In fact, it proves to be impossible to deduce suitable values currently, so I set defaults. Enchant will need
> changes to support this (unless one finds the provider for each dictionary and uses ispell.el's existing
> back-end-specific code, but this is a nasty solution). This simple solution gets the correct encoding at least
> (Enchant is always utf8), and hence fixes the misalignment errors.
> [...]
> +(defun ispell-find-enchant-dictionaries ()
> +  "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
> +  (let* ((dictionaries
> +	  (split-string
> +	   (with-temp-buffer
> +	     (ispell-call-process ispell-program-name nil t nil "-D")
> +	     (buffer-string))))
> +         (found
> +          (mapcar #'(lambda (lang)
> +                      `(,lang "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8))
> +                  dictionaries)))

I'm sorry, I don't think this is good enough for any language but
English.  I looked at a few Hunspell dictionaries I have, and almost
all of them have some non-[:alpha:] characters in what becomes
CASECHARS when ispell.el sets up itself for that dictionary.  So this
seems to be really important for correct spell-checking of almost any
language.  Moreover, these settings must be in sync with the words in
the dictionary, so it isn't just some general setting determined by
the dictionary's language, it must come from the dictionary or its
authors.

I understand that Enchant doesn't yet support these features, but in
that case we must provide our users with some way of customizing these
settings, we cannot silently use defaults that are most probably
inappropriate, even if they are 90% correct.

Do the dictionaries used by Enchant really fail to provide this
information?  Aren't they Aspell dictionaries or similar?

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 2523 bytes --]

On 13 December 2016 at 16:37, Eli Zaretskii <eliz@gnu.org> wrote:

>
> I'm sorry, I don't think this is good enough for any language but
> English.  I looked at a few Hunspell dictionaries I have, and almost
> all of them have some non-[:alpha:] characters in what becomes
> CASECHARS when ispell.el sets up itself for that dictionary.  So this
> seems to be really important for correct spell-checking of almost any
> language.  Moreover, these settings must be in sync with the words in
> the dictionary, so it isn't just some general setting determined by
> the dictionary's language, it must come from the dictionary or its
> authors.
>

​I agree, this is something that seems to be lacking in Enchant.​


> I understand that Enchant doesn't yet support these features, but in
> that case we must provide our users with some way of customizing these
> settings, we cannot silently use defaults that are most probably
> inappropriate, even if they are 90% correct.
>

​The way to customize it is surely the same as currently, namely​ to
customize ispell-dictionary-alist?​


> Do the dictionaries used by Enchant really fail to provide this
> information?  Aren't they Aspell dictionaries or similar?
>

​​Enchant has no dictionaries of its own, it's just a proxy for hunspell,
aspell &c. So, as I said before, it would be possible to find out which
back-end is being used for a particular dictionary, and then to try to use
the existing ispell.el code to scan for that dictionary. However, this is
not an ideal solution, because it is going "behind Enchant's back".

Given that even management of personal word lists needs a patch to Enchant,
it seems to me better to first try to engage with the Enchant maintainers
on this. There's also one obvious question, namely, what do other Enchant
users do? Enchant is used in Abiword (free word processor), Pidgin (free IM
client) etc. So, I shall look into this.

However, I think that this patch can be installed anyway, as out of the box
Enchant will never be used (as I have commented in the code, auto-detection
of Enchant is not safe until a new version is widespread, as unlike old
versions of ispell and aspell, a current version of Enchant is not merely
lacking functions, it is useless), and for some Emacs users, it could be
useful, as it allows access to spellcheckers currently unsupported by Emacs
(albeit, as you note, probably requiring customized dictionary setup).

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 3670 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Tue, 13 Dec 2016 18:26:43 +0000
> Cc: 17742@debbugs.gnu.org
> 
>  I understand that Enchant doesn't yet support these features, but in
>  that case we must provide our users with some way of customizing these
>  settings, we cannot silently use defaults that are most probably
>  inappropriate, even if they are 90% correct.
> 
> ​The way to customize it is surely the same as currently, namely​ to customize ispell-dictionary-alist?​

That's okay, but then ispell.el shouldn't be generating dictionary
entries, but instead should leave that to the user, as we do with
Ispell.

> Given that even management of personal word lists needs a patch to Enchant, it seems to me better to first
> try to engage with the Enchant maintainers on this. There's also one obvious question, namely, what do other
> Enchant users do? Enchant is used in Abiword (free word processor), Pidgin (free IM client) etc. So, I shall
> look into this.

Thanks.

> However, I think that this patch can be installed anyway, as out of the box Enchant will never be used (as I
> have commented in the code, auto-detection of Enchant is not safe until a new version is widespread, as
> unlike old versions of ispell and aspell, a current version of Enchant is not merely lacking functions, it is
> useless), and for some Emacs users, it could be useful, as it allows access to spellcheckers currently
> unsupported by Emacs (albeit, as you note, probably requiring customized dictionary setup).

I'm okay with installing this, provided that the above-mentioned issue
is taken care of, if only by leaving it to the user to provide the
dictionary entries by hand.  I don't think it's a good idea to
auto-generate entries that are only 80% correct, as that will
certainly trigger bug reports.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1.1: Type: text/plain, Size: 738 bytes --]

On 13 December 2016 at 18:54, Eli Zaretskii <eliz@gnu.org> wrote:

>
> I'm okay with installing this, provided that the above-mentioned issue
> is taken care of, if only by leaving it to the user to provide the
> dictionary entries by hand.  I don't think it's a good idea to
> auto-generate entries that are only 80% correct, as that will
> certainly trigger bug reports.
>

​Agreed. I have prepared a separate patch (so it can be easily reverted
later) to disable the use of ispell-find-enchant-dictionaries, along with a
rationale in comments.

If that is satisfactory, I can add NEWS documentation for the new feature,
along with a warning about its limitations, and install the patches.

-- 
http://rrt.sc3d.org

[-- Attachment #1.2: Type: text/html, Size: 1420 bytes --]

[-- Attachment #2: 0001-Add-Enchant-support-to-ispell.el-Bug-17742.patch --]
[-- Type: text/x-patch, Size: 8581 bytes --]

From 7787517c94c41c2a4093acb3f9102745e28f7a06 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 4 Dec 2016 22:39:27 +0000
Subject: [PATCH 1/3] Add Enchant support to ispell.el (Bug#17742)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/textmodes/ispell.el (ispell-program-name): Add “enchant”.
(ispell-really-enchant): Add variable.
(ispell-check-version): If using Enchant, check it’s new enough (at
least 1.6.1).  (Like the ispell check, this is absolute: cannot work
without.)
(ispell-enchant-dictionary-alist): Add variable.
(ispell-find-enchant-dictionaries): Add function, based on
ispell-find-aspell-dictionaries.
(ispell-set-spellchecker-params): Allow dictionary auto-detection for
Enchant, and call ispell-find-enchant-dictionaries to find them.  Use
old ispell name to locale mapping code for Enchant too.
(ispell-send-replacement): Make it work with Enchant.
---
 lisp/textmodes/ispell.el | 77 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 63 insertions(+), 14 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 37a6ffc..19a2e02 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -208,6 +208,10 @@ ispell-choices-win-default-height
   :type 'integer
   :group 'ispell)
 
+;; XXX Add enchant to this list once enchant >= 1.6.1 is widespread.
+;; Before that, adding it is useless, as if it is found, it will just
+;; cause an error; and one of the other spelling engines below is
+;; almost certainly installed in any case, for enchant to use.
 (defcustom ispell-program-name
   (or (executable-find "aspell")
       (executable-find "ispell")
@@ -605,6 +609,8 @@ ispell-really-aspell
   "Non-nil if we can use Aspell extensions.")
 (defvar ispell-really-hunspell nil
   "Non-nil if we can use Hunspell extensions.")
+(defvar ispell-really-enchant nil
+  "Non-nil if we can use Enchant extensions.")
 (defvar ispell-encoding8-command nil
   "Command line option prefix to select encoding if supported, nil otherwise.
 If setting the encoding is supported by spellchecker and is selectable from
@@ -739,17 +745,26 @@ ispell-check-version
 		  (and (search-forward-regexp
 			"(but really Hunspell \\([0-9]+\\.[0-9\\.-]+\\)?)"
                         nil t)
+		       (match-string 1)))
+            (setq ispell-really-enchant
+		  (and (search-forward-regexp
+			"(but really Enchant \\([0-9]+\\.[0-9\\.-]+\\)?)"
+                        nil t)
 		       (match-string 1)))))
 
       (let* ((aspell8-minver   "0.60")
              (ispell-minver    "3.1.12")
              (hunspell8-minver "1.1.6")
+             (enchant-minver   "1.6.1")
              (minver (cond
                       ((not (version<= ispell-minver ispell-program-version))
                        ispell-minver)
                       ((and ispell-really-aspell
                             (not (version<= aspell8-minver ispell-really-aspell)))
-                       aspell8-minver))))
+                       aspell8-minver)
+                      ((and ispell-really-enchant
+                            (not (version<= enchant-minver ispell-really-enchant)))
+                       enchant-minver))))
 
         (if minver
 	    (error "%s release %s or greater is required"
@@ -1173,6 +1188,34 @@ ispell-find-hunspell-dictionaries
                     (list dict))
                   ispell-hunspell-dictionary-alist :test #'equal))))
 
+;; Make ispell.el work better with enchant.
+
+(defvar ispell-enchant-dictionary-alist nil
+  "An alist of parsed Enchant dicts and associated parameters.
+Internal use.")
+
+(defun ispell-find-enchant-dictionaries ()
+  "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
+  (let* ((dictionaries
+	  (split-string
+	   (with-temp-buffer
+	     (ispell-call-process ispell-program-name nil t nil "-D")
+	     (buffer-string))))
+         (found
+          (mapcar #'(lambda (lang)
+                      `(,lang "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8))
+                  dictionaries)))
+    ;; Merge into FOUND any elements from the standard ispell-dictionary-base-alist
+    ;; which have no element in FOUND at all.
+    (dolist (dict ispell-dictionary-base-alist)
+      (unless (assoc (car dict) found)
+	(setq found (nconc found (list dict)))))
+    (setq ispell-enchant-dictionary-alist found)
+    ;; Add a default entry
+    (let ((default-dict
+           '(nil "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8)))
+      (push default-dict ispell-enchant-dictionary-alist))))
+
 ;; Set params according to the selected spellchecker
 
 (defvar ispell-last-program-name nil
@@ -1198,7 +1241,7 @@ ispell-set-spellchecker-params
 		   (setq ispell-library-directory (ispell-check-version))
 		   t)
 	       (error nil))
-	     ispell-encoding8-command)
+	     (or ispell-encoding8-command ispell-really-enchant))
 	;; auto-detection will only be used if spellchecker is not
 	;; ispell and supports a way to set communication to UTF-8.
 	(if ispell-really-aspell
@@ -1206,11 +1249,14 @@ ispell-set-spellchecker-params
 		(ispell-find-aspell-dictionaries))
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
-		  (ispell-find-hunspell-dictionaries)))))
+		  (ispell-find-hunspell-dictionaries))
+            (if ispell-really-enchant
+                (or ispell-enchant-dictionary-alist
+                    (ispell-find-enchant-dictionaries))))))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-    ;; If a recent aspell or hunspell, use the list of really
+    ;; With programs that support it, use the list of really
     ;; installed dictionaries and add to it elements of the original
     ;; list that are not present there. Allow distro info.
     (let ((found-dicts-alist
@@ -1219,17 +1265,19 @@ ispell-set-spellchecker-params
 		   ispell-aspell-dictionary-alist
 		 (if ispell-really-hunspell
 		     ispell-hunspell-dictionary-alist))
-	     nil))
+	     (if ispell-really-enchant
+                 ispell-enchant-dictionary-alist
+               nil)))
 	  (ispell-dictionary-base-alist ispell-dictionary-base-alist)
 	  ispell-base-dicts-override-alist ; Override only base-dicts-alist
 	  all-dicts-alist)
 
       ;; While ispell and aspell (through aliases) use the traditional
-      ;; dict naming originally expected by ispell.el, hunspell
-      ;; uses locale based names with no alias.  We need to map
+      ;; dict naming originally expected by ispell.el, hunspell & Enchant
+      ;; use locale-based names with no alias.  We need to map
       ;; standard names to locale based names to make default dict
-      ;; definitions available for hunspell.
-      (if ispell-really-hunspell
+      ;; definitions available to these programs.
+      (if (or ispell-really-hunspell ispell-really-enchant)
 	  (let (tmp-dicts-alist)
 	    (dolist (adict ispell-dictionary-base-alist)
 	      (let* ((dict-name (nth 0 adict))
@@ -1254,7 +1302,7 @@ ispell-set-spellchecker-params
 			(setq ispell-args
 			      (nconc ispell-args (list "-d" dict-equiv)))
 		      (message
-		       "ispell-set-spellchecker-params: Missing Hunspell equiv for \"%s\". Skipping."
+		       "ispell-set-spellchecker-params: Missing equivalent for \"%s\". Skipping."
 		       dict-name)
 		      (setq skip-dict t)))
 
@@ -1296,7 +1344,7 @@ ispell-set-spellchecker-params
                          (nth 4 adict)   ; many-otherchars-p
                          (nth 5 adict)   ; ispell-args
                          (nth 6 adict)   ; extended-character-mode
-                         (if ispell-encoding8-command
+                         (if (or ispell-encoding8-command ispell-really-enchant)
                              'utf-8
                            (nth 7 adict)))
                       adict)
@@ -1732,9 +1780,10 @@ ispell-accept-output
 	    (erase-buffer)))))))
 
 (defun ispell-send-replacement (misspelled replacement)
-  "Notify Aspell that MISSPELLED should be spelled REPLACEMENT.
-This allows improving the suggestion list based on actual misspellings."
-  (and ispell-really-aspell
+  "Notify spell checker that MISSPELLED should be spelled REPLACEMENT.
+This allows improving the suggestion list based on actual misspellings.
+Only works for Aspell and Enchant."
+  (and (or ispell-really-aspell ispell-really-enchant)
        (ispell-send-string (concat "$$ra " misspelled "," replacement "\n"))))
 
 
-- 
2.7.4


[-- Attachment #3: 0002-Disable-auto-scanning-of-Enchant-dictionaries.patch --]
[-- Type: text/x-patch, Size: 1905 bytes --]

From 7efe625cd7395e011fa5bffe5dab56b5accabbe0 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Tue, 13 Dec 2016 21:12:13 +0000
Subject: [PATCH 2/3] Disable auto-scanning of Enchant dictionaries

lisp/textmodes/ispell.el (ispell-set-spellchecker-params): Do not call
ispell-find-enchant-dictionaries.  Add a comment noting that it is
disabled until it is possible to set good CASECHARS and NOT-CASECHARS
values.  (Bug#17742; see also
https://github.com/AbiWord/enchant/issues/17 )
---
 lisp/textmodes/ispell.el | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 19a2e02..e17d432 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -1194,6 +1194,8 @@ ispell-enchant-dictionary-alist
   "An alist of parsed Enchant dicts and associated parameters.
 Internal use.")
 
+;; XXX Not used: needs to be fixed to generate good CASECHARS and
+;; NOT-CASECHARS values.
 (defun ispell-find-enchant-dictionaries ()
   "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
   (let* ((dictionaries
@@ -1250,9 +1252,12 @@ ispell-set-spellchecker-params
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
 		  (ispell-find-hunspell-dictionaries))
-            (if ispell-really-enchant
-                (or ispell-enchant-dictionary-alist
-                    (ispell-find-enchant-dictionaries))))))
+            ;; XXX Disable until ispell-find-enchant-dictionaries can
+            ;; generate good CASECHARS and NOT-CASECHARS values.
+            ;; (if ispell-really-enchant
+            ;;     (or ispell-enchant-dictionary-alist
+            ;;         (ispell-find-enchant-dictionaries)))
+            )))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1.1: Type: text/plain, Size: 380 bytes --]

This updates the first patch to use enchant-lsmod to list the dictionaries
in ispell-find-enchant-dictionaries, rather than the (now retracted) patch
I previously made to enchant, not realising that the functionality existed
in enchant-lsmod.

Of course, this dictionary-scanning functionality is still disabled
currently by default for Enchant in Emacs.

-- 
http://rrt.sc3d.org

[-- Attachment #1.2: Type: text/html, Size: 788 bytes --]

[-- Attachment #2: 0001-Add-Enchant-support-to-ispell.el-Bug-17742.patch --]
[-- Type: text/x-patch, Size: 8648 bytes --]

From ea6115153585c777ecb626470c2e7b5426d906f9 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 4 Dec 2016 22:39:27 +0000
Subject: [PATCH 1/3] Add Enchant support to ispell.el (Bug#17742)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/textmodes/ispell.el (ispell-program-name): Add “enchant”.
(ispell-really-enchant): Add variable.
(ispell-check-version): If using Enchant, check it’s new enough (at
least 1.6.1).  (Like the ispell check, this is absolute: cannot work
without.)
(ispell-enchant-dictionary-alist): Add variable.
(ispell-find-enchant-dictionaries): Add function, based on
ispell-find-aspell-dictionaries.
(ispell-set-spellchecker-params): Allow dictionary auto-detection for
Enchant, and call ispell-find-enchant-dictionaries to find them.  Use
old ispell name to locale mapping code for Enchant too.
(ispell-send-replacement): Make it work with Enchant.
---
 lisp/textmodes/ispell.el | 79 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 65 insertions(+), 14 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 37a6ffc..bc25945 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -208,6 +208,10 @@ ispell-choices-win-default-height
   :type 'integer
   :group 'ispell)
 
+;; XXX Add enchant to this list once enchant >= 1.6.1 is widespread.
+;; Before that, adding it is useless, as if it is found, it will just
+;; cause an error; and one of the other spelling engines below is
+;; almost certainly installed in any case, for enchant to use.
 (defcustom ispell-program-name
   (or (executable-find "aspell")
       (executable-find "ispell")
@@ -605,6 +609,8 @@ ispell-really-aspell
   "Non-nil if we can use Aspell extensions.")
 (defvar ispell-really-hunspell nil
   "Non-nil if we can use Hunspell extensions.")
+(defvar ispell-really-enchant nil
+  "Non-nil if we can use Enchant extensions.")
 (defvar ispell-encoding8-command nil
   "Command line option prefix to select encoding if supported, nil otherwise.
 If setting the encoding is supported by spellchecker and is selectable from
@@ -739,17 +745,26 @@ ispell-check-version
 		  (and (search-forward-regexp
 			"(but really Hunspell \\([0-9]+\\.[0-9\\.-]+\\)?)"
                         nil t)
+		       (match-string 1)))
+            (setq ispell-really-enchant
+		  (and (search-forward-regexp
+			"(but really Enchant \\([0-9]+\\.[0-9\\.-]+\\)?)"
+                        nil t)
 		       (match-string 1)))))
 
       (let* ((aspell8-minver   "0.60")
              (ispell-minver    "3.1.12")
              (hunspell8-minver "1.1.6")
+             (enchant-minver   "1.6.1")
              (minver (cond
                       ((not (version<= ispell-minver ispell-program-version))
                        ispell-minver)
                       ((and ispell-really-aspell
                             (not (version<= aspell8-minver ispell-really-aspell)))
-                       aspell8-minver))))
+                       aspell8-minver)
+                      ((and ispell-really-enchant
+                            (not (version<= enchant-minver ispell-really-enchant)))
+                       enchant-minver))))
 
         (if minver
 	    (error "%s release %s or greater is required"
@@ -1173,6 +1188,36 @@ ispell-find-hunspell-dictionaries
                     (list dict))
                   ispell-hunspell-dictionary-alist :test #'equal))))
 
+;; Make ispell.el work better with enchant.
+
+(defvar ispell-enchant-dictionary-alist nil
+  "An alist of parsed Enchant dicts and associated parameters.
+Internal use.")
+
+(defun ispell-find-enchant-dictionaries ()
+  "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
+  (let* ((dictionaries
+	  (split-string
+	   (with-temp-buffer
+	     (ispell-call-process
+              (concat ispell-program-name "-lsmod") nil t nil "-list-dicts")
+	     (buffer-string))
+           " ([^)]+)\n"))
+         (found
+          (mapcar #'(lambda (lang)
+                      `(,lang "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8))
+                  dictionaries)))
+    ;; Merge into FOUND any elements from the standard ispell-dictionary-base-alist
+    ;; which have no element in FOUND at all.
+    (dolist (dict ispell-dictionary-base-alist)
+      (unless (assoc (car dict) found)
+	(setq found (nconc found (list dict)))))
+    (setq ispell-enchant-dictionary-alist found)
+    ;; Add a default entry
+    (let ((default-dict
+           '(nil "[[:alpha:]]" "[^[:alpha:]]" "['.’-]" t nil nil utf-8)))
+      (push default-dict ispell-enchant-dictionary-alist))))
+
 ;; Set params according to the selected spellchecker
 
 (defvar ispell-last-program-name nil
@@ -1198,7 +1243,7 @@ ispell-set-spellchecker-params
 		   (setq ispell-library-directory (ispell-check-version))
 		   t)
 	       (error nil))
-	     ispell-encoding8-command)
+	     (or ispell-encoding8-command ispell-really-enchant))
 	;; auto-detection will only be used if spellchecker is not
 	;; ispell and supports a way to set communication to UTF-8.
 	(if ispell-really-aspell
@@ -1206,11 +1251,14 @@ ispell-set-spellchecker-params
 		(ispell-find-aspell-dictionaries))
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
-		  (ispell-find-hunspell-dictionaries)))))
+		  (ispell-find-hunspell-dictionaries))
+            (if ispell-really-enchant
+                (or ispell-enchant-dictionary-alist
+                    (ispell-find-enchant-dictionaries))))))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-    ;; If a recent aspell or hunspell, use the list of really
+    ;; With programs that support it, use the list of really
     ;; installed dictionaries and add to it elements of the original
     ;; list that are not present there. Allow distro info.
     (let ((found-dicts-alist
@@ -1219,17 +1267,19 @@ ispell-set-spellchecker-params
 		   ispell-aspell-dictionary-alist
 		 (if ispell-really-hunspell
 		     ispell-hunspell-dictionary-alist))
-	     nil))
+	     (if ispell-really-enchant
+                 ispell-enchant-dictionary-alist
+               nil)))
 	  (ispell-dictionary-base-alist ispell-dictionary-base-alist)
 	  ispell-base-dicts-override-alist ; Override only base-dicts-alist
 	  all-dicts-alist)
 
       ;; While ispell and aspell (through aliases) use the traditional
-      ;; dict naming originally expected by ispell.el, hunspell
-      ;; uses locale based names with no alias.  We need to map
+      ;; dict naming originally expected by ispell.el, hunspell & Enchant
+      ;; use locale-based names with no alias.  We need to map
       ;; standard names to locale based names to make default dict
-      ;; definitions available for hunspell.
-      (if ispell-really-hunspell
+      ;; definitions available to these programs.
+      (if (or ispell-really-hunspell ispell-really-enchant)
 	  (let (tmp-dicts-alist)
 	    (dolist (adict ispell-dictionary-base-alist)
 	      (let* ((dict-name (nth 0 adict))
@@ -1254,7 +1304,7 @@ ispell-set-spellchecker-params
 			(setq ispell-args
 			      (nconc ispell-args (list "-d" dict-equiv)))
 		      (message
-		       "ispell-set-spellchecker-params: Missing Hunspell equiv for \"%s\". Skipping."
+		       "ispell-set-spellchecker-params: Missing equivalent for \"%s\". Skipping."
 		       dict-name)
 		      (setq skip-dict t)))
 
@@ -1296,7 +1346,7 @@ ispell-set-spellchecker-params
                          (nth 4 adict)   ; many-otherchars-p
                          (nth 5 adict)   ; ispell-args
                          (nth 6 adict)   ; extended-character-mode
-                         (if ispell-encoding8-command
+                         (if (or ispell-encoding8-command ispell-really-enchant)
                              'utf-8
                            (nth 7 adict)))
                       adict)
@@ -1732,9 +1782,10 @@ ispell-accept-output
 	    (erase-buffer)))))))
 
 (defun ispell-send-replacement (misspelled replacement)
-  "Notify Aspell that MISSPELLED should be spelled REPLACEMENT.
-This allows improving the suggestion list based on actual misspellings."
-  (and ispell-really-aspell
+  "Notify spell checker that MISSPELLED should be spelled REPLACEMENT.
+This allows improving the suggestion list based on actual misspellings.
+Only works for Aspell and Enchant."
+  (and (or ispell-really-aspell ispell-really-enchant)
        (ispell-send-string (concat "$$ra " misspelled "," replacement "\n"))))
 
 
-- 
2.7.4


[-- Attachment #3: 0002-Disable-auto-scanning-of-Enchant-dictionaries.patch --]
[-- Type: text/x-patch, Size: 1905 bytes --]

From f4252552d7670e25160cc3e77d2c95e64f831332 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Tue, 13 Dec 2016 21:12:13 +0000
Subject: [PATCH 2/3] Disable auto-scanning of Enchant dictionaries

lisp/textmodes/ispell.el (ispell-set-spellchecker-params): Do not call
ispell-find-enchant-dictionaries.  Add a comment noting that it is
disabled until it is possible to set good CASECHARS and NOT-CASECHARS
values.  (Bug#17742; see also
https://github.com/AbiWord/enchant/issues/17 )
---
 lisp/textmodes/ispell.el | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index bc25945..43eb672 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -1194,6 +1194,8 @@ ispell-enchant-dictionary-alist
   "An alist of parsed Enchant dicts and associated parameters.
 Internal use.")
 
+;; XXX Not used: needs to be fixed to generate good CASECHARS and
+;; NOT-CASECHARS values.
 (defun ispell-find-enchant-dictionaries ()
   "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
   (let* ((dictionaries
@@ -1252,9 +1254,12 @@ ispell-set-spellchecker-params
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
 		  (ispell-find-hunspell-dictionaries))
-            (if ispell-really-enchant
-                (or ispell-enchant-dictionary-alist
-                    (ispell-find-enchant-dictionaries))))))
+            ;; XXX Disable until ispell-find-enchant-dictionaries can
+            ;; generate good CASECHARS and NOT-CASECHARS values.
+            ;; (if ispell-really-enchant
+            ;;     (or ispell-enchant-dictionary-alist
+            ;;         (ispell-find-enchant-dictionaries)))
+            )))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Tue, 13 Dec 2016 21:17:25 +0000
> Cc: 17742@debbugs.gnu.org
> 
>  I'm okay with installing this, provided that the above-mentioned issue
>  is taken care of, if only by leaving it to the user to provide the
>  dictionary entries by hand. I don't think it's a good idea to
>  auto-generate entries that are only 80% correct, as that will
>  certainly trigger bug reports.
> 
> ​Agreed. I have prepared a separate patch (so it can be easily reverted later) to disable the use of
> ispell-find-enchant-dictionaries, along with a rationale in comments.
> 
> If that is satisfactory, I can add NEWS documentation for the new feature, along with a warning about its
> limitations, and install the patches.

OK, I guess.  Maybe add in a comment an example of setting up Enchant
for some language, in case users may find that helpful.

Thanks.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1338 bytes --]

On 14 December 2016 at 15:42, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Reuben Thomas <rrt@sc3d.org>
> > Date: Tue, 13 Dec 2016 21:17:25 +0000
> > Cc: 17742@debbugs.gnu.org
> >
> >  I'm okay with installing this, provided that the above-mentioned issue
> >  is taken care of, if only by leaving it to the user to provide the
> >  dictionary entries by hand. I don't think it's a good idea to
> >  auto-generate entries that are only 80% correct, as that will
> >  certainly trigger bug reports.
> >
> > ​Agreed. I have prepared a separate patch (so it can be easily reverted
> later) to disable the use of
> > ispell-find-enchant-dictionaries, along with a rationale in comments.
> >
> > If that is satisfactory, I can add NEWS documentation for the new
> feature, along with a warning about its
> > limitations, and install the patches.
>
> OK, I guess.  Maybe add in a comment an example of setting up Enchant
> for some language, in case users may find that helpful.
>

​I'll bear that in mind, but it occurs to me that since the code relies on
an unreleased version of Enchant, there is little to be gained by
installing it until the release is made, and better to wait, in case I have
to change anything. So I'll keep this bug open and not install the patch
for now.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2149 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 2550 bytes --]

On 13 December 2016 at 18:54, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Reuben Thomas <rrt@sc3d.org>
> > Date: Tue, 13 Dec 2016 18:26:43 +0000
> > Cc: 17742@debbugs.gnu.org
> >
> >  I understand that Enchant doesn't yet support these features, but in
> >  that case we must provide our users with some way of customizing these
> >  settings, we cannot silently use defaults that are most probably
> >  inappropriate, even if they are 90% correct.
> >
> > ​The way to customize it is surely the same as currently, namely​ to
> customize ispell-dictionary-alist?​
>
> That's okay, but then ispell.el shouldn't be generating dictionary
> entries, but instead should leave that to the user, as we do with
> Ispell.
>
> > Given that even management of personal word lists needs a patch to
> Enchant, it seems to me better to first
> > try to engage with the Enchant maintainers on this. There's also one
> obvious question, namely, what do other
> > Enchant users do? Enchant is used in Abiword (free word processor),
> Pidgin (free IM client) etc. So, I shall
> > look into this.
>
> Thanks.
>

​I have not had any response to my enquiries yet, but I did some research,
and neither GNU Aspell nor hunspell offer any way to get this information
(about character classes of dictionaries) via their APIs.

This suggests that they do not see a need for it. Perhaps it is worth
confirming whether Emacs really needs this information?

As far as I can see, it is used only in flyspell-word, for per-word
spell-checking. (The only caller outside flyspell.el is erc, which has a
FIXME saying not to call flyspell-word.) As far as I can see, the code
assumes that words are a convenient unit to check and cache, though there's
no definite requirement for that: in particular, the spelling checkers will
say what words are incorrectly spelled and where they are without having to
be given precisely the word. I guess that other editors and word processors
work this way.

For example, aspell.h contains the following notice about
aspell_document_checker_process:

 * The string passed in should only be split on
 * white space characters.

So a reasonable definition of cache units (I will not say "words" any more)
might be whitespace-delimited strings. This would not need casechars,
not-casechars, otherchars and many-otherchars-p.

I've put this on my to-investigate list for the future; of course, if
there's an obvious reason it wouldn't work, do let me know!

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 4034 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]

On 18 December 2016 at 23:39, Reuben Thomas <rrt@sc3d.org> wrote:

>
> So a reasonable definition of cache units (I will not say "words" any
> more) might be whitespace-delimited strings. This would not need casechars,
> not-casechars, otherchars and many-otherchars-p.
>
> I've put this on my to-investigate list for the future; of course, if
> there's an obvious reason it wouldn't work, do let me know!
>

​Basic tests using [[:alpha:]] for casechars and [^[:alpha:]] for
not-casechars seem to work OK.

​What would be a more rigorous test? So far I just tried some
spell-checking and use of flyspell.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1577 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On 19 December 2016 at 01:02, Reuben Thomas <rrt@sc3d.org> wrote:

> On 18 December 2016 at 23:39, Reuben Thomas <rrt@sc3d.org> wrote:
>
>>
>> So a reasonable definition of cache units (I will not say "words" any
>> more) might be whitespace-delimited strings. This would not need casechars,
>> not-casechars, otherchars and many-otherchars-p.
>>
>> I've put this on my to-investigate list for the future; of course, if
>> there's an obvious reason it wouldn't work, do let me know!
>>
>
> ​Basic tests using [[:alpha:]] for casechars and [^[:alpha:]] for
> not-casechars seem to work OK.
>

​I meant [[:graph:]] and [^[:graph:]].​


> ​What would be a more rigorous test? So far I just tried some
> spell-checking and use of flyspell.
>

​Also, as I realised while preparing the patch for bug#25230, it is only
hunspell that has special information about character classes. All the
others just use [:alpha:]. So if it's good enough for ispell and aspell,
can't it be good enough for enchant? (It just means that for now "direct
Hunspell" is arguably better than "Hunspell via Enchant".)

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 2629 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Sun, 18 Dec 2016 23:39:54 +0000
> Cc: 17742@debbugs.gnu.org
> 
> I have not had any response to my enquiries yet, but I did some research, and neither GNU Aspell nor hunspell offer any way to get this information (about character classes of dictionaries) via their APIs.

They provide this information in the dictionaries, and we glean it
from there.  See ispell-parse-hunspell-affix-file and
ispell-aspell-find-dictionary.

> This suggests that they do not see a need for it. Perhaps it is worth confirming whether Emacs really needs this information?
> 
> As far as I can see, it is used only in flyspell-word, for per-word spell-checking. (The only caller outside flyspell.el is erc, which has a FIXME saying not to call flyspell-word.) As far as I can see, the code assumes that words are a convenient unit to check and cache, though there's no definite requirement for that: in particular, the spelling checkers will say what words are incorrectly spelled and where they are without having to be given precisely the word. I guess that other editors and word processors work this way.

Maybe there's a misunderstanding: I'm talking about the CASECHARS,
NOT-CASECHARS, and OTHERCHARS parts of the dictionary data in
ispell-dictionary-alist.  These are definitely used in ispell.el, via
the corresponding accessor functions ispell-get-casechars,
ispell-get-not-casechars, and ispell-get-otherchars, which see.

Each dictionary can (and many do) use some of the punctuation
characters in the words it can handle.  A notable example is the
apostrophe ' in English, used for the various suffixes that spellers
support; similar features exist in other languages, but with possibly
different punctuation characters.  Ispell.el must match that by using
the speller's notion of a word, which must be independent of the
current major mode's idea of what a word is.  This is where these
character sets come into play, and I really cannot see how can
ispell.el work well without using them as it does now.

So we do need this information.  If Enchant doesn't provide it, we
could still use the same technique as with Aspell and Hunspell,
provided that we can figure out which back end(s) is/are used by
Enchant.  Is that doable?

> For example, aspell.h contains the following notice about aspell_document_checker_process:
> 
>  * The string passed in should only be split on
>  * white space characters.

Ispell.el also supports spell-checking by words, in which case the
above is not useful, because we need to figure out what is a word.
Moreover, even when we send entire lines to the speller, we want to
skip lines that include only non-word characters.  Just look at the
callers of the above-mentioned accessor functions, and you will see
how we use them.

> Basic tests using [[:alpha:]] for casechars and [^[:alpha:]] for not-casechars seem to work OK.

For which language and dictionary?  This will definitely do the wrong
thing for Hunspell he_IL dictionary I have here, which says:

  WORDCHARS אבגדהוזחטיכלמנסעפצקרשתםןךףץ'"

That is, it wants ' and " to be treated as word-constituent
characters.  As another example, I can envision a dictionary of
acronyms and abbreviations, which might want to treat the period as a
word-constituent character, to support the likes of "a.k.a.".
Etc. etc. -- this is up to the dictionary to decide, and Emacs must
follow suit.

Also, please note that [:alpha:] in Emacs 25 means a much larger set
of characters than in previous versions, see NEWS.  It will in general
catch strings of characters that cannot possibly be TRT for a
single-language dictionary.  E.g.,

  (string-match "[[:alpha:]]+" "aβגд") => 0

> ​I meant [[:graph:]] and [^[:graph:]].​

This will match an even larger set in Emacs 25, I don't think we will
ever want that for spell-checking.

> ​Also, as I realised while preparing the patch for bug#25230, it is only hunspell that has special information
> about character classes. All the others just use [:alpha:]. So if it's good enough for ispell and aspell, can't it be
> good enough for enchant? (It just means that for now "direct Hunspell" is arguably better than "Hunspell via
> Enchant".)

Hunspell is the most modern and sophisticated speller, we certainly
don't want to degrade it.  Also, Aspell uses the dictionaries at least
for some of this info, see the function I pointed to above.

Once again, if Enchant uses a back-end for which we know how to find
this information, we should do so.

Bottom line, this information cannot be thrown away or ignored.  It is
important for correctly interfacing with a dictionary and for doing
TRT as the users expect.  Any modern speller program would benefit
from it, and therefore we should strive to provide such information to
ispell.el whenever we possibly can.

Thanks.

bug#17742: Acknowledgement (Support for enchant?)

[Agustin Martin]

On Mon, Dec 19, 2016 at 06:01:27PM +0200, Eli Zaretskii wrote:
> > From: Reuben Thomas
> 
> > Basic tests using [[:alpha:]] for casechars and [^[:alpha:]] for not-casechars seem to work OK.
> 
> For which language and dictionary?  This will definitely do the wrong
> thing for Hunspell he_IL dictionary I have here, which says:
> 
>   WORDCHARS אבגדהוזחטיכלמנסעפצקרשתםןךףץ'"
> 
> That is, it wants ' and " to be treated as word-constituent
> characters.  As another example, I can envision a dictionary of
> acronyms and abbreviations, which might want to treat the period as a
> word-constituent character, to support the likes of "a.k.a.".
> Etc. etc. -- this is up to the dictionary to decide, and Emacs must
> follow suit.
> 
> Also, please note that [:alpha:] in Emacs 25 means a much larger set
> of characters than in previous versions, see NEWS.  It will in general
> catch strings of characters that cannot possibly be TRT for a
> single-language dictionary.  E.g.,
> 
>   (string-match "[[:alpha:]]+" "aβגд") => 0
> 
> > ​I meant [[:graph:]] and [^[:graph:]].​
> 
> This will match an even larger set in Emacs 25, I don't think we will
> ever want that for spell-checking.

Hi,

Not following this very closely, but ispell.el still use [:alpha:] for
aspell and hunspell. If I remember this properly, old meaning means
something like "as for current locale" while it has now a much wider
meaning.

For the vast majority of systems this should not be a problem, but I wonder
if this can have some side effects for ispell.el in corner cases.
 
> > ​Also, as I realised while preparing the patch for bug#25230, it is only hunspell that has special information
> > about character classes. All the others just use [:alpha:]. So if it's good enough for ispell and aspell, can't it be
> > good enough for enchant? (It just means that for now "direct Hunspell" is arguably better than "Hunspell via
> > Enchant".)
> 
> Hunspell is the most modern and sophisticated speller, we certainly
> don't want to degrade it.  Also, Aspell uses the dictionaries at least
> for some of this info, see the function I pointed to above.
> 
> Once again, if Enchant uses a back-end for which we know how to find
> this information, we should do so.

About Enchant, last time I looked at it it was mostly intented for use
through libenchant, not through the standalone enchant binary, which was
more like some kind of testing tool. As a matter of fact its list of
options is quite short and it seems to lack support for personal
dictionaries. Since Emacs uses a pipe for spellchecking I do not think
we should worry too much about the enchant binary.

Things may have changed recently in enchant, but I would not expect that too
much, its man page still mentions myspell and not at all hunspell (so it
may be a bit outdated), although it seems to be able to use libhunspell.

Also, there is no easy way to know which particular spellchecking engine is
being used. Enchant uses $(datadir)/enchant and ~/.enchant config files to
define preferences, but I see no way to make enchant tell which one is being
used. So, it is not easy to parse dictionary info.

Sorry if I have missed some things. Gmail tags some of Reuben mails as spam
and puts them out of my usual workflow.

-- 
Agustin

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> Date: Mon, 19 Dec 2016 18:37:19 +0100
> From: Agustin Martin <agustin6martin@gmail.com>
> Cc: Reuben Thomas <rrt@sc3d.org>
> 
> >   (string-match "[[:alpha:]]+" "aβגд") => 0
> > 
> > > ​I meant [[:graph:]] and [^[:graph:]].​
> > 
> > This will match an even larger set in Emacs 25, I don't think we will
> > ever want that for spell-checking.
> 
> Hi,
> 
> Not following this very closely, but ispell.el still use [:alpha:] for
> aspell and hunspell. If I remember this properly, old meaning means
> something like "as for current locale" while it has now a much wider
> meaning.
> 
> For the vast majority of systems this should not be a problem, but I wonder
> if this can have some side effects for ispell.el in corner cases.

Yes, I think we should revise those places not to use [:alpha:].

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

On 19 December 2016 at 18:09, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Mon, 19 Dec 2016 18:37:19 +0100
> > From: Agustin Martin <agustin6martin@gmail.com>
> > Cc: Reuben Thomas <rrt@sc3d.org>
> >
> > >   (string-match "[[:alpha:]]+" "aβגд") => 0
> > >
> > > > ​I meant [[:graph:]] and [^[:graph:]].​
> > >
> > > This will match an even larger set in Emacs 25, I don't think we will
> > > ever want that for spell-checking.
> >
> > Hi,
> >
> > Not following this very closely, but ispell.el still use [:alpha:] for
> > aspell and hunspell. If I remember this properly, old meaning means
> > something like "as for current locale" while it has now a much wider
> > meaning.
> >
> > For the vast majority of systems this should not be a problem, but I
> wonder
> > if this can have some side effects for ispell.el in corner cases.
>
> Yes, I think we should revise those places not to use [:alpha:].
>

​I have proposed a revision of my patch for bug #​25230 to not use
[:alpha:] for the built-in ispell dictionaries.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 1933 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 1936 bytes --]

On 19 December 2016 at 17:37, Agustin Martin <agustin6martin@gmail.com>
wrote:

>
> About Enchant, last time I looked at it it was mostly intented for use
> through libenchant, not through the standalone enchant binary, which was
> more like some kind of testing tool. As a matter of fact its list of
> options is quite short and it seems to lack support for personal
> dictionaries.


​I have a patch currently under consideration by enchant's maintainer to
upgrade the functionality of the standalone enchant binary to be similar to
ispell, hunspell etc.​

See https://github.com/rrthomas/enchant/


> Also, there is no easy way to know which particular spellchecking engine is
> being used. Enchant uses $(datadir)/enchant and ~/.enchant config files to
> define preferences, but I see no way to make enchant tell which one is
> being
> used. So, it is not easy to parse dictionary info.
>

​enchant-lsmod lists the engine used for each dictionary, so this is
certainly possible, as I said earlier. However, it's not a good idea, I
think, to directly search for dictionary files and parse them, because this
breaks the abstraction of using the binary and/or library.​

Sorry if I have missed some things. Gmail tags some of Reuben mails as spam
> and puts them out of my usual workflow.
>

​This is caused by an unfortunate interaction between my domain's mail
configuration (which uses strict DMARC, so that if email does not match
some cryptographic signature, it should be discarded by receiving domains)
and the gnu.org listserv (which changes outgoing mail but keeps the
original sender, so that messages appear to come from me but do not match
the signature).

I have raised this before with the listserv admins, but they've not yet
been able to update their configuration (I know that sufficiently recent
mailman can be configured to work).​

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 3265 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 5130 bytes --]

On 19 December 2016 at 16:01, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Reuben Thomas <rrt@sc3d.org>
> > Date: Sun, 18 Dec 2016 23:39:54 +0000
> > Cc: 17742@debbugs.gnu.org
> >
> > I have not had any response to my enquiries yet, but I did some
> research, and neither GNU Aspell nor hunspell offer any way to get this
> information (about character classes of dictionaries) via their APIs.
>
> They provide this information in the dictionaries, and we glean it
> from there.  See ispell-parse-hunspell-affix-file and
> ispell-aspell-find-dictionary.
>

​The dictionaries are not part of the API (even where the format is
documented, the location may not be fixed), so it's not a good idea to rely
on them.

​Having discovered that Aspell does not provide this information (I checked
again, and ispell-aspell-find-dictionary does not find this information in
the dictionaries, except for limited information about otherchars; for
casechars and not-casechars it defaults to [:alpha:]), I shall investigate
with the hunspell maintainers.​


> Maybe there's a misunderstanding: I'm talking about the CASECHARS,
> NOT-CASECHARS, and OTHERCHARS parts of the dictionary data in
> ispell-dictionary-alist.


​There's no misunderstanding here, that's what I'm talking about.​

Each dictionary can (and many do) use some of the punctuation
> characters in the words it can handle.  A notable example is the
> apostrophe ' in English, used for the various suffixes that spellers
> support; similar features exist in other languages, but with possibly
> different punctuation characters.  Ispell.el must match that by using
> the speller's notion of a word, which must be independent of the
> current major mode's idea of what a word is.  This is where these
> character sets come into play, and I really cannot see how can
> ispell.el work well without using them as it does now.
>

​Currently, using casechars = [[:graph:]], if I put point over part of the
string " (XP) ", and run M-x ispell-word, it says "(XP) is correct". That's
good enough for me!

Note that merely using the characters declared in the dictionary may not be
enough: I have words like SC³D (I spell my company that way) in my personal
word lists. Other users might be more imaginative, and for example have
sequences of emoji. The list of characters in the dictionary is only a
minimum.​


> So we do need this information.  If Enchant doesn't provide it, we
> could still use the same technique as with Aspell and Hunspell,
> provided that we can figure out which back end(s) is/are used by
> Enchant.  Is that doable?
>

​Yes, that can be done, but it's fragile; that's why I'm trying to avoid
it.​

Ispell.el also supports spell-checking by words, in which case the
> above is not useful, because we need to figure out what is a word.
>

​See above. It's not clear to me that we need a very precise idea of what
constitutes a word.​

Moreover, even when we send entire lines to the speller, we want to
> skip lines that include only non-word characters.


​Why?​

Just look at the
> ​ ​
> callers of the above-mentioned accessor functions, and you will see
> how we use them.
>

​I have read this code. I see how we use them; it's just not clear to me
that it's necessary to use them thus.​

Hunspell is the most modern and sophisticated speller, we certainly
>
don't want to degrade it.


​No chance of that, this patch is only about Enchant.​

  Also, Aspell uses the dictionaries at least
> for some of this info, see the function I pointed to above.
>

​Only for otherchars, not casechars/not-casechars.​

Bottom line, this information cannot be thrown away or ignored.  It is
> important for correctly interfacing with a dictionary and for doing
> TRT as the users expect.  Any modern speller program would benefit
> from it, and therefore we should strive to provide such information to
> ispell.el whenever we possibly can.
>

​It is not a question of throwing away or ignoring information: the
information is simply not available through documented channels (at least
for Enchant). Yes, one can find the underlying engine and then use that
information to (try to) find the dictionaries, but one is then making a
number of brittle assumptions. And it's not clear that the information is
actually necessary to have.

It would be helpful if you could show a situation in which using [:graph:]
for enchant dictionaries. actually misbehaves in some way.

In fact, reading enchant's source code, it uses a fixed set of Unicode
classes for its own internal equivalent of casechars. Using that would make
sense (for Enchant! again, I'm not suggesting changing how we use hunspell).

One other data point: a senior LyX maintainer, Jean-Marc Lasgouttes, agrees
with you:

https://github.com/AbiWord/enchant/issues/17#issuecomment-267924304

He says that LyX has a "bug open somewhere" that suggests using this
information (but he didn't know it was available!).

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 8971 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 2089 bytes --]

On 19 December 2016 at 21:47, Reuben Thomas <rrt@sc3d.org> wrote:

>
> ​Having discovered that Aspell does not provide this information (I
> checked again, and ispell-aspell-find-dictionary does not find this
> information in the dictionaries, except for limited information about
> otherchars; for casechars and not-casechars it defaults to [:alpha:]), I
> shall investigate with the hunspell maintainers.​
>

​Hunspell has an open bug to export an API to get the casechars (which
hunspell calls "WORDCHARS"):

https://github.com/hunspell/hunspell/issues/282

It has been added to the 2.0 milestone. So it seems there is light at the
end of this particular tunnel.

I've added a comment that it would be good to have a way to get this
information from the hunspell binary too.

My suggestion on how to proceed would therefore be:

1. Assuming my patches to enchant are accepted sooner rather than later,
and there's a minor Enchant release soon, then accept an initial
implementation of Enchant support in Emacs with a fixed casechars value.
This is no worse than for Aspell.

1a. If someone wants to add a way for Emacs to parse hunspell dictionaries
when used via Enchant, that's fine by me as a temporary workaround.

2. When hunspell 2 is released, hopefully there will be an official channel
for Emacs to get this information. Any workaround introduced as per 1a
above would now be more solid.

3. Once one of Enchant's supported engines has an official way to get this
information, then it's a good time to add an API to Enchant too (and
support in the standalone binary).

Overall, there's no hurry. We have precise casechars for hunspell
dictionaries today (though as I mentioned elsewhere, there may still be
problems with using them). Enchant support for now is useful for the
spelling checkers it supports that Emacs does not; obviously, Emacs's
direct hunspell support is better for now than via Enchant. It would be
nice to fix that eventually and use only Enchant, but there's no need to
rush.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 3921 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Mon, 19 Dec 2016 21:27:18 +0000
> Cc: 17742@debbugs.gnu.org
> 
>     Also, there is no easy way to know which particular spellchecking engine is
>     being used. Enchant uses $(datadir)/enchant and ~/.enchant config files to
>     define preferences, but I see no way to make enchant tell which one is being
>     used. So, it is not easy to parse dictionary info.
> 
> ​enchant-lsmod lists the engine used for each dictionary, so this is certainly possible, as I said earlier. However, it's not a good idea, I think, to directly search for dictionary files and parse them, because this breaks the abstraction of using the binary and/or library

Emacs should cater to user expectations before it caters to
abstractions, IMO.  It is good to use abstractions, but if an
abstraction cannot support some useful feature, it should take a back
seat, and we should implement the feature based on lower-level
infrastructure.

In this case, using enchant-lsmod to figure out the engine, and then
using the existing code in ispell.el that supports that engine is an
okay solution when Enchant itself cannot provide the information we
need.

Of course, if/when Enchant provides the means to get at the
information we need, we should switch to using those means.

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Mon, 19 Dec 2016 21:47:42 +0000
> Cc: 17742@debbugs.gnu.org
> 
>     neither GNU Aspell nor hunspell offer any way to get this information (about character classes of dictionaries) via their APIs.
> 
>     They provide this information in the dictionaries, and we glean it
>     from there.  See ispell-parse-hunspell-affix-file and
>     ispell-aspell-find-dictionary.
> 
> ​The dictionaries are not part of the API (even where the format is documented, the location may not be fixed), so it's not a good idea to rely on them.

If there's no better way, then I see no problem in relying on the
dictionaries, and de-facto the results are satisfactory.

> ​Having discovered that Aspell does not provide this information (I checked again, and ispell-aspell-find-dictionary does not find this information in the dictionaries, except for limited information about otherchars; for casechars and not-casechars it defaults to [:alpha:]), I shall investigate with the hunspell maintainers.​

Aspell provides some of that, and there's no reason to ignore what it
does provide.

> ​Currently, using casechars = [[:graph:]], if I put point over part of the string " (XP) ", and run M-x ispell-word, it says "(XP) is correct". That's good enough for me!

Whether it's good enough depends on the dictionary and on what "(XP)"
means.  It could be that "(XP)", including the parentheses, is a word
the dictionary recognizes, something akin to "(C)", i.e. copyright
sign.  And it could be that the correct word is "XP", with the
parentheses acting as punctuation.  And there could be additional
alternatives.  Only the dictionary "knows" what is the right
alternative, and ispell.el should abide by the dictionary's rules, or
else it will not do what the user wants.  E.g., "XP" could not be in
the dictionary (as in fact I get when I try that with Hunspell), but
"(XP)" is.  So CASECHARS should be set up according to what the
dictionary expects, or you will have false positives and false
negatives.

> Note that merely using the characters declared in the dictionary may not be enough: I have words like SC³D (I spell my company that way) in my personal word lists. Other users might be more imaginative, and for example have sequences of emoji. The list of characters in the dictionary is only a minimum.​
 
That's why personal word list go together with dictionaries: they both
must use the same affix rules, so if you change to another dictionary
for the same language, your personal word list should also change, or
else you will get false negatives.

>     So we do need this information.  If Enchant doesn't provide it, we
>     could still use the same technique as with Aspell and Hunspell,
>     provided that we can figure out which back end(s) is/are used by
>     Enchant.  Is that doable?
> 
> ​Yes, that can be done, but it's fragile; that's why I'm trying to avoid it.​

I don't see why it would be fragile with Enchant when it isn't with
its back-ends.  And avoiding even fragile methods is worse than using
them, when there's no better way of gleaning the same information, and
the information is important (as it is in this case).

>     Ispell.el also supports spell-checking by words, in which case the
>     above is not useful, because we need to figure out what is a word.
>
> ​See above. It's not clear to me that we need a very precise idea of what constitutes a word.​

I think you are drawing too radical conclusions from trying that with
a single word and a single dictionary.  Which string was sent to the
speller in this case, and is that the string you expected to be sent?

>     Moreover, even when we send entire lines to the speller, we want to
>     skip lines that include only non-word characters.
>
> ​Why?​

To avoid false positives and false negatives, as explained above.

>     Hunspell is the most modern and sophisticated speller, we certainly
>     don't want to degrade it.
> 
> ​No chance of that, this patch is only about Enchant.​

First, Enchant could be using Hunspell as its engine, right?

And second, AFAIU this discussion started by you proposing to get rid
of CASECHARS etc., for all spellers, not just for Enchant, something
that will definitely cause degradation.

>       Also, Aspell uses the dictionaries at least
>     for some of this info, see the function I pointed to above.
>
> ​Only for otherchars, not casechars/not-casechars.​

Partial information is better than no information, IMO.

>     Bottom line, this information cannot be thrown away or ignored.  It is
>     important for correctly interfacing with a dictionary and for doing
>     TRT as the users expect.  Any modern speller program would benefit
>     from it, and therefore we should strive to provide such information to
>     ispell.el whenever we possibly can.
>
> ​It is not a question of throwing away or ignoring information: the information is simply not available through documented channels (at least for Enchant). Yes, one can find the underlying engine and then use that information to (try to) find the dictionaries, but one is then making a number of brittle assumptions. And it's not clear that the information is actually necessary to have.

It sounds like the important part of our disagreement is in the last
sentence.  If so, I hope I've succeeded to change your mind.  Failing
that, all I can suggest is to study the spelling rules of modern
speller, such as Hunspell, and see how this information is used there.

> It would be helpful if you could show a situation in which using [:graph:] for enchant dictionaries. actually misbehaves in some way.

I tried to explain that above: you will get falses and/or irrelevant
or missing corrections from the speller.  For example, if you send
"foo.bar", and the speller doesn't support '.' as a word-constituent
character, you will get separate suggestions for "foo" and "bar", and
won't get "foobar".

I also don't understand why you want to remove this information, that
is already there, is not harder to get with Enchant than it is without
it, and the code which supports it is already there?

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Mon, 19 Dec 2016 22:04:55 +0000
> Cc: 17742@debbugs.gnu.org
> 
> 1. Assuming my patches to enchant are accepted sooner rather than later, and there's a minor Enchant release soon, then accept an initial implementation of Enchant support in Emacs with a fixed casechars value. This is no worse than for Aspell.
> 
> 1a. If someone wants to add a way for Emacs to parse hunspell dictionaries when used via Enchant, that's fine by me as a temporary workaround.
> 
> 2. When hunspell 2 is released, hopefully there will be an official channel for Emacs to get this information. Any workaround introduced as per 1a above would now be more solid.
> 
> 3. Once one of Enchant's supported engines has an official way to get this information, then it's a good time to add an API to Enchant too (and support in the standalone binary).
> 
> Overall, there's no hurry. We have precise casechars for hunspell dictionaries today (though as I mentioned elsewhere, there may still be problems with using them). Enchant support for now is useful for the spelling checkers it supports that Emacs does not; obviously, Emacs's direct hunspell support is better for now than via Enchant. It would be nice to fix that eventually and use only Enchant, but there's no need to rush.

For someone who would like to use Enchant with Hunspell as its engine,
this sounds sub-optimal to me, if 1a is not implemented as part of the
initial Enchant support, or at all.  How will we be able in good faith
to advertise support for a new speller when in fact it degrades, even
if slightly, the direct interface with that very engine?

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 5818 bytes --]

On 20 December 2016 at 15:40, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Reuben Thomas <rrt@sc3d.org>
> > Date: Mon, 19 Dec 2016 21:47:42 +0000
> > Cc: 17742@debbugs.gnu.org
> >
> >     neither GNU Aspell nor hunspell offer any way to get this
> information (about character classes of dictionaries) via their APIs.
> >
> >     They provide this information in the dictionaries, and we glean it
> >     from there.  See ispell-parse-hunspell-affix-file and
> >     ispell-aspell-find-dictionary.
> >
> > ​The dictionaries are not part of the API (even where the format is
> documented, the location may not be fixed), so it's not a good idea to rely
> on them.
>
> If there's no better way, then I see no problem in relying on the
> dictionaries, and de-facto the results are satisfactory.
>

​Agreed.​


> > ​Having discovered that Aspell does not provide this information (I
> checked again, and ispell-aspell-find-dictionary does not find this
> information in the dictionaries, except for limited information about
> otherchars; for casechars and not-casechars it defaults to [:alpha:]), I
> shall investigate with the hunspell maintainers.​
>
> Aspell provides some of that, and there's no reason to ignore what it
> does provide.
>

​Agreed.​

​​
> Whether it's good enough depends on the dictionary and on what "(XP)"
> means.  It could be that "(XP)", including the parentheses, is a word
> the dictionary recognizes, something akin to "(C)", i.e. copyright
> sign.


​Thanks, that's a good example.​

So, if "(C)" is in the dictionary, then with [:graph:] as casechars, if I
run ispell-word with point anywhere in "(C)", Emacs will send "(C)", and it
will come back as correct. If casechars were only [:alpha:], then Emacs
would send "C", and it would come back as wrong.

Conversely, if "C" is in the dictionary, then if I run ispell-word with
casechars set to [:graph:] then Emacs will send "(C)" and it will come back
as correct (because Hunspell will ignore the non-wordchars characters). It
would also work with casechars set to [:alpha:].

So with casechars set to [:graph:], there's no false positive or false
negative.


> I don't see why it would be fragile with Enchant when it isn't with
> its back-ends.


​Because there's no guarantee that Enchant will continue to use backends in
the same way as at present.​

  And avoiding even fragile methods is worse than using
> them, when there's no better way of gleaning the same information, and
> the information is important (as it is in this case).
>

​Agreed.


> I think you are drawing too radical conclusions from trying that with
> a single word and a single dictionary.  Which string was sent to the
> speller in this case,


"​(XP)"​

and is that the string you expected to be sent?
>

​I don't have strong feelings about that.​

>     Moreover, even when we send entire lines to the speller, we want to
> >     skip lines that include only non-word characters.
> >
> > ​Why?​
>
> To avoid false positives and false negatives, as explained above.
>

​But such characters will be ignored by the spellchecker (unless perhaps
they occur in the personal word list). So I'm not sure how they would
generate false positives or negatives.​

First, Enchant could be using Hunspell as its engine, right?
>

​Sure.​

And second, AFAIU this discussion started by you proposing to get rid
> of CASECHARS etc., for all spellers, not just for Enchant, something
> that will definitely cause degradation.
>

​I didn't mean to propose that. I'm sorry if I gave that impression.​ I'm
just saying I don't want to put in the work now to add that support for
Enchant. I have not changed (and do not propose to change) the support for
Hunspell.

It sounds like the important part of our disagreement is in the last
> sentence.  If so, I hope I've succeeded to change your mind.  Failing
> that, all I can suggest is to study the spelling rules of modern
> speller, such as Hunspell, and see how this information is used there.
>

​As I already said, Hunspell does not provide this information to
applications. So consumers of Hunspell have two choices:

1. Use side channels (as Emacs does).

2. Have some arbitrary idea of what constitutes a word.

The fact that an API to get the wordchars from hunspell is only now being
considered for addition suggests to me that neither the maintainers of
hunspell nor the developers of hunspell-using programs have thought this
was particularly important.​

I tried to explain that above: you will get falses and/or irrelevant
> or missing corrections from the speller.  For example, if you send
> "foo.bar", and the speller doesn't support '.' as a word-constituent
> character, you will get separate suggestions for "foo" and "bar", and
> won't get "foobar".
>

What happens at the moment (with my Enchant patch) is I get the error
"Ispell and its process have different character maps". I wouldn't expect
"foobar" in any case, if "." is not a constituent character, though I might
be surprised to get a correction for a word I thought I wasn't pointing at
(but I could be surprised in this way in any case, if the dictionary has a
surprising set of wordchars).


> I also don't understand why you want to remove this information, that
> is already there, is not harder to get with Enchant than it is without
> it, and the code which supports it is already there?
>

​I'm not proposing to remove this information. I am proposing not to add it
for Enchant yet (because that will require extra work and code), and I am
hoping to end up with a simpler way to get it, via the API.

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 10148 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Tue, 20 Dec 2016 21:43:32 +0000
> Cc: 17742@debbugs.gnu.org
> 
> So with casechars set to [:graph:], there's no false positive or false negative.
 
The same argument could be used to claim that casechars can be removed
altogether, and we should just send everything to the speller.

What I'm saying is that using the correct character set there will
ensure that the user gets back useful suggestions, something that is
less probable without that.

>     I don't see why it would be fragile with Enchant when it isn't with
>     its back-ends.
> 
> ​Because there's no guarantee that Enchant will continue to use backends in the same way as at present.​

When it doesn't, we will revisit this issue and revise the code if
needed.  That is a hypothetical problem, so we don't need to solve it
yet.

>     >     Moreover, even when we send entire lines to the speller, we want to
>     >     skip lines that include only non-word characters.
>     >
>     > ​Why?​
>
>     To avoid false positives and false negatives, as explained above.
> 
> ​But such characters will be ignored by the spellchecker (unless perhaps they occur in the personal word list). So I'm not sure how they would generate false positives or negatives.​

You _assume_ they will be ignored.  You don't know that.  Using the
correct character set frees us from the need of making any such
assumtions.

> The fact that an API to get the wordchars from hunspell is only now being considered for addition suggests to me that neither the maintainers of hunspell nor the developers of hunspell-using programs have thought this was particularly important.​

I don't see any significance of that fact for us.  We understood the
importance of that data, and we use it to benefit our users.

>     I tried to explain that above: you will get falses and/or irrelevant
>     or missing corrections from the speller.  For example, if you send
>     "foo.bar", and the speller doesn't support '.' as a word-constituent
>     character, you will get separate suggestions for "foo" and "bar", and
>     won't get "foobar".
> 
> What happens at the moment (with my Enchant patch) is I get the error "Ispell and its process have different character maps". I wouldn't expect "foobar" in any case, if "." is not a constituent character, though I might be surprised to get a correction for a word I thought I wasn't pointing at (but I could be surprised in this way in any case, if the dictionary has a surprising set of wordchars).
 
Try the same example with foo'bar (including the apostrophe), and see
what I mean.

Once again, using the correct casechars makes the spell-checking
commands more useful than not using them.  A mismatch between what the
user considers casechars and the corresponding notion of the
speller/dictionary will make the results less useful.  Sure, it won't
be a total failure, but why give up quality if we can have it for a
small price of parsing a simple text file?  One of the strong points
of Aspell and Hunspell over Ispell is that the former are much smarter
in producing possible corrections.  This is what distinguishes an
excellent speller from a merely good one.  We should strive to support
that quality as best we can, IMO.

>     I also don't understand why you want to remove this information, that
>     is already there, is not harder to get with Enchant than it is without
>     it, and the code which supports it is already there?
> 
> ​I'm not proposing to remove this information. I am proposing not to add it for Enchant yet (because that will require extra work and code), and I am hoping to end up with a simpler way to get it, via the API.

I'm uneasy about that, as I already wrote.  If we add support for
Enchant now without also providing those character sets when we can,
then users of Hunspell will suddenly experience some degradation, even
if minor one, if they switch to using Hunspell via Enchant.  Soon
enough we will get bug reports about that.  So I'd rather we avoid
that.

Maybe we should simply wait until Enchant acquires those APIs.

Thanks.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

On 21 December 2016 at 17:13, Eli Zaretskii <eliz@gnu.org> wrote:

>
> Maybe we should simply wait until Enchant acquires those APIs.
>

​I'll revisit this when a new Enchant release is made. The timing and
future plans for Enchant and Emacs might affect the decision at that point,
or I might have found more energy to implement hunspell dictionary scanning
for Enchant!

-- 
http://rrt.sc3d.org

[-- Attachment #2: Type: text/html, Size: 962 bytes --]

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1.1: Type: text/plain, Size: 896 bytes --]

On 21 December 2016 at 17:32, Reuben Thomas <rrt@sc3d.org> wrote:

> On 21 December 2016 at 17:13, Eli Zaretskii <eliz@gnu.org> wrote:
>
>>
>> Maybe we should simply wait until Enchant acquires those APIs.
>>
>
> ​I'll revisit this when a new Enchant release is made. The timing and
> future plans for Enchant and Emacs might affect the decision at that point,
> or I might have found more energy to implement hunspell dictionary scanning
> for Enchant!
>
​
I attach updated patches, one for Enchant support, and the other to remove
a little historical cruft from ispell.el.

Enchant 2.1.0 (just released) adds support for getting the otherchars
character list (currently only supported on hunspell), and the ispell.el
support uses it.

Hence, I believe that the Enchant support should now satisfy Eli's
requirements.

-- 
https://rrt.sc3d.org <http://rrt.sc3d.org>

[-- Attachment #1.2: Type: text/html, Size: 2110 bytes --]

[-- Attachment #2: 0001-Add-Enchant-support-to-ispell.el-Bug-17742.patch --]
[-- Type: text/x-patch, Size: 9340 bytes --]

From 76ff43068ec914fdb405b675d139a94439dc698c Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 4 Dec 2016 22:39:27 +0000
Subject: [PATCH 1/2] Add Enchant support to ispell.el (Bug#17742)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/textmodes/ispell.el (ispell-program-name): Add “enchant”.
(ispell-really-enchant): Add variable.
(ispell-check-version): If using Enchant, check it’s new enough (at
least 1.6.1).  (Like the ispell check, this is absolute: cannot work
without.)
(ispell-enchant-dictionary-alist): Add variable.
(ispell-find-enchant-dictionaries): Add function, based on
ispell-find-aspell-dictionaries.
(ispell-set-spellchecker-params): Allow dictionary auto-detection for
Enchant, and call ispell-find-enchant-dictionaries to find them.  Use
old ispell name to locale mapping code for Enchant too.
(ispell-send-replacement): Make it work with Enchant.
---
 lisp/textmodes/ispell.el | 92 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 78 insertions(+), 14 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index 773023a..e6ca32f 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -208,6 +208,10 @@ ispell-choices-win-default-height
   :type 'integer
   :group 'ispell)
 
+;; XXX Add enchant to this list once enchant >= 2.1.0 is widespread.
+;; Before that, adding it is useless, as if it is found, it will just
+;; cause an error; and one of the other spelling engines below is
+;; almost certainly installed in any case, for enchant to use.
 (defcustom ispell-program-name
   (or (executable-find "aspell")
       (executable-find "ispell")
@@ -605,6 +609,8 @@ ispell-really-aspell
   "Non-nil if we can use Aspell extensions.")
 (defvar ispell-really-hunspell nil
   "Non-nil if we can use Hunspell extensions.")
+(defvar ispell-really-enchant nil
+  "Non-nil if we can use Enchant extensions.")
 (defvar ispell-encoding8-command nil
   "Command line option prefix to select encoding if supported, nil otherwise.
 If setting the encoding is supported by spellchecker and is selectable from
@@ -739,17 +745,26 @@ ispell-check-version
 		  (and (search-forward-regexp
 			"(but really Hunspell \\([0-9]+\\.[0-9\\.-]+\\)?)"
                         nil t)
+		       (match-string 1)))
+            (setq ispell-really-enchant
+		  (and (search-forward-regexp
+			"(but really Enchant \\([0-9]+\\.[0-9\\.-]+\\)?)"
+                        nil t)
 		       (match-string 1)))))
 
       (let* ((aspell8-minver   "0.60")
              (ispell-minver    "3.1.12")
              (hunspell8-minver "1.1.6")
+             (enchant-minver   "2.1.0")
              (minver (cond
                       ((not (version<= ispell-minver ispell-program-version))
                        ispell-minver)
                       ((and ispell-really-aspell
                             (not (version<= aspell8-minver ispell-really-aspell)))
-                       aspell8-minver))))
+                       aspell8-minver)
+                      ((and ispell-really-enchant
+                            (not (version<= enchant-minver ispell-really-enchant)))
+                       enchant-minver))))
 
         (if minver
 	    (error "%s release %s or greater is required"
@@ -1183,6 +1198,49 @@ ispell-find-hunspell-dictionaries
                     (list dict))
                   ispell-hunspell-dictionary-alist :test #'equal))))
 
+;; Make ispell.el work better with enchant.
+
+(defvar ispell-enchant-dictionary-alist nil
+  "An alist of parsed Enchant dicts and associated parameters.
+Internal use.")
+
+(defun ispell--call-enchant-lsmod (&rest args)
+  "Call enchant-lsmod with ARGS and return the output as string."
+  (with-output-to-string
+    (with-current-buffer
+        standard-output
+        (apply 'ispell-call-process
+               (concat ispell-program-name "-lsmod") nil t nil args))))
+
+(defun ispell--get-extra-word-characters (&optional lang)
+  "Get the extra word characters for LANG as a character class.
+If LANG is omitted, get the extra word characters for the default language."
+  (concat "[" (string-trim-right (apply 'ispell--call-enchant-lsmod
+                                        (append '("-word-chars") (if lang `(,lang))))) "]"))
+
+(defun ispell-find-enchant-dictionaries ()
+  "Find Enchant's dictionaries, and record in `ispell-enchant-dictionary-alist'."
+  (let* ((dictionaries
+	  (split-string
+	   (ispell--call-enchant-lsmod "-list-dicts" (buffer-string)) " ([^)]+)\n"))
+         (found
+          (mapcar #'(lambda (lang)
+                      `(,lang "[[:alpha:]]" "[^[:alpha:]]"
+                              ,(ispell--get-extra-word-characters) t nil nil utf-8))
+                  dictionaries)))
+    ;; Merge into FOUND any elements from the standard ispell-dictionary-base-alist
+    ;; which have no element in FOUND at all.
+    (dolist (dict ispell-dictionary-base-alist)
+      (unless (assoc (car dict) found)
+	(setq found (nconc found (list dict)))))
+    (setq ispell-enchant-dictionary-alist found)
+    ;; Add a default entry
+    (let ((default-dict
+            `(nil "[[:alpha:]]" "[^[:alpha:]]"
+                  ,(ispell--get-extra-word-characters)
+                  t nil nil utf-8)))
+      (push default-dict ispell-enchant-dictionary-alist))))
+
 ;; Set params according to the selected spellchecker
 
 (defvar ispell-last-program-name nil
@@ -1208,7 +1266,7 @@ ispell-set-spellchecker-params
 		   (setq ispell-library-directory (ispell-check-version))
 		   t)
 	       (error nil))
-	     ispell-encoding8-command)
+	     (or ispell-encoding8-command ispell-really-enchant))
 	;; auto-detection will only be used if spellchecker is not
 	;; ispell and supports a way to set communication to UTF-8.
 	(if ispell-really-aspell
@@ -1216,11 +1274,14 @@ ispell-set-spellchecker-params
 		(ispell-find-aspell-dictionaries))
 	  (if ispell-really-hunspell
 	      (or ispell-hunspell-dictionary-alist
-		  (ispell-find-hunspell-dictionaries)))))
+		  (ispell-find-hunspell-dictionaries))
+            (if ispell-really-enchant
+                (or ispell-enchant-dictionary-alist
+                    (ispell-find-enchant-dictionaries))))))
 
     ;; Substitute ispell-dictionary-alist with the list of
     ;; dictionaries corresponding to the given spellchecker.
-    ;; If a recent aspell or hunspell, use the list of really
+    ;; With programs that support it, use the list of really
     ;; installed dictionaries and add to it elements of the original
     ;; list that are not present there. Allow distro info.
     (let ((found-dicts-alist
@@ -1229,17 +1290,19 @@ ispell-set-spellchecker-params
 		   ispell-aspell-dictionary-alist
 		 (if ispell-really-hunspell
 		     ispell-hunspell-dictionary-alist))
-	     nil))
+	     (if ispell-really-enchant
+                 ispell-enchant-dictionary-alist
+               nil)))
 	  (ispell-dictionary-base-alist ispell-dictionary-base-alist)
 	  ispell-base-dicts-override-alist ; Override only base-dicts-alist
 	  all-dicts-alist)
 
       ;; While ispell and aspell (through aliases) use the traditional
-      ;; dict naming originally expected by ispell.el, hunspell
-      ;; uses locale based names with no alias.  We need to map
+      ;; dict naming originally expected by ispell.el, hunspell & Enchant
+      ;; use locale-based names with no alias.  We need to map
       ;; standard names to locale based names to make default dict
-      ;; definitions available for hunspell.
-      (if ispell-really-hunspell
+      ;; definitions available to these programs.
+      (if (or ispell-really-hunspell ispell-really-enchant)
 	  (let (tmp-dicts-alist)
 	    (dolist (adict ispell-dictionary-base-alist)
 	      (let* ((dict-name (nth 0 adict))
@@ -1264,7 +1327,7 @@ ispell-set-spellchecker-params
 			(setq ispell-args
 			      (nconc ispell-args (list "-d" dict-equiv)))
 		      (message
-		       "ispell-set-spellchecker-params: Missing Hunspell equiv for \"%s\". Skipping."
+		       "ispell-set-spellchecker-params: Missing equivalent for \"%s\". Skipping."
 		       dict-name)
 		      (setq skip-dict t)))
 
@@ -1306,7 +1369,7 @@ ispell-set-spellchecker-params
                          (nth 4 adict)   ; many-otherchars-p
                          (nth 5 adict)   ; ispell-args
                          (nth 6 adict)   ; extended-character-mode
-                         (if ispell-encoding8-command
+                         (if (or ispell-encoding8-command ispell-really-enchant)
                              'utf-8
                            (nth 7 adict)))
                       adict)
@@ -1742,9 +1805,10 @@ ispell-accept-output
 	    (erase-buffer)))))))
 
 (defun ispell-send-replacement (misspelled replacement)
-  "Notify Aspell that MISSPELLED should be spelled REPLACEMENT.
-This allows improving the suggestion list based on actual misspellings."
-  (and ispell-really-aspell
+  "Notify spell checker that MISSPELLED should be spelled REPLACEMENT.
+This allows improving the suggestion list based on actual misspellings.
+Only works for Aspell and Enchant."
+  (and (or ispell-really-aspell ispell-really-enchant)
        (ispell-send-string (concat "$$ra " misspelled "," replacement "\n"))))
 
 
-- 
2.7.4


[-- Attachment #3: 0002-Remove-old-comments-and-a-redundant-FIXME.patch --]
[-- Type: text/x-patch, Size: 3570 bytes --]

From 11a78847f2ffb87cdf93f00fd13ad7a5ed3f2a08 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Tue, 8 Aug 2017 15:56:03 +0100
Subject: [PATCH 2/2] Remove old comments and a redundant FIXME

* lisp/textmodes/ispell.el (ispell-process-line): Remove some old
commented code, a redundant FIXME, and outdated usage instructions.
---
 lisp/textmodes/ispell.el | 30 ++++--------------------------
 1 file changed, 4 insertions(+), 26 deletions(-)

diff --git a/lisp/textmodes/ispell.el b/lisp/textmodes/ispell.el
index e6ca32f..7ae2c0c 100644
--- a/lisp/textmodes/ispell.el
+++ b/lisp/textmodes/ispell.el
@@ -1,10 +1,8 @@
-;;; ispell.el --- interface to International Ispell Versions 3.1 and 3.2  -*- lexical-binding:t -*-
+;;; ispell.el --- interface to spell checkers  -*- lexical-binding:t -*-
 
 ;; Copyright (C) 1994-1995, 1997-2017 Free Software Foundation, Inc.
 
 ;; Author:           Ken Stevens <k.stevens@ieee.org>
-;; Status          : Release with 3.1.12+ and 3.2.0+ ispell.
-;; Keywords: unix wp
 
 ;; This file is part of GNU Emacs.
 
@@ -21,23 +19,11 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
 
-;; Note: version numbers and time stamp are not updated
-;;   when this file is edited for release with GNU Emacs.
-
 ;;; Commentary:
 
 ;; INSTRUCTIONS
 
-;;   This code contains a section of user-settable variables that you
-;; should inspect prior to installation.  Look past the end of the history
-;; list.  Set them up for your locale and the preferences of the majority
-;; of the users.  Otherwise the users may need to set a number of variables
-;; themselves.
-;;   You particularly may want to change the default dictionary for your
-;; country and language.
-;;   Most dictionary changes should be made in this file so all users can
-;; enjoy them.  Local or modified dictionaries are supported in your .emacs
-;; file.  Use the variable `ispell-local-dictionary-alist' to specify
+;;   Use the variable `ispell-local-dictionary-alist' to specify
 ;; your own dictionaries.
 
 ;;  Depending on the mail system you use, you may want to include these:
@@ -112,7 +98,7 @@
 ;;  Need a way to select between different character mappings without separate
 ;;    dictionary entries.
 ;;  Multi-byte characters if not defined by current dictionary may result in the
-;;    evil "misalignment error" in some versions of MULE Emacs.
+;;    evil "misalignment error" in some versions of Emacs.
 ;;  On some versions of Emacs, growing the minibuffer fails.
 ;;    see `ispell-help-in-bufferp'.
 ;;  Recursive edits (?C-r or ?R) inside a keyboard text replacement check (?r)
@@ -3524,17 +3510,9 @@ ispell-process-line
                       (setq ispell-filter recheck-region
                             recheck-region nil
                             replace replace-word)))))
+              (setq shift (+ shift (- (length replace) word-len)))))
 
-              (setq shift (+ shift (- (length replace) word-len)))
-
-              ;; Move line-start across word...
-              ;; new shift function does this now...
-              ;;(set-marker line-start (+ line-start
-              ;;			(- (length replace)
-              ;;			   (length (car poss)))))
-              ))
             (if (not ispell-quit)
-                ;; FIXME: remove redundancy with identical code above.
                 (let (message-log-max)
                   (message
                    "Continuing spelling check using %s with %s dictionary..."
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Wed, 9 Aug 2017 12:35:16 +0100
> Cc: 17742@debbugs.gnu.org
> 
> On 21 December 2016 at 17:32, Reuben Thomas <rrt@sc3d.org> wrote:
> 
>  On 21 December 2016 at 17:13, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>  Maybe we should simply wait until Enchant acquires those APIs.
> 
>  ​I'll revisit this when a new Enchant release is made. The timing and future plans for Enchant and Emacs
>  might affect the decision at that point, or I might have found more energy to implement hunspell
>  dictionary scanning for Enchant!
> 
> ​I attach updated patches, one for Enchant support, and the other to remove a little historical cruft from
> ispell.el.
> 
> Enchant 2.1.0 (just released) adds support for getting the otherchars character list (currently only supported
> on hunspell), and the ispell.el support uses it.
> 
> Hence, I believe that the Enchant support should now satisfy Eli's requirements.

Thanks, this looks OK to me.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

On 18 August 2017 at 09:54, Eli Zaretskii <eliz@gnu.org> wrote:
>
> Thanks, this looks OK to me.

Thanks for the review. I attach one final patch, to document the
changes; is that OK? I have removed the list of spell-checkers from
all but one place in the manual, for ease of future maintenance.

-- 
https://rrt.sc3d.org

[-- Attachment #2: 0003-Document-Enchant-support.patch --]
[-- Type: text/x-patch, Size: 4357 bytes --]

From cf74c27ba1401aba216267b5a9900e659d1b2a25 Mon Sep 17 00:00:00 2001
From: Reuben Thomas <rrt@sc3d.org>
Date: Sun, 20 Aug 2017 13:56:38 +0100
Subject: [PATCH 3/3] Document Enchant support

* doc/emacs/fixit.texi: Mention Enchant.
* doc/misc/efaq.texi: Likewise.
* etc/NEWS: Add an item on Enchant support.
---
 doc/emacs/fixit.texi | 14 +++++++-------
 doc/misc/efaq.texi   | 12 ++++++++----
 etc/NEWS             | 11 +++++++++++
 3 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/doc/emacs/fixit.texi b/doc/emacs/fixit.texi
index f833f57..f2dba83 100644
--- a/doc/emacs/fixit.texi
+++ b/doc/emacs/fixit.texi
@@ -227,7 +227,7 @@ Spelling
 
   This section describes the commands to check the spelling of a
 single word or of a portion of a buffer.  These commands only work if
-the spelling checker program Aspell, Ispell or Hunspell is installed.
+the spelling checker program Hunspell, Aspell, Ispell or Enchant is installed.
 These programs are not part of Emacs, but one of them is usually
 installed in GNU/Linux and other free operating systems.
 @ifnottex
@@ -249,9 +249,9 @@ Spelling
 Check and correct spelling in a draft mail message, excluding cited
 material.
 @item M-x ispell-change-dictionary @key{RET} @var{dict} @key{RET}
-Restart the Aspell/Ispell/Hunspell process, using @var{dict} as the dictionary.
+Restart the spell-checker process, using @var{dict} as the dictionary.
 @item M-x ispell-kill-ispell
-Kill the Aspell/Ispell/Hunspell subprocess.
+Kill the spell-checker subprocess.
 @item M-@key{TAB}
 @itemx @key{ESC} @key{TAB}
 @itemx C-M-i
@@ -319,8 +319,8 @@ Spelling
 editing session and for this buffer.
 
 @item i
-Insert this word in your private dictionary file so that Aspell or Ispell
-or Hunspell will consider it correct from now on, even in future sessions.
+Insert this word in your private dictionary file so that it will be
+considered correct from now on, even in future sessions.
 
 @item m
 Like @kbd{i}, but you can also specify dictionary completion
@@ -364,7 +364,7 @@ Spelling
 
 @cindex @code{ispell} program
 @findex ispell-kill-ispell
-  Once started, the Aspell or Ispell or Hunspell subprocess continues
+  Once started, the spell-checker subprocess continues
 to run, waiting for something to do, so that subsequent spell checking
 commands complete more quickly.  If you want to get rid of the
 process, use @kbd{M-x ispell-kill-ispell}.  This is not usually
@@ -375,7 +375,7 @@ Spelling
 @vindex ispell-local-dictionary
 @vindex ispell-personal-dictionary
 @findex ispell-change-dictionary
-  Ispell, Aspell and Hunspell look up spelling in two dictionaries:
+  Spell-checkers look up spelling in two dictionaries:
 the standard dictionary and your personal dictionary.  The standard
 dictionary is specified by the variable @code{ispell-local-dictionary}
 or, if that is @code{nil}, by the variable @code{ispell-dictionary}.
diff --git a/doc/misc/efaq.texi b/doc/misc/efaq.texi
index e9cfe7a..c329984 100644
--- a/doc/misc/efaq.texi
+++ b/doc/misc/efaq.texi
@@ -3464,22 +3464,26 @@ Spell-checkers
 @section Spell-checkers
 @cindex Spell-checker
 @cindex Checking spelling
-@cindex Ispell
-@cindex Aspell
 @cindex Hunspell
+@cindex Aspell
+@cindex Ispell
+@cindex Enchant
 
 Various spell-checkers are compatible with Emacs, including:
 
 @table @b
 
+@item Hunspell
+@uref{http://hunspell.sourceforge.net/}
+
 @item GNU Aspell
 @uref{http://aspell.net/}
 
 @item Ispell
 @uref{http://fmg-www.cs.ucla.edu/geoff/ispell.html}
 
-@item Hunspell
-@uref{http://hunspell.sourceforge.net/}
+@item Enchant
+@uref{https://abiword.github.io/enchant/}
 
 @end table
 
diff --git a/etc/NEWS b/etc/NEWS
index 7774d75..0939033 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1084,6 +1084,17 @@ file.
 ** New DNS mode command 'dns-mode-ipv6-to-nibbles' to convert IPv6 addresses
 to a format suitable for reverse lookup zone files.
 
+** Ispell
+
++++
+*** Enchant (version 2.1.0 or later required) is now supported as a
+spell-checker.  Enchant is a meta-spell-checker that uses providers
+such as Hunspell to do the actual checking.  With it, users can use
+spell-checkers not directly supported by Emacs, such as Voikko, Hspell
+and AppleSpell, more easily share personal word-lists with other
+programs, and configure different spelling-checkers for different
+languages.
+
 ** Flymake
 
 +++
-- 
2.7.4

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Sun, 20 Aug 2017 14:02:06 +0100
> Cc: 17742@debbugs.gnu.org
> 
> On 18 August 2017 at 09:54, Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > Thanks, this looks OK to me.
> 
> Thanks for the review. I attach one final patch, to document the
> changes; is that OK? I have removed the list of spell-checkers from
> all but one place in the manual, for ease of future maintenance.

This is fine, thanks.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

On 20 August 2017 at 15:42, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Reuben Thomas <rrt@sc3d.org>
>> Date: Sun, 20 Aug 2017 14:02:06 +0100
>> Cc: 17742@debbugs.gnu.org
>>
>> On 18 August 2017 at 09:54, Eli Zaretskii <eliz@gnu.org> wrote:
>> >
>> > Thanks, this looks OK to me.
>>
>> Thanks for the review. I attach one final patch, to document the
>> changes; is that OK? I have removed the list of spell-checkers from
>> all but one place in the manual, for ease of future maintenance.
>
> This is fine, thanks.

Thanks again; changes pushed.

-- 
https://rrt.sc3d.org

bug#17742:

[Reuben Thomas]

bug#17742: Acknowledgement (Support for enchant?)

[Eli Zaretskii]

> From: Reuben Thomas <rrt@sc3d.org>
> Date: Sun, 20 Aug 2017 15:50:32 +0100
> Cc: 17742@debbugs.gnu.org
> 
> Thanks again; changes pushed.

Could you please take care of this warning:

  In end of data:
  textmodes/ispell.el:4221:1:Warning: the function `string-trim-right' is not
      known to be defined.

Thanks.

bug#17742: Acknowledgement (Support for enchant?)

[Reuben Thomas]

On 20 August 2017 at 20:34, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Reuben Thomas <rrt@sc3d.org>
>> Date: Sun, 20 Aug 2017 15:50:32 +0100
>> Cc: 17742@debbugs.gnu.org
>>
>> Thanks again; changes pushed.
>
> Could you please take care of this warning:
>
>   In end of data:
>   textmodes/ispell.el:4221:1:Warning: the function `string-trim-right' is not
>       known to be defined.
>
> Thanks.

Apologies; fix pushed.

-- 
https://rrt.sc3d.org