bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

Hello,

I have hunspell installed in the system with two default dictionaries.
I was trying to setup ispell so that it works properly and I couldn't.
So I started looking at the sources for ispell and I found one part that
can't work.  It may be intentional, but the real problem is that I can't
use spellchecker at all.  When `emacs -Q` is ran, it is enough to do M-$
with *no additinal settings*.  The error message (and following messages
for consecutive M-$) are visible below in 'Recent messages'.

I could only trace the problem to the ispell.el where the error is
printed.  The reason for that is probably the fact that `hunspell -D`
has this output on my system:

SEARCH PATH:
.::/usr/share/hunspell:/usr/share/myspell:/usr/share/myspell/dicts:/Library/Spelling:/home/nert/.openoffice.org/3/user/wordbook:.openoffice.org2/user/wordbook:.openoffice.org2.0/user/wordbook:Library/Spelling:/opt/openoffice.org/basis3.0/share/dict/ooo:/usr/lib/openoffice.org/basis3.0/share/dict/ooo:/opt/openoffice.org2.4/share/dict/ooo:/usr/lib/openoffice.org2.4/share/dict/ooo:/opt/openoffice.org2.3/share/dict/ooo:/usr/lib/openoffice.org2.3/share/dict/ooo:/opt/openoffice.org2.2/share/dict/ooo:/usr/lib/openoffice.org2.2/share/dict/ooo:/opt/openoffice.org2.1/share/dict/ooo:/usr/lib/openoffice.org2.1/share/dict/ooo:/opt/openoffice.org2.0/share/dict/ooo:/usr/lib/openoffice.org2.0/share/dict/ooo
AVAILABLE DICTIONARIES (path is not mandatory for -d option):
/usr/share/hunspell/cs_CZ
/usr/share/hunspell/en_CA
/usr/share/hunspell/en_ZA
/usr/share/hunspell/en_US
/usr/share/hunspell/en_GB
/usr/share/hunspell/en_AU
/usr/share/myspell/cs_CZ
/usr/share/myspell/en_CA
/usr/share/myspell/hyph_cs_CZ
/usr/share/myspell/en_ZA
/usr/share/myspell/en_US
/usr/share/myspell/en_GB
/usr/share/myspell/en_AU
/usr/share/myspell/hyph_en_GB
LOADED DICTIONARY:
/usr/share/hunspell/en_GB.aff
/usr/share/hunspell/en_GB.dic
LOADED DICTIONARY:
/usr/share/hunspell/cs_CZ.aff
/usr/share/hunspell/cs_CZ.dic
Hunspell 1.6.0

I tried various additional settings, setting the multi-dic for hunspell,
setting default dictionaries and nothing helped.  Can't ispell just
detect all the .aff files and then automatically use en_GB,cs_CZ as a
multi-dictionary for hunspell?  That's what I want anyway.

Let me know if I can provide any more information and thank you very
much in advance for looking into this issue.

Martin

P.S.: I seem to recall my previous configuration that made this working,
      but it was probably before lot of emacs and hunspell upgrades.
      The spellchecking is not working for some time for me, it's just
      now that I started looking into this issue.

P.P.S.: If I have (flyspell-mode) in a 'message-mode-hook and I start
        emacsclient with a file that automatically starts in
        message-mode, then emacs crashes due to the default dictionary
        being nil or something in that regard.


In GNU Emacs 25.2.1 (x86_64-pc-linux-gnu)
 of 2017-02-20 built on caroline
System Description:	Gentoo Base System release 2.3

Configured using:
 'configure --prefix=/usr --build=x86_64-pc-linux-gnu
 --host=x86_64-pc-linux-gnu --mandir=/usr/share/man
 --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
 --localstatedir=/var/lib --disable-dependency-tracking
 --disable-silent-rules --docdir=/usr/share/doc/emacs-25.2_rc1
 --htmldir=/usr/share/doc/emacs-25.2_rc1/html --libdir=/usr/lib64
 --program-suffix=-emacs-25 --infodir=/usr/share/info/emacs-25
 --localstatedir=/var
 --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
 --with-gameuser=:gamestat --without-compress-install
 --with-file-notification=inotify --enable-acl --with-dbus
 --without-modules --with-gpm --without-hesiod --with-kerberos
 --with-kerberos5 --with-xml2 --without-selinux --with-gnutls
 --with-wide-int --with-zlib --with-sound=alsa --with-x --without-ns
 --without-gconf --without-gsettings --with-toolkit-scroll-bars
 --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
 --with-imagemagick --with-xft --without-cairo --without-libotf
 --without-m17n-flt --with-x-toolkit=no
 GENTOO_PACKAGE=app-editors/emacs-25.2_rc1 'CFLAGS=-O2 -pipe -ggdb
 -march=native' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS NOTIFY ACL GNUTLS
LIBXML2 FREETYPE XFT ZLIB X11

Important settings:
  value of $LC_CTYPE: en_GB.utf8
  value of $LANG: en_GB.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
ispell-find-hunspell-dictionaries: ispell-fhd: Default dict already defined as (en_GB /usr/share/hunspell/en_GB.aff).  Not using /usr/share/hunspell/cs_CZ.aff.

Starting new Ispell process /usr/bin/hunspell with default dictionary...
split-string: Wrong type argument: stringp, nil
next-line: End of buffer
Starting new Ispell process /usr/bin/hunspell with default dictionary...
split-string: Wrong type argument: stringp, nil
Starting new Ispell process /usr/bin/hunspell with default dictionary...
split-string: Wrong type argument: stringp, nil
Starting new Ispell process /usr/bin/hunspell with default dictionary...
split-string: Wrong type argument: stringp, nil

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils ispell term/xterm xterm
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
font-render-setting x multi-tty make-network-process emacs)

Memory information:
((conses 16 91424 6545)
 (symbols 48 20210 0)
 (miscs 40 43 160)
 (strings 32 15556 4780)
 (string-bytes 1 446860)
 (vectors 16 10101)
 (vector-slots 8 389043 14706)
 (floats 8 167 358)
 (intervals 56 258 0)
 (buffers 976 18)
 (heap 1024 13197 776))

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Tue, 21 Feb 2017 10:18:57 +0100
> From: Martin Kletzander <mk@linux.com>
> 
> I have hunspell installed in the system with two default dictionaries.

Please tell more details about the meaning of this.  Do you mean you
wanted to start Hunspell with 2 dictionaries so that it could
spell-check text that mixes 2 different languages?  Or did you mean
something else?  IOW, the "with two default dictionaries" part
confuses me.

> I was trying to setup ispell so that it works properly and I couldn't.
> So I started looking at the sources for ispell and I found one part that
> can't work.  It may be intentional, but the real problem is that I can't
> use spellchecker at all.  When `emacs -Q` is ran, it is enough to do M-$
> with *no additinal settings*.  The error message (and following messages
> for consecutive M-$) are visible below in 'Recent messages'.
> 
> I could only trace the problem to the ispell.el where the error is
> printed.  The reason for that is probably the fact that `hunspell -D`
> has this output on my system:

I don't immediately see anything wrong here.  Does spell-checking work
if you set it up to use just one default dictionary, as usual?

Thanks.

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Tue, Feb 21, 2017 at 07:23:08PM +0200, Eli Zaretskii wrote:
>> Date: Tue, 21 Feb 2017 10:18:57 +0100
>> From: Martin Kletzander <mk@linux.com>
>>
>> I have hunspell installed in the system with two default dictionaries.
>
>Please tell more details about the meaning of this.  Do you mean you
>wanted to start Hunspell with 2 dictionaries so that it could
>spell-check text that mixes 2 different languages?  Or did you mean
>something else?  IOW, the "with two default dictionaries" part
>confuses me.
>

Sorry, Idon't know spellcheckers that much.  Ilooked at hunspell's man
page, learned a bit, so I'll try to explain it better this time.

I have multiple dictionaries installed.  Hunspell finds all of them, but
selects *two* of them.  According to the man page, that is done
according to LC_ALL, LC_MESSAGES, and LANG environment variables.
Becuase hunspell selects more of them (which is something I want it to
do), it lists both of them as "LOADED DICTIONARY", ispell parses those
(actually just lines ending with ".aff") and if it finds more than one,
it errors out.

>> I was trying to setup ispell so that it works properly and I couldn't.
>> So I started looking at the sources for ispell and I found one part that
>> can't work.  It may be intentional, but the real problem is that I can't
>> use spellchecker at all.  When `emacs -Q` is ran, it is enough to do M-$
>> with *no additinal settings*.  The error message (and following messages
>> for consecutive M-$) are visible below in 'Recent messages'.
>>
>> I could only trace the problem to the ispell.el where the error is
>> printed.  The reason for that is probably the fact that `hunspell -D`
>> has this output on my system:
>
>I don't immediately see anything wrong here.  Does spell-checking work
>if you set it up to use just one default dictionary, as usual?
>

That's another problem.  Since the initialization itself fails, it
errors out before it checks any set variables.  So no matter what I set
up, it ends up with the same behaviour.  If I remove the second
dictionary from my system, however, it works.

I tried workaround with a wrapper around hunspell that modifies the
output of `hunspell -D` and it works, but that's really ugly.

>Thanks.

Thank you,
Martin

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Tue, 21 Feb 2017 21:24:54 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> I have multiple dictionaries installed.  Hunspell finds all of them, but
> selects *two* of them.  According to the man page, that is done
> according to LC_ALL, LC_MESSAGES, and LANG environment variables.

What are the values of those environment variables?

> Becuase hunspell selects more of them (which is something I want it to
> do), it lists both of them as "LOADED DICTIONARY", ispell parses those
> (actually just lines ending with ".aff") and if it finds more than one,
> it errors out.

The way to run Hunspell with more than one dictionary is hinted upon
in the doc string of the command ispell-hunspell-add-multi-dic.  After
you invoke that command and specify the dictionaries you want to use
(e.g., in your case "en_GB,cs_CZ"), type this:

  M-x ispell-change-dictionary RET en_GB,cs_CZ RET

Then try spell-checking.  It should work with that arrangement.  It is
possible that you will first need to modify your environment variables
to prevent Hunspell from loading more than one dictionary by default.

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Tue, Feb 21, 2017 at 10:44:03PM +0200, Eli Zaretskii wrote:
>> Date: Tue, 21 Feb 2017 21:24:54 +0100
>> From: Martin Kletzander <mk@linux.com>
>> Cc: 25830@debbugs.gnu.org
>>
>> I have multiple dictionaries installed.  Hunspell finds all of them, but
>> selects *two* of them.  According to the man page, that is done
>> according to LC_ALL, LC_MESSAGES, and LANG environment variables.
>
>What are the values of those environment variables?
>

$ locale
LANG=en_GB.utf8
LC_CTYPE=en_GB.utf8
LC_NUMERIC="en_GB.utf8"
LC_TIME="en_GB.utf8"
LC_COLLATE="en_GB.utf8"
LC_MONETARY="en_GB.utf8"
LC_MESSAGES="en_GB.utf8"
LC_PAPER="en_GB.utf8"
LC_NAME="en_GB.utf8"
LC_ADDRESS="en_GB.utf8"
LC_TELEPHONE="en_GB.utf8"
LC_MEASUREMENT="en_GB.utf8"
LC_IDENTIFICATION="en_GB.utf8"
LC_ALL=

Honestly, I don't know where it takes the cs_CZ from, *but* if it
didn't, I would have to modify it so that it does.  Simply because
that's what I want it to do by default.

>> Becuase hunspell selects more of them (which is something I want it to
>> do), it lists both of them as "LOADED DICTIONARY", ispell parses those
>> (actually just lines ending with ".aff") and if it finds more than one,
>> it errors out.
>
>The way to run Hunspell with more than one dictionary is hinted upon
>in the doc string of the command ispell-hunspell-add-multi-dic.  After

If I try

  M-x ispell-change-dictionary RET en_GB,cs_CZ RET

I get the same error.

>you invoke that command and specify the dictionaries you want to use
>(e.g., in your case "en_GB,cs_CZ"), type this:
>
>  M-x ispell-change-dictionary RET en_GB,cs_CZ RET
>

M-x ispell-change-dictionary RET errors out with the same error.  If I
run it again, there are no dictionaries loaded, so I cannot use any
parameter for it except 'default'.

>Then try spell-checking.  It should work with that arrangement.  It is
>possible that you will first need to modify your environment variables
>to prevent Hunspell from loading more than one dictionary by default.

Why would I need to do that?  For me hunspell is working as I want it
to, exactly.

I honestly think that ispell should accept multiple dictionaries and
either behave like a multi-dic by default or just not error out when it
parses the second dictionary (less favourable option, though).  I would
love to add that functionality, but I'm not that familiar with the code,
unfortunately =(

Thanks,
Martin

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Wed, 22 Feb 2017 11:40:25 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> $ locale
> LANG=en_GB.utf8
> LC_CTYPE=en_GB.utf8
> LC_NUMERIC="en_GB.utf8"
> LC_TIME="en_GB.utf8"
> LC_COLLATE="en_GB.utf8"
> LC_MONETARY="en_GB.utf8"
> LC_MESSAGES="en_GB.utf8"
> LC_PAPER="en_GB.utf8"
> LC_NAME="en_GB.utf8"
> LC_ADDRESS="en_GB.utf8"
> LC_TELEPHONE="en_GB.utf8"
> LC_MEASUREMENT="en_GB.utf8"
> LC_IDENTIFICATION="en_GB.utf8"
> LC_ALL=
> 
> Honestly, I don't know where it takes the cs_CZ from, *but* if it
> didn't, I would have to modify it so that it does.  Simply because
> that's what I want it to do by default.

Interesting.  What version of Hunspell is that?

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Wed, Feb 22, 2017 at 06:20:19PM +0200, Eli Zaretskii wrote:
>> Date: Wed, 22 Feb 2017 11:40:25 +0100
>> From: Martin Kletzander <mk@linux.com>
>> Cc: 25830@debbugs.gnu.org
>>
>> $ locale
>> LANG=en_GB.utf8
>> LC_CTYPE=en_GB.utf8
>> LC_NUMERIC="en_GB.utf8"
>> LC_TIME="en_GB.utf8"
>> LC_COLLATE="en_GB.utf8"
>> LC_MONETARY="en_GB.utf8"
>> LC_MESSAGES="en_GB.utf8"
>> LC_PAPER="en_GB.utf8"
>> LC_NAME="en_GB.utf8"
>> LC_ADDRESS="en_GB.utf8"
>> LC_TELEPHONE="en_GB.utf8"
>> LC_MEASUREMENT="en_GB.utf8"
>> LC_IDENTIFICATION="en_GB.utf8"
>> LC_ALL=
>>
>> Honestly, I don't know where it takes the cs_CZ from, *but* if it
>> didn't, I would have to modify it so that it does.  Simply because
>> that's what I want it to do by default.
>
>Interesting.  What version of Hunspell is that?

1.6.0

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Wed, 22 Feb 2017 17:43:10 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> On Wed, Feb 22, 2017 at 06:20:19PM +0200, Eli Zaretskii wrote:
> >> Date: Wed, 22 Feb 2017 11:40:25 +0100
> >> From: Martin Kletzander <mk@linux.com>
> >> Cc: 25830@debbugs.gnu.org
> >>
> >> $ locale
> >> LANG=en_GB.utf8
> >> LC_CTYPE=en_GB.utf8
> >> LC_NUMERIC="en_GB.utf8"
> >> LC_TIME="en_GB.utf8"
> >> LC_COLLATE="en_GB.utf8"
> >> LC_MONETARY="en_GB.utf8"
> >> LC_MESSAGES="en_GB.utf8"
> >> LC_PAPER="en_GB.utf8"
> >> LC_NAME="en_GB.utf8"
> >> LC_ADDRESS="en_GB.utf8"
> >> LC_TELEPHONE="en_GB.utf8"
> >> LC_MEASUREMENT="en_GB.utf8"
> >> LC_IDENTIFICATION="en_GB.utf8"
> >> LC_ALL=
> >>
> >> Honestly, I don't know where it takes the cs_CZ from, *but* if it
> >> didn't, I would have to modify it so that it does.  Simply because
> >> that's what I want it to do by default.
> >
> >Interesting.  What version of Hunspell is that?
> 
> 1.6.0

Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
automatically when we find more than one dictionary loaded by
Hunspell, as it reports when invoked with the -D option.

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Sat, Feb 25, 2017 at 10:24:38AM +0200, Eli Zaretskii wrote:
>> Date: Wed, 22 Feb 2017 17:43:10 +0100
>> From: Martin Kletzander <mk@linux.com>
>> Cc: 25830@debbugs.gnu.org
>>
>> On Wed, Feb 22, 2017 at 06:20:19PM +0200, Eli Zaretskii wrote:
>> >> Date: Wed, 22 Feb 2017 11:40:25 +0100
>> >> From: Martin Kletzander <mk@linux.com>
>> >> Cc: 25830@debbugs.gnu.org
>> >>
>> >> $ locale
>> >> LANG=en_GB.utf8
>> >> LC_CTYPE=en_GB.utf8
>> >> LC_NUMERIC="en_GB.utf8"
>> >> LC_TIME="en_GB.utf8"
>> >> LC_COLLATE="en_GB.utf8"
>> >> LC_MONETARY="en_GB.utf8"
>> >> LC_MESSAGES="en_GB.utf8"
>> >> LC_PAPER="en_GB.utf8"
>> >> LC_NAME="en_GB.utf8"
>> >> LC_ADDRESS="en_GB.utf8"
>> >> LC_TELEPHONE="en_GB.utf8"
>> >> LC_MEASUREMENT="en_GB.utf8"
>> >> LC_IDENTIFICATION="en_GB.utf8"
>> >> LC_ALL=
>> >>
>> >> Honestly, I don't know where it takes the cs_CZ from, *but* if it
>> >> didn't, I would have to modify it so that it does.  Simply because
>> >> that's what I want it to do by default.
>> >
>> >Interesting.  What version of Hunspell is that?
>>
>> 1.6.0
>
>Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
>automatically when we find more than one dictionary loaded by
>Hunspell, as it reports when invoked with the -D option.

Thank you.  Feel free too Cc me on any patch for that, I'll gladly test
it out (and learn more about the underlying code).

Martin

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Sat, 25 Feb 2017 13:15:23 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> >Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
> >automatically when we find more than one dictionary loaded by
> >Hunspell, as it reports when invoked with the -D option.
> 
> Thank you.  Feel free too Cc me on any patch for that, I'll gladly test
> it out (and learn more about the underlying code).

OK, thanks, will do.

Btw, do you have a DICTIONARY environment variable defined?  If yes,
what is its value?

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Sat, Feb 25, 2017 at 02:41:27PM +0200, Eli Zaretskii wrote:
>> Date: Sat, 25 Feb 2017 13:15:23 +0100
>> From: Martin Kletzander <mk@linux.com>
>> Cc: 25830@debbugs.gnu.org
>>
>> >Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
>> >automatically when we find more than one dictionary loaded by
>> >Hunspell, as it reports when invoked with the -D option.
>>
>> Thank you.  Feel free too Cc me on any patch for that, I'll gladly test
>> it out (and learn more about the underlying code).
>
>OK, thanks, will do.
>
>Btw, do you have a DICTIONARY environment variable defined?  If yes,
>what is its value?

Oh, I have, and it's (not surprisingly) "en_GB,cs_CZ".  I'm going to
find out where that comes from.

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Sat, 25 Feb 2017 13:15:23 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> >Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
> >automatically when we find more than one dictionary loaded by
> >Hunspell, as it reports when invoked with the -D option.
> 
> Thank you.  Feel free too Cc me on any patch for that, I'll gladly test
> it out (and learn more about the underlying code).

Please try the patch below.  (If it doesn't apply to your version of
ispell.el, tell me which version of Emacs you have, and I will try to
produce a compatible patch.)

TIA

--- lisp/textmodes/ispell.el~0	2017-01-08 06:23:52.000000000 +0200
+++ lisp/textmodes/ispell.el	2017-02-27 17:42:50.340586100 +0200
@@ -829,6 +829,15 @@
 				   (fboundp 'accept-process-output))
   "Non-nil means that the OS supports asynchronous processes.")
 
+(defvar ispell-current-dictionary nil
+  "The name of the current dictionary, or nil for the default.
+This is passed to the Ispell process using the `-d' switch and is
+used as key in `ispell-local-dictionary-alist' and `ispell-dictionary-alist'.")
+
+(defvar ispell-current-personal-dictionary nil
+  "The name of the current personal dictionary, or nil for the default.
+This is passed to the Ispell process using the `-p' switch.")
+
 ;; Make ispell.el work better with aspell.
 
 (defvar ispell-aspell-dictionary-alist nil
@@ -1087,6 +1096,7 @@
       (ispell-find-hunspell-dictionaries)
       (setq ispell-dictionary-alist ispell-hunspell-dictionary-alist))
     (or (assoc first-dict ispell-local-dictionary-alist)
+        (assoc first-dict ispell-hunspell-dictionary-alist)
         (assoc first-dict ispell-dictionary-alist)
         (error "Unknown dictionary: %s" first-dict)))
   (cl-pushnew (list dict '()) ispell-dictionary-alist :test #'equal)
@@ -1110,18 +1120,25 @@
 	  "[\n\r]+"
 	  t))
 	hunspell-default-dict
-	hunspell-default-dict-entry)
+	hunspell-default-dict-entry
+        hunspell-multi-dict)
     (dolist (dict hunspell-found-dicts)
       (let* ((full-name (file-name-nondirectory dict))
 	     (basename  (file-name-sans-extension full-name))
 	     (affix-file (concat dict ".aff")))
 	(if (string-match "\\.aff$" dict)
 	    ;; Found default dictionary
-	    (if hunspell-default-dict
-		(error "ispell-fhd: Default dict already defined as %s.  Not using %s.\n"
-		       hunspell-default-dict dict)
-	      (setq affix-file dict)
-	      (setq hunspell-default-dict (list basename affix-file)))
+            (progn
+              (if hunspell-default-dict
+                  (setq hunspell-multi-dict
+                        (concat (or hunspell-multi-dict
+                                    (car hunspell-default-dict))
+                                "," basename))
+                (setq affix-file dict)
+                (setq hunspell-default-dict (list basename affix-file)))
+              (ispell-print-if-debug
+               "++ ispell-fhd: default dict-entry:%s name:%s basename:%s\n"
+               dict full-name basename))
 	  (if (and (not (assoc basename ispell-hunspell-dict-paths-alist))
 		   (file-exists-p affix-file))
 	      ;; Entry has an associated .aff file and no previous value.
@@ -1171,7 +1188,12 @@
       (cl-pushnew (if (string= dict hunspell-default-dict)
                       hunspell-default-dict-entry
                     (list dict))
-                  ispell-hunspell-dictionary-alist :test #'equal))))
+                  ispell-hunspell-dictionary-alist :test #'equal))
+    ;; If we found more than one loaded dictionary, set up for using
+    ;; multiple dictionaries.
+    (when hunspell-multi-dict
+      (ispell-hunspell-add-multi-dic hunspell-multi-dict)
+      (setq ispell-current-dictionary hunspell-multi-dict))))
 
 ;; Set params according to the selected spellchecker
 
@@ -1416,15 +1438,6 @@
 
 ;;; **********************************************************************
 
-(defvar ispell-current-dictionary nil
-  "The name of the current dictionary, or nil for the default.
-This is passed to the Ispell process using the `-d' switch and is
-used as key in `ispell-local-dictionary-alist' and `ispell-dictionary-alist'.")
-
-(defvar ispell-current-personal-dictionary nil
-  "The name of the current personal dictionary, or nil for the default.
-This is passed to the Ispell process using the `-p' switch.")
-
 (defun ispell-decode-string (str)
   "Decodes multibyte character strings."
   (decode-coding-string str (ispell-get-coding-system)))

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Martin Kletzander]

On Mon, Feb 27, 2017 at 05:54:44PM +0200, Eli Zaretskii wrote:
>> Date: Sat, 25 Feb 2017 13:15:23 +0100
>> From: Martin Kletzander <mk@linux.com>
>> Cc: 25830@debbugs.gnu.org
>>
>> >Thanks.  I'll look into invoking ispell-hunspell-add-multi-dic
>> >automatically when we find more than one dictionary loaded by
>> >Hunspell, as it reports when invoked with the -D option.
>>
>> Thank you.  Feel free too Cc me on any patch for that, I'll gladly test
>> it out (and learn more about the underlying code).
>
>Please try the patch below.  (If it doesn't apply to your version of
>ispell.el, tell me which version of Emacs you have, and I will try to
>produce a compatible patch.)
>

Sorry I didn't get to this earlier.  One of the hunks didn't apply
correctly, so I used the file from master and it applied correctly.  So
with your patch applied on top of ispell.el from current master, the
situation is way better.  But I feel like it still doesn't work as
expected.  Not that anything would be broken, but it uses only the first
dictionary found.  I have to manually do:

  (ispell-hunspell-add-multi-dic "en_GB,cs_CZ")
  (ispell-change-dictionary "en_GB,cs_CZ")

I confirmed that both of them needed to be ran.  However after that
everything works perfectly.  So the only reason why I said it doesn't
work properly is that I see you running similar code in the patch:

   (when hunspell-multi-dict
     (ispell-hunspell-add-multi-dic hunspell-multi-dict)
     (setq ispell-current-dictionary hunspell-multi-dict))

When it works, I see hunspell is ran with '-d en_GB,cs_CZ', but it would
also work if hunspell was ran without '-d', but my guess is that you
don't want to do that.

Anyway, to find out a bit more, I did some debugging.  I'm not sure if
it helps or if it makes it more confusing, though.  Calling the function
returns "en_GB,cs_CZ".  When I add some prin1 statements around the
'add-multi-dic' and 'setq' I see it is called before the hunspell is ran
and it has the proper values set.  However when I check the values from
the scratch buffer after it was ran (either manually or by
spell-checking a word), it's like it didn't do anything.  It behaves
like it did all the right things, but in a different scope, even though
the variables are not buffer-local or anything.

Unsure how to continue or debug this more, I am giving up for today.
Let me know if I should debug it a bit more somehow.

Have a nice day,
Martin

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Tue, 28 Feb 2017 22:53:52 +0100
> From: Martin Kletzander <mk@linux.com>
> Cc: 25830@debbugs.gnu.org
> 
> >Please try the patch below.  (If it doesn't apply to your version of
> >ispell.el, tell me which version of Emacs you have, and I will try to
> >produce a compatible patch.)
> >
> 
> Sorry I didn't get to this earlier.

Now it's my turn to apologize for the delay...

> One of the hunks didn't apply correctly, so I used the file from
> master and it applied correctly.  So with your patch applied on top
> of ispell.el from current master, the situation is way better.  But
> I feel like it still doesn't work as expected.  Not that anything
> would be broken, but it uses only the first dictionary found.

You are right, that was the wrong direction to try to solve the
problem.

Here's another attempt.  It should be applied _instead_ of the
previous patch, to the original version of ispell.el.  Let me know if
it works OK for you.

Thanks.

--- lisp/textmodes/ispell.el~0	2017-01-08 06:23:52.000000000 +0200
+++ lisp/textmodes/ispell.el	2017-03-05 17:21:10.441710400 +0200
@@ -1094,10 +1094,10 @@
 
 (defun ispell-find-hunspell-dictionaries ()
   "Look for installed Hunspell dictionaries.
-Will initialize `ispell-hunspell-dictionary-alist' and
-`ispell-hunspell-dictionary-alist' after values found
-and remove `ispell-dicts-name2locale-equivs-alist'
-entries if a specific dictionary was found."
+Will initialize `ispell-hunspell-dictionary-alist' according
+to dictionaries found, and will remove aliases from the list
+in `ispell-dicts-name2locale-equivs-alist' if an explicit
+dictionary from that list was found."
   (let ((hunspell-found-dicts
 	 (split-string
 	  (with-temp-buffer
@@ -1110,18 +1110,27 @@
 	  "[\n\r]+"
 	  t))
 	hunspell-default-dict
-	hunspell-default-dict-entry)
+	hunspell-default-dict-entry
+	hunspell-multi-dict)
     (dolist (dict hunspell-found-dicts)
       (let* ((full-name (file-name-nondirectory dict))
 	     (basename  (file-name-sans-extension full-name))
 	     (affix-file (concat dict ".aff")))
 	(if (string-match "\\.aff$" dict)
 	    ;; Found default dictionary
-	    (if hunspell-default-dict
-		(error "ispell-fhd: Default dict already defined as %s.  Not using %s.\n"
-		       hunspell-default-dict dict)
-	      (setq affix-file dict)
-	      (setq hunspell-default-dict (list basename affix-file)))
+	    (progn
+	      (if hunspell-default-dict
+		  (setq hunspell-multi-dict
+			(concat (or hunspell-multi-dict
+				    (car hunspell-default-dict))
+				"," basename))
+		(setq affix-file dict)
+		;; FIXME: The cdr of the list we cons below is never
+		;; used.  Why do we need a list?
+		(setq hunspell-default-dict (list basename affix-file)))
+	      (ispell-print-if-debug
+	       "++ ispell-fhd: default dict-entry:%s name:%s basename:%s\n"
+	       dict full-name basename))
 	  (if (and (not (assoc basename ispell-hunspell-dict-paths-alist))
 		   (file-exists-p affix-file))
 	      ;; Entry has an associated .aff file and no previous value.
@@ -1161,7 +1170,8 @@
 	      (cl-pushnew (list dict-equiv-key affix-file)
                           ispell-hunspell-dict-paths-alist :test #'equal)))))
     ;; Parse and set values for default dictionary.
-    (setq hunspell-default-dict (car hunspell-default-dict))
+    (setq hunspell-default-dict (or hunspell-multi-dict
+				    (car hunspell-default-dict)))
     (setq hunspell-default-dict-entry
 	  (ispell-parse-hunspell-affix-file hunspell-default-dict))
     ;; Create an alist of found dicts with only names, except for default dict.

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

Ping!  (The previous message to you bounced.)

> Date: Sun, 05 Mar 2017 17:42:14 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 25830@debbugs.gnu.org
> 
> > Date: Tue, 28 Feb 2017 22:53:52 +0100
> > From: Martin Kletzander <mk@linux.com>
> > Cc: 25830@debbugs.gnu.org
> > 
> > >Please try the patch below.  (If it doesn't apply to your version of
> > >ispell.el, tell me which version of Emacs you have, and I will try to
> > >produce a compatible patch.)
> > >
> > 
> > Sorry I didn't get to this earlier.
> 
> Now it's my turn to apologize for the delay...
> 
> > One of the hunks didn't apply correctly, so I used the file from
> > master and it applied correctly.  So with your patch applied on top
> > of ispell.el from current master, the situation is way better.  But
> > I feel like it still doesn't work as expected.  Not that anything
> > would be broken, but it uses only the first dictionary found.
> 
> You are right, that was the wrong direction to try to solve the
> problem.
> 
> Here's another attempt.  It should be applied _instead_ of the
> previous patch, to the original version of ispell.el.  Let me know if
> it works OK for you.
> 
> Thanks.
> 
> --- lisp/textmodes/ispell.el~0	2017-01-08 06:23:52.000000000 +0200
> +++ lisp/textmodes/ispell.el	2017-03-05 17:21:10.441710400 +0200
> @@ -1094,10 +1094,10 @@
>  
>  (defun ispell-find-hunspell-dictionaries ()
>    "Look for installed Hunspell dictionaries.
> -Will initialize `ispell-hunspell-dictionary-alist' and
> -`ispell-hunspell-dictionary-alist' after values found
> -and remove `ispell-dicts-name2locale-equivs-alist'
> -entries if a specific dictionary was found."
> +Will initialize `ispell-hunspell-dictionary-alist' according
> +to dictionaries found, and will remove aliases from the list
> +in `ispell-dicts-name2locale-equivs-alist' if an explicit
> +dictionary from that list was found."
>    (let ((hunspell-found-dicts
>  	 (split-string
>  	  (with-temp-buffer
> @@ -1110,18 +1110,27 @@
>  	  "[\n\r]+"
>  	  t))
>  	hunspell-default-dict
> -	hunspell-default-dict-entry)
> +	hunspell-default-dict-entry
> +	hunspell-multi-dict)
>      (dolist (dict hunspell-found-dicts)
>        (let* ((full-name (file-name-nondirectory dict))
>  	     (basename  (file-name-sans-extension full-name))
>  	     (affix-file (concat dict ".aff")))
>  	(if (string-match "\\.aff$" dict)
>  	    ;; Found default dictionary
> -	    (if hunspell-default-dict
> -		(error "ispell-fhd: Default dict already defined as %s.  Not using %s.\n"
> -		       hunspell-default-dict dict)
> -	      (setq affix-file dict)
> -	      (setq hunspell-default-dict (list basename affix-file)))
> +	    (progn
> +	      (if hunspell-default-dict
> +		  (setq hunspell-multi-dict
> +			(concat (or hunspell-multi-dict
> +				    (car hunspell-default-dict))
> +				"," basename))
> +		(setq affix-file dict)
> +		;; FIXME: The cdr of the list we cons below is never
> +		;; used.  Why do we need a list?
> +		(setq hunspell-default-dict (list basename affix-file)))
> +	      (ispell-print-if-debug
> +	       "++ ispell-fhd: default dict-entry:%s name:%s basename:%s\n"
> +	       dict full-name basename))
>  	  (if (and (not (assoc basename ispell-hunspell-dict-paths-alist))
>  		   (file-exists-p affix-file))
>  	      ;; Entry has an associated .aff file and no previous value.
> @@ -1161,7 +1170,8 @@
>  	      (cl-pushnew (list dict-equiv-key affix-file)
>                            ispell-hunspell-dict-paths-alist :test #'equal)))))
>      ;; Parse and set values for default dictionary.
> -    (setq hunspell-default-dict (car hunspell-default-dict))
> +    (setq hunspell-default-dict (or hunspell-multi-dict
> +				    (car hunspell-default-dict)))
>      (setq hunspell-default-dict-entry
>  	  (ispell-parse-hunspell-affix-file hunspell-default-dict))
>      ;; Create an alist of found dicts with only names, except for default dict.
> 
> 
> 
> 

bug#25830: 25.2; ispell doesn't parse hunspell dictionaries properly

[Eli Zaretskii]

> Date: Tue, 07 Mar 2017 18:51:54 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 25830@debbugs.gnu.org
> 
> Ping!  (The previous message to you bounced.)

No further comments, so I pushed the proposed changes, and I'm marking
this bug done.

Thanks for the feedback.