bug & idea how to fix: ispell.el fails checking English in a buffer with Russian+English

bug & idea how to fix: ispell.el fails checking English in a buffer with Russian+English

[Ivan Zakharyaschev]

	Hello!

Recently, there has been a report on ispell.el not working perfectly in
GNU Emacs 21.2:

The report:

Consider a buffer with mixed languages (Russian and English). We try to
check spelling of both languages. First, we set the distionary to
Russian, and ispell works: checks Russian words and ignores English. But
the other way round it deosn't: quite soon ispell.el reports
"misalignment" and doesn't go on spelling.

The minimal test case is like this: in a clear plain text buffer, write

кошка cat

(The first word the Russian for "cat".) Then switch to "enghlish"
dictionary and M-x ispell-buffer. We get this message in the minibuffer:

Ispell misalignment: word 'LZ' point 3; probably incompatible versions

The reason & possible solution:

The first thing that can be noticed that changing the value of
`coding-system' field for the English entry in `ispell-dictionary-alist'
from `iso-8859-1' to `iso-safe' solves the problem. For example, after
the substitution:

("british"                           ; British version
    "[A-Za-z]" "[^A-Za-z]" "[']" nil ("-B" "-d" "british") nil
iso-safe)

instead of

("british"                           ; British version
    "[A-Za-z]" "[^A-Za-z]" "[']" nil ("-B" "-d" "british") nil
iso-8859-1)

the test case example is checked successfully with dictionary "british".


I see the reason for this problem in ispell.el, more precisely in
`ispell-get-line' function (or at the points where it is called:
particularly, `ispell-region'): it doesn't honour the information about
the alphabet of the language/dictionary (specified in the corrsponding
`ispell-dictionary-alist''s entry), but sends the whole line "as is"
without extracting only those characters that are valid for the
specified language/dictionary. And so the line gets sent to ispell
process with the internal multibyte representation of the Russian
characters. (Now we see why setting the `coding-system' to `iso-safe'
helps: then the Russian characters get encoded with harmless question
marks that get sent to the ispell process.)

So, the reason i sin the work of the pair of functions
`ispell-get-line'/`ispell-region'.

There is a similar pair of function is ispell.el:
`ispell-get-word'/`ispell-word'. It works much better: it extracts words
consisting only of the characters from the specified distionary's
alphabet.

Another similar el-library -- flyspell.el -- also works better:
`flyspell-word' extracts "good" words, and `flyspell-region' iterates
over the region with succeeding calls to `flyspell-word': so only "good"
words get sent to the ispell process.

This could be a variant of the fix for ispell.el: `ispell-region' calls
`ispell-word' or `ispell-get-word'.


BTW, it is almost a year ispell.el 3.5 is out, but GNU Emacs still has
3.4 inside.

References: if ypu can read Russian, here are the original messages
discussing this problem:

http://www.altlinux.ru/pipermail/sisyphus/2002-September/030603.html

http://www.altlinux.ru/pipermail/sisyphus/2002-September/030610.html

Regards,

-- 
Ivan Zakharyaschev
ALT Linux Team member, Sisyphus developer
http://www.altlinux.ru, http://www.altlinux.com