bug#12768: 24.1; flyspell highlights words which ispell accepts

[Agustin Martin <agustin.martin@hispalinux.es>]

On Thu, Nov 01, 2012 at 04:44:01PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 1 Nov 2012 01:10:11 +0100
> > From: Agustin Martin <agustin.martin@hispalinux.es>
> > Cc: agustin.martin@upm.es
> > 
> > Really strange. With your steps I can even reproduce the problem with
> > something as simple as
> > 
> > dnl -- D
> > 
> > If I add something before dnl  I still get the problem.
> > 
> > sdfdsfsdnl -- D
> > 
> > IIf I either break the double dash (no double dashes at all) or the
> > dnl or remove the "D" after the dashes problem disappear
> > 
> > dfnl -- D
> > dnl - - D
> > dnl --
> > 
> > autotest-mode.el is a small file, but II  do not see any hint about
> > what is happening here. Seems to do something strange with double
> > dashes associated with dnl, but I am absolutely clueless, help
> > welcome.
> 
> If you step through ispell-region, you will see that it goes bananas
> because ispell-skip-region-alist has "--+" as part of it, which
> matches the dashes between "dnl" and the text of the comment.  What
> ispell-region does is remove the dashes, and then _prepend_ "dnl"
> (because it's a comment-start string).  The result is that we send
> something like "^dnl FOO" to the speller, and the character positions
> get screwed up, because the characters between "dnl" and "FOO" were
> removed.

I have been dealing with this a bit more and seems I now I understand
things better.

(ispell-region) splits regions to spell in lines, and inside lines in
portions separated by skip-regions, which are sent to ispell pipe. 
When inside a comment, the comment string is added as string prefix
before sending to the ispell pipe  (I guess to make ispell know that
the string is part of a comment).

However ispell understand this prefix in few modes and of course not in
autoconf mode, where `dnl' is spell-checked as any other word (is the only
mode I remember now where comment string is an alphabetic string). For
other modes the comment string is just ignored by ispell for anything than
knowing that string is inside a comment (but properly shifted), and things
work with proper shifts.

However, for autoconf mode `dnl -- D' dnl is checked first and later is
`D' what is sent is `dnl  D' (with a dnl prefix), where `dnl' is checked
again messing up everything because ispell checks dnl where `D' position
is tracked (and expected by ispell.el instead of `dnl').

I have committed a change (r110817) that simply does not prefix the
strings when comment-start when in autoconf-mode (ispell does not support
that prefix anyway).

ispell still does not skip `dnl' as comment header and is shown as a
misspelling, but for everything else seems to work. However, please test
extensively.

> Btw, WIBNI ispell.el had a debug option, which, if set, would collect
> in a special buffer everything we send to the speller and its
> responses.  I find myself adding '(message "sent %s" strin)'
> etc. every time I need to debug the communications between ispell.el
> and the speller program.  WDYT?

I have also added a proof of concept for that (r110816), mostly intended to
be enabled through dedicated wrapper functions that when called create the
debug buffer and enable debugging messages sent through a common function. 
This is still very preliminary, with only `ispell-buffer-with-debug' using
it, but provides some info and as problems get debugged can be extended. 
Suggestions are welcome, even about a complete re-design if really needed.

-- 
Agustin