Flyspell-region bug

Flyspell-region bug

[Slawomir Nowaczyk]

Hello,

I think there is a small problem in flyspell.el. If you evaluate the
following two expressions in this buffer:

(setq flyspell-large-region 1)
(flyspell-buffer)

Washington washington

the uncapitalised version is not marked as misspelled -- while it
should be. The following patch corrects the immediate problem:

**************************************************

*** c:/Emacs/lisp/textmodes/flyspell.el Tue Feb  7 17:21:31 2006
--- c:/temp/buffer-content-476I-k       Wed Feb 22 13:37:31 2006
***************
*** 1322,1328 ****
  The buffer to mark them in is `flyspell-large-region-buffer'."
    (let (words-not-found
        (ispell-otherchars (ispell-get-otherchars))
!       (buffer-scan-pos flyspell-large-region-beg))
      (with-current-buffer flyspell-external-ispell-buffer
        (goto-char (point-min))
        ;; Loop over incorrect words, in the order they were reported,
--- 1322,1329 ----
  The buffer to mark them in is `flyspell-large-region-buffer'."
    (let (words-not-found
        (ispell-otherchars (ispell-get-otherchars))
!       (buffer-scan-pos flyspell-large-region-beg)
!       (case-fold-search nil))
      (with-current-buffer flyspell-external-ispell-buffer
        (goto-char (point-min))
        ;; Loop over incorrect words, in the order they were reported,

**************************************************

Nevertheless, I find it somewhat disturbing that flyspell *did not*
complain but silently decided to ignore the misspelling ispell
reported...

Wouldn't it be better to at least show a message when this happens?

-- 
 Best wishes,
   Slawomir Nowaczyk
     ( slawomir.nowaczyk.847@student.lu.se )

UNIX is a clever beast:
  % got a light?
  No match.

Re: Flyspell-region bug

[Agustin Martin]

On Wed, Feb 22, 2006 at 01:53:14PM +0100, Slawomir Nowaczyk wrote:
> Hello,
> 
> I think there is a small problem in flyspell.el. If you evaluate the
> following two expressions in this buffer:
> 
> (setq flyspell-large-region 1)
> (flyspell-buffer)
> 
> Washington washington
> 
> the uncapitalised version is not marked as misspelled -- while it
> should be.

I first thought this was non reproducible here, but I had case-fold-search
set to nil in ~/.emacs. Can now reproduce it after commenting that entry.

> The following patch corrects the immediate problem:

Your patch seems to do the right thing, making sure case-fold-search is nil
for the search.

> Nevertheless, I find it somewhat disturbing that flyspell *did not*
> complain but silently decided to ignore the misspelling ispell
> reported...

Note that flyspell found it, but with the wrong case.

> Wouldn't it be better to at least show a message when this happens?

In this case no message would have been issued since misspelling was
(wrongly) found, so words-not-found should be finally empty.

-- 
Agustin

Re: Flyspell-region bug

[Slawomir Nowaczyk]

On Fri, 24 Feb 2006 12:44:41 +0100
Agustin Martin <agustin.martin@hispalinux.es> wrote:

#> > Nevertheless, I find it somewhat disturbing that flyspell *did not*
#> > complain but silently decided to ignore the misspelling ispell
#> > reported...
#> 
#> Note that flyspell found it, but with the wrong case.

Well, it didn't really find it... it found the word but decided it is
*not* a misspelling, despite the fact that ispell said it is.

I do not know if this is supposed to be a good thing in any normal
situation. I am far from flyspell expert, so there may be some cases
where it makes sense that I don't know of, but it seems to me that
something is not right if ispell reports a word and flyspell-word
doesn't mark it as a misspelling.

#> > Wouldn't it be better to at least show a message when this happens?
#> 
#> In this case no message would have been issued since misspelling was
#> (wrongly) found, so words-not-found should be finally empty.

Well, I think I would like a message in this case as well: namely when
ispell reports a misspelling, the word is found, but flyspell-word
(the one in line 1387 of flyspell.el) doesn't mark it.

*To me*, it seems to indicate a likely problem. In what situation
would something like this not be a bug?

-- 
 Best wishes,
   Slawomir Nowaczyk
     ( slawomir.nowaczyk.847@student.lu.se )

Vuja De - the feeling you'll be here again later.

Re: Flyspell-region bug

[Agustin Martin]

On Mon, Feb 27, 2006 at 11:50:13AM +0100, Slawomir Nowaczyk wrote:
> On Fri, 24 Feb 2006 12:44:41 +0100
> Agustin Martin <agustin.martin@hispalinux.es> wrote:
> 
> #> > Nevertheless, I find it somewhat disturbing that flyspell *did not*
> #> > complain but silently decided to ignore the misspelling ispell
> #> > reported...
> #> 
> #> Note that flyspell found it, but with the wrong case.
> 
> Well, it didn't really find it... it found the word but decided it is
> *not* a misspelling, despite the fact that ispell said it is.

Hi, Slawomir

The process is as follows, from

Washington washington

ispell listed "washington" as misspelling, flyspell.el then looked for
"washington", but since search was case-insensitive, found "Washington"
first, which is actually not a misspelling, and validated it, so flyspell
thought "washington" is correct and tried next misspelling (Was really
validated by size, but validating by flyspell-word success would have
also succeeded). An alternative would be validating by string=, but if
search is forced to be case-sensitive I think is not needed.

> I am far from flyspell expert, 

Same for me,

> #> > Wouldn't it be better to at least show a message when this happens?
> #> 
> #> In this case no message would have been issued since misspelling was
> #> (wrongly) found, so words-not-found should be finally empty.
> 
> Well, I think I would like a message in this case as well: namely when
> ispell reports a misspelling, the word is found, but flyspell-word
> (the one in line 1387 of flyspell.el) doesn't mark it.
> 
> *To me*, it seems to indicate a likely problem. In what situation
> would something like this not be a bug?

There are some places where this can happen,

flyspell-large-region passes buffer to ispell as plain text, because
misspellings search in that buffer will be done with normal emacs search
functions as plain text. That makes most sgml or LaTeX tags be marked as
misspellings by ispell, but ignored by flyspell that knows that the buffer
is TeX/sgml. {ispell,flyspell}.el sgml deformatter is better than ispell
one, so telling ispell that file is sgml is not enough. Same for TeX, with
the added problem that things will be broken for 8bit chars (ispell would
expect things like \'a here). I think I mailed the list about this last,
but cannot find now the reference.

Another example where that can happen is with session localwords,
which are not passed to ispell in (flyspell-large-region) function, but
are recognized as correct by (flyspell-word). I have thinking about handling
them through flyspell-process-localwords, but I still have some things
unclear.

This kind of things can also appear when there is a boundary-chars mismatch
between what ispell.el considers valid and what ispell considers valid,
sometimes in obscure and confusing ways from ~/.emacs user entries.
Depending on where are more chars accepted, words-not-found will not be
nil of flyspell-word will be tried on a correct word.

I think the best for now is fix (flyspell-external-point-words) as you
suggested, making sure search is case-sensitive, as well as
(flyspell-process-localwords) which suffers from the same problem
(patch already sent). Messages might otherwise become too noisy for no good
reason.

Note that all of flyspell-large-region is a *big* hack to make spellchecking
of large regions minimally efficient, and this has some drawbacks (of
course, if we can fix them, better)

-- 
Agustin