bug#25766: aspell problem in LaTeX mode

bug#25766: aspell problem in LaTeX mode

[Bob Alice]

[-- Attachment #1: Type: text/plain, Size: 4194 bytes --]

The following is a minimal example of LaTeX code that breaks the spell
checker:

\caption{5\% \label{_fred}}
\autoref{alice} envolope

When I spell-check this it fails to notice that 'envolope' is mis-spelled.
If I remove the underscore from _fred or remove the \% then the spell
checker works fine.

Also, If I change the emacs default to use ispell instead of aspell then it
works fine. Running the file through aspell on the command line also finds
the spelling error. I can only conclude that there's a bug in the bit of
emacs that runs LaTeX code through aspell.

Any ideas?

I'm using emacs version 24.4.1 and aspell version 3.1.20 (but really
0.60.7-20110707); The default packages are from the Debian Jessie repo.

P.S I know the LaTeX code isn't complete, but my original comipled just
fine and aspell still wouldn't find the spelling error.
Configured using:
 `configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.4/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.4/site-lisp:/usr/share/emacs/site-lisp
 --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.4/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.4/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat
 -Werror=format-security -Wall' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-z,relro'

Important settings:
  value of $LANG: en_GB.utf8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: LaTeX

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/24.4/site-lisp/debian-startup hides
/usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs24/site-lisp/dictionaries-common/ispell hides
/usr/share/emacs/24.4/lisp/textmodes/ispell
/usr/share/emacs24/site-lisp/dictionaries-common/flyspell hides
/usr/share/emacs/24.4/lisp/textmodes/flyspell
/usr/share/emacs/site-lisp/rst hides
/usr/share/emacs/24.4/lisp/textmodes/rst

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
help-fns mail-prsvr mail-utils help-mode easymenu ispell tex-mode
compile shell pcomplete comint ansi-color ring latexenc cc-styles
cc-align cc-engine cc-vars cc-defs time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 95454 3258)
 (symbols 48 20235 0)
 (miscs 40 59 219)
 (strings 32 16158 4364)
 (string-bytes 1 513962)
 (vectors 16 11281)
 (vector-slots 8 407759 3174)
 (floats 8 71 533)
 (intervals 56 360 16)
 (buffers 960 13)
 (heap 1024 43833 1092))

[-- Attachment #2: Type: text/html, Size: 4593 bytes --]

bug#25766: aspell problem in LaTeX mode

[Eli Zaretskii]

> From: Bob Alice <bobfredalice@googlemail.com>
> Date: Thu, 16 Feb 2017 20:52:17 +0000
> 
> The following is a minimal example of LaTeX code that breaks the spell checker:
> 
> \caption{5\% \label{_fred}}
> \autoref{alice} envolope
> 
> When I spell-check this it fails to notice that 'envolope' is mis-spelled. If I remove the underscore from _fred or
> remove the \% then the spell checker works fine.
> 
> Also, If I change the emacs default to use ispell instead of aspell then it works fine. Running the file through
> aspell on the command line also finds the spelling error. I can only conclude that there's a bug in the bit of
> emacs that runs LaTeX code through aspell. 

I don't have aspell installed; I tried hunspell, and it did detect the
"envolope" mis-spelling.

> Any ideas?

Turn on ispell.el debugging and see what Emacs submits to aspell.

> I'm using emacs version 24.4.1 and aspell version 3.1.20 (but really 0.60.7-20110707); The default packages
> are from the Debian Jessie repo.

I tried both the current RC of Emacs 25.2 and Emacs 24.4, they both
work with hunspell.

bug#25766: aspell problem in LaTeX mode

[Eli Zaretskii]

[Please keep the bug address on the CC list.]

> From: Bob Alice <bobfredalice@googlemail.com>
> Date: Fri, 17 Feb 2017 09:36:29 +0000
> 
> Thanks for the suggestions.
> I tried hunspell and that worked. The debugging output is identical for aspell and hunspell (apart from the
> name). Debugger doesn't show input to aspell/hunspell command and I can't figure out how to make it do so.

Can you tell how aspell is invoked by Emacs?  (One way of finding that
out is to use the Emacs "M-x proced" command.)  Then try invoking
aspell in the same way from the shell prompt, passing it the offending
line of text ("^\autoref{alice} envolope"), and see what aspell
returns for that.

bug#25766: aspell problem in LaTeX mode

[Eli Zaretskii]

[Please keep the bug address, 25766@debbugs.gnu.org, on the CC list.]

> From: Bob Alice <bobfredalice@googlemail.com>
> Date: Fri, 17 Feb 2017 10:53:44 +0000
> 
> It doesn't work. I tried:
> while((1));do ps -Alf|grep aspell >>results; done
> 
> but the only thing I get is:
> 0 S meuser 4839 4828 0 80 0 - 8093 - 10:42 ? 00:00:00 /usr/bin/aspell -a -m -d en --encoding=utf-8

This is all we need to know, so this did work.

> It doesn't show me what's being passed to aspell even for text where it finds spelling errors.

It isn't supposed to.

Now invoke aspell from your shell prompt exactly as shown above, i.e.

  /usr/bin/aspell -a -m -d en --encoding=utf-8

Aspell will then wait for input from standard input.  Type the string
you see in the ispell.el's debug buffer.  E.g., with hunspell, when I
invoke the command ispell-buffer-with-debug on your LaTeX file, I see
this in the ispell debug buffer:

  ispell-region: string pos (29->53), eol: 53, [in-comment]: [nil], [add-comment]: [nil], [string]: [^\autoref{alice} envolope
]

which tells me that the string it will pass to the speller is
"^\autoref{alice} envolope" (without the quotes).  So type this string
and hit [Enter].  Then see what aspell outputs in return.

Once again, please CC the bug address so that this discussion gets
recorded there.

Thanks.

bug#25766: aspell problem in LaTeX mode

[Bob Alice]

[-- Attachment #1: Type: text/plain, Size: 1960 bytes --]

Note: I removed \autoref{alice}. It wasn't necessary. New file is:

\caption{5\% \label{_fred}}
envolope

Debugger gives:
ispell-region: string pos (29->37), eol: 37, [in-comment]: [nil],
[add-comment]: [nil], [string]: [^envolope

Running:
/usr/bin/aspell -a -m -d en --encoding=utf-8

Inputting:
^envolope

Output:
& envolope 7 1: envelope, enveloper, envelop, enveloped, envelopes,
envelops, envelope's

This output looks sensible, but the spelling options do not appear in the
emacs buffer.



On 17 February 2017 at 13:29, Eli Zaretskii <eliz@gnu.org> wrote:

>
> [Please keep the bug address, 25766@debbugs.gnu.org, on the CC list.]
>
> > From: Bob Alice <bobfredalice@googlemail.com>
> > Date: Fri, 17 Feb 2017 10:53:44 +0000
> >
> > It doesn't work. I tried:
> > while((1));do ps -Alf|grep aspell >>results; done
> >
> > but the only thing I get is:
> > 0 S meuser 4839 4828 0 80 0 - 8093 - 10:42 ? 00:00:00 /usr/bin/aspell -a
> -m -d en --encoding=utf-8
>
> This is all we need to know, so this did work.
>
> > It doesn't show me what's being passed to aspell even for text where it
> finds spelling errors.
>
> It isn't supposed to.
>
> Now invoke aspell from your shell prompt exactly as shown above, i.e.
>
>   /usr/bin/aspell -a -m -d en --encoding=utf-8
>
> Aspell will then wait for input from standard input.  Type the string
> you see in the ispell.el's debug buffer.  E.g., with hunspell, when I
> invoke the command ispell-buffer-with-debug on your LaTeX file, I see
> this in the ispell debug buffer:
>
>   ispell-region: string pos (29->53), eol: 53, [in-comment]: [nil],
> [add-comment]: [nil], [string]: [^\autoref{alice} envolope
> ]
>
> which tells me that the string it will pass to the speller is
> "^\autoref{alice} envolope" (without the quotes).  So type this string
> and hit [Enter].  Then see what aspell outputs in return.
>
> Once again, please CC the bug address so that this discussion gets
> recorded there.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 2688 bytes --]

bug#25766: aspell problem in LaTeX mode

[Eli Zaretskii]

> From: Bob Alice <bobfredalice@googlemail.com>
> Date: Fri, 17 Feb 2017 16:21:11 +0000
> Cc: 25766@debbugs.gnu.org
> 
> Note: I removed \autoref{alice}. It wasn't necessary. New file is:
> 
> \caption{5\% \label{_fred}}
> envolope
> 
> Debugger gives:
> ispell-region: string pos (29->37), eol: 37, [in-comment]: [nil], [add-comment]: [nil], [string]: [^envolope
> 
> Running:
> /usr/bin/aspell -a -m -d en --encoding=utf-8
> 
> Inputting:
> ^envolope
> 
> Output:
> & envolope 7 1: envelope, enveloper, envelop, enveloped, envelopes, envelops, envelope's
> 
> This output looks sensible, but the spelling options do not appear in the emacs buffer.

Strange.  I guess the next step is to look at what Emacs receives from
aspell.  I think if you display the value of ispell-filter in
ispell-process-line, it will show that.  Here's the relevant part of
ispell-process-line:

    ;; send string to spell process and get input.
    (ispell-send-string string)
    (while (progn
	     (ispell-accept-output)
	     ;; Last item of output contains a blank line. <<<<<<<<<<<<
	     (not (string= "" (car ispell-filter)))))

Insert

    (message "%s" ispell-filter)

before the marked line, then load ispell.el manually, and repeat the
experiment.  The results should be in the *Messages* buffer.  Maybe if
we see what Emacs gets back from aspell, we will see the light.

Thanks.

bug#25766: aspell problem in LaTeX mode

[Bob Alice]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

ispell-filter is empty. Here's the partial output (I used
ispell-print-if-debug to get the messages in the same buffer):
.....
ispell-region: string pos (22->28), eol: 28, [in-comment]: [%],
[add-comment]: [%], [string]: [^%fred}}
]
DEBUG-ISPELL-FILTER: ()
ispell-region: string pos (28->28), eol: 37, [in-comment]: [nil],
[add-comment]: [nil], [string]: [nil]
ispell-region: string pos (29->37), eol: 37, [in-comment]: [nil],
[add-comment]: [nil], [string]: [^envolope
]
DEBUG-ISPELL-FILTER: ()

I notice that it is sending two queries (one of which is nil) before it
attempts to read the output. Perhaps this is the problem?

Somebody else sent me a mail saying this isn't a problem in the latest
emacs. I'm happy to continue trying to debug if you think it's useful
otherwise should I just upgrade and stop here?


On 17 February 2017 at 16:37, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Bob Alice <bobfredalice@googlemail.com>
> > Date: Fri, 17 Feb 2017 16:21:11 +0000
> > Cc: 25766@debbugs.gnu.org
> >
> > Note: I removed \autoref{alice}. It wasn't necessary. New file is:
> >
> > \caption{5\% \label{_fred}}
> > envolope
> >
> > Debugger gives:
> > ispell-region: string pos (29->37), eol: 37, [in-comment]: [nil],
> [add-comment]: [nil], [string]: [^envolope
> >
> > Running:
> > /usr/bin/aspell -a -m -d en --encoding=utf-8
> >
> > Inputting:
> > ^envolope
> >
> > Output:
> > & envolope 7 1: envelope, enveloper, envelop, enveloped, envelopes,
> envelops, envelope's
> >
> > This output looks sensible, but the spelling options do not appear in
> the emacs buffer.
>
> Strange.  I guess the next step is to look at what Emacs receives from
> aspell.  I think if you display the value of ispell-filter in
> ispell-process-line, it will show that.  Here's the relevant part of
> ispell-process-line:
>
>     ;; send string to spell process and get input.
>     (ispell-send-string string)
>     (while (progn
>              (ispell-accept-output)
>              ;; Last item of output contains a blank line. <<<<<<<<<<<<
>              (not (string= "" (car ispell-filter)))))
>
> Insert
>
>     (message "%s" ispell-filter)
>
> before the marked line, then load ispell.el manually, and repeat the
> experiment.  The results should be in the *Messages* buffer.  Maybe if
> we see what Emacs gets back from aspell, we will see the light.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 3164 bytes --]

bug#25766: aspell problem in LaTeX mode

[Eli Zaretskii]

> From: Bob Alice <bobfredalice@googlemail.com>
> Date: Sat, 18 Feb 2017 10:27:49 +0000
> Cc: 25766@debbugs.gnu.org
> 
> Somebody else sent me a mail saying this isn't a problem in the latest emacs. I'm happy to continue trying to
> debug if you think it's useful otherwise should I just upgrade and stop here?

If you upgrade and the problem goes away, we don't need to investigate
further.

Thanks.

bug#25766: aspell problem in LaTeX mode

[Bob Alice]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

Let's leave it there then. Thanks for all your help.
Will Debian take note of this and fix Jessie, or will it just be left until
the next major upgrade?


On 18 February 2017 at 11:52, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Bob Alice <bobfredalice@googlemail.com>
> > Date: Sat, 18 Feb 2017 10:27:49 +0000
> > Cc: 25766@debbugs.gnu.org
> >
> > Somebody else sent me a mail saying this isn't a problem in the latest
> emacs. I'm happy to continue trying to
> > debug if you think it's useful otherwise should I just upgrade and stop
> here?
>
> If you upgrade and the problem goes away, we don't need to investigate
> further.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 1121 bytes --]

bug#25766: aspell problem in LaTeX mode

[Glenn Morris]

Bob Alice wrote:

> \caption{5\% \label{_fred}}
> \autoref{alice} envolope
>
> When I spell-check this it fails to notice that 'envolope' is mis-spelled.
[...]
> I'm using emacs version 24.4.1 and aspell version 3.1.20 (but really
> 0.60.7-20110707); The default packages are from the Debian Jessie repo.

I see this on Debian testing with the same aspell and Emacs 24.5, but
not with 25.1. So I suggest you upgrade Emacs.

bug#25766: aspell problem in LaTeX mode

[Bob Alice]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

How did you do that? I added the backports repository to my list and ran:
apt-get -t jessie-backports install emacs

but it claimed it was already at the latest version.
Are you suggesting I just install it outside of Debian?

On 17 February 2017 at 16:46, Glenn Morris <rgm@gnu.org> wrote:

> Bob Alice wrote:
>
> > \caption{5\% \label{_fred}}
> > \autoref{alice} envolope
> >
> > When I spell-check this it fails to notice that 'envolope' is
> mis-spelled.
> [...]
> > I'm using emacs version 24.4.1 and aspell version 3.1.20 (but really
> > 0.60.7-20110707); The default packages are from the Debian Jessie repo.
>
> I see this on Debian testing with the same aspell and Emacs 24.5, but
> not with 25.1. So I suggest you upgrade Emacs.
>

[-- Attachment #2: Type: text/html, Size: 1129 bytes --]