bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

Consider the following 3 lines:

  conférence
  confÃ\251rence
  conférence

all 3 have been inserted typing exactly the same characters in a
terminal emacsclient (i.e. emacsclient -t)

- the first on, which shows the expected behavior, has been obtained
  with flyspell-mode turned off

- the second one, with garbled characters near the attempt to insert the
  character "é" (letter "e" with an acute accent on top), has been
  obtained with flyspell-mode turned on

- the third one has been obtained with flyspell-mode turned on, but by
  typing "confe" (without accent), then hitting backspace, then
  restarting with "é" (letter "e" with accent). I.e. it seems that
  accented letters at the beginning of words / after spaces are fine,
  whereas they garble the input when they're in the middle or at the end
  of words

The problem does not manifest itself in GUI clients (i.e. emacsclient
-c) or in stand-alone Emacs instances that do not use Emacs server.


Cheers.


PS whereas I do use viper mode, and I'm using it to report this bug, I
   can reproduce the bug using "emacs -Q", then activating server-mode
   in it, and connecting terminal-based emacsclients.


In GNU Emacs 24.3.50.1 (x86_64-pc-linux-gnu, GTK+ Version 3.8.2)
 of 2013-07-01 on dex, modified by Debian
 (emacs-snapshot package, version 2:20130701-1)
Windowing system distributor `The X.Org Foundation', version 11.0.11204000
System Description:	Debian GNU/Linux unstable (sid)

Configured using:
 `configure --build x86_64-linux-gnu --host x86_64-linux-gnu
 --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var --infodir=/usr/share/info --mandir=/usr/share/man
 --with-pop=yes
 --enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/24.3.50/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3.50/site-lisp:/usr/share/emacs/site-lisp
 --without-compress-info --with-crt-dir=/usr/lib/x86_64-linux-gnu/
 --with-x=yes --with-x-toolkit=gtk3 --with-imagemagick=yes
 CFLAGS='-DDEBIAN -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2'
 CPPFLAGS='-D_FORTIFY_SOURCE=2' LDFLAGS='-g -Wl,--as-needed
 -znocombreloc''

Important settings:
  value of $LC_MESSAGES: en_US.UTF-8
  value of $LC_MONETARY: it_IT.utf8
  value of $LC_NUMERIC: it_IT.utf8
  value of $LC_TIME: it_IT.utf8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  flyspell-mode: t
  winner-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
RET C h e e r s . ESC ESC < C-x C-s j j j ESC [ 6 ~ 
ESC [ 6 ~ ESC [ 5 ~ ESC < j j j C-x C-s A ESC x C-g 
C-g C-x C-s C-x C-s C-x # ESC [ > 1 ; 3 4 0 6 ; 0 c 
i p e r c h é RET ESC x f l y s p TAB C-g DEL DEL DEL 
DEL DEL DEL DEL DEL c o n f é r e n c e SPC DEL RET 
ESC x f l s p DEL DEL u s DEL DEL y s p TAB m o TAB 
RET C-c d f r RET c o n f à e n c e ESC 0 o c 
o n f à e n c e DEL DEL DEL DEL DEL DEL DEL DEL 
f SPC DEL é r e n c e ESC j j i c o n f ` DEL é DEL 
e DEL é r e n c e ESC d d C-x C-c j d d . . i c o n 
f é r e n c e <return> M-x C-g <backspace> a SPC <backspace> 
<backspace> <return> c o n f é r e n c e <escape> u 
j j j j j l l C-l M-x r e p o r t <tab> b <tab> <return> 
C-g M-x b u <tab> g <tab> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> r e p o r t - e m a <tab> <return> f l 
y s p e l l - m o d e SPC i n SPC <switch-frame> C-g 
C-x C-c C-g M-x <up> <return>

Recent messages:
Undo!
line-move-1: End of buffer [5 times]
call-interactively: End of buffer [2 times]
Making completion list...
Quit
Making completion list...
Back to top level.
When done with this frame, type C-x 5 0
Quit
(No files need saving)
Quit

Load-path shadows:
/usr/share/emacs/24.3.50/site-lisp/cmake-data/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs/24.3.50/site-lisp/puppet-el/puppet-mode hides /usr/share/emacs/site-lisp/puppet-mode
/home/zack/.emacs.d/elpa/tabulated-list-20120406.2251/tabulated-list hides /usr/share/emacs/24.3.50/lisp/emacs-lisp/tabulated-list
/usr/share/emacs-snapshot/site-lisp/auctex/tex-info hides /usr/share/emacs/site-lisp/auctex/tex-info
/usr/share/emacs-snapshot/site-lisp/auctex/tex-mik hides /usr/share/emacs/site-lisp/auctex/tex-mik
/usr/share/emacs-snapshot/site-lisp/auctex/context-nl hides /usr/share/emacs/site-lisp/auctex/context-nl
/usr/share/emacs-snapshot/site-lisp/auctex/font-latex hides /usr/share/emacs/site-lisp/auctex/font-latex
/usr/share/emacs-snapshot/site-lisp/auctex/tex hides /usr/share/emacs/site-lisp/auctex/tex
/usr/share/emacs-snapshot/site-lisp/auctex/context hides /usr/share/emacs/site-lisp/auctex/context
/usr/share/emacs-snapshot/site-lisp/auctex/tex-font hides /usr/share/emacs/site-lisp/auctex/tex-font
/usr/share/emacs-snapshot/site-lisp/auctex/plain-tex hides /usr/share/emacs/site-lisp/auctex/plain-tex
/usr/share/emacs-snapshot/site-lisp/auctex/multi-prompt hides /usr/share/emacs/site-lisp/auctex/multi-prompt
/usr/share/emacs-snapshot/site-lisp/auctex/latex hides /usr/share/emacs/site-lisp/auctex/latex
/usr/share/emacs-snapshot/site-lisp/auctex/tex-style hides /usr/share/emacs/site-lisp/auctex/tex-style
/usr/share/emacs-snapshot/site-lisp/auctex/preview hides /usr/share/emacs/site-lisp/auctex/preview
/usr/share/emacs-snapshot/site-lisp/auctex/prv-emacs hides /usr/share/emacs/site-lisp/auctex/prv-emacs
/usr/share/emacs-snapshot/site-lisp/auctex/tex-buf hides /usr/share/emacs/site-lisp/auctex/tex-buf
/usr/share/emacs-snapshot/site-lisp/auctex/tex-jp hides /usr/share/emacs/site-lisp/auctex/tex-jp
/usr/share/emacs-snapshot/site-lisp/auctex/tex-bar hides /usr/share/emacs/site-lisp/auctex/tex-bar
/usr/share/emacs-snapshot/site-lisp/auctex/context-en hides /usr/share/emacs/site-lisp/auctex/context-en
/usr/share/emacs-snapshot/site-lisp/auctex/tex-fold hides /usr/share/emacs/site-lisp/auctex/tex-fold
/usr/share/emacs-snapshot/site-lisp/auctex/toolbar-x hides /usr/share/emacs/site-lisp/auctex/toolbar-x
/usr/share/emacs-snapshot/site-lisp/auctex/bib-cite hides /usr/share/emacs/site-lisp/auctex/bib-cite
/usr/share/emacs-snapshot/site-lisp/auctex/texmathp hides /usr/share/emacs/site-lisp/auctex/texmathp

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
todoo markdown-mode cl lbdb sh-script smie executable conf-mode
help-mode imenu debian-bug debian-copyright debian-changelog-mode
add-log debian-control-mode python rx tramp tramp-compat auth-source
eieio eieio-core password-cache tramp-loaddefs trampver shell hippie-exp
thingatpt pp icalendar diary-lib diary-loaddefs org-wl org-w3m org-vm
org-rmail org-mhe org-mew org-irc org-jsinfo org-infojs org-html
org-info org-gnus gnus-util org-docview org-bibtex bibtex org-bbdb
org-icalendar org-exp ob-exp org-exp-blocks org-agenda vc-git misearch
multi-isearch tabify rect flyspell ispell sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils xterm server
edit-server-autoloads evil-autoloads evil-numbers-autoloads
moinmoin-mode-autoloads rainbow-mode-autoloads screen-lines-autoloads
tabulated-list-autoloads undo-tree-autoloads package org-crypt
org-protocol org byte-opt warnings bytecomp byte-compile cconv ob-tangle
ob-ref ob-lob ob-table org-footnote org-src ob-comint ob-keys
org-pcomplete pcomplete comint ansi-color org-list org-faces
org-entities noutline outline org-version ob-emacs-lisp ob org-compat
org-macs ob-eval org-loaddefs format-spec find-func cal-menu easymenu
calendar cal-loaddefs windmove winner ido uniquify midnight time-date
paren cus-start cus-load viper edmacro kmacro nadvice viper-ex
viper-macs viper-mous viper-cmd viper-keym ring viper-util viper-init
advice help-fns debian-el debian-el-loaddefs scala-mode-auto
emacs-goodies-el emacs-goodies-custom emacs-goodies-loaddefs easy-mmode
dpkg-dev-el dpkg-dev-el-loaddefs devhelp preview-latex tex-site
auto-loads cl-macs gv cl-loaddefs cl-lib tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Eli Zaretskii]

> From: Stefano Zacchiroli <zack@upsilon.cc>
> Date: Thu, 04 Jul 2013 00:19:29 +0200
> 
> 
> Consider the following 3 lines:
> 
>   conférence
>   confÃ\251rence
>   conférence
> 
> all 3 have been inserted typing exactly the same characters in a
> terminal emacsclient (i.e. emacsclient -t)
> 
> - the first on, which shows the expected behavior, has been obtained
>   with flyspell-mode turned off
> 
> - the second one, with garbled characters near the attempt to insert the
>   character "é" (letter "e" with an acute accent on top), has been
>   obtained with flyspell-mode turned on
> 
> - the third one has been obtained with flyspell-mode turned on, but by
>   typing "confe" (without accent), then hitting backspace, then
>   restarting with "é" (letter "e" with accent). I.e. it seems that
>   accented letters at the beginning of words / after spaces are fine,
>   whereas they garble the input when they're in the middle or at the end
>   of words
> 
> The problem does not manifest itself in GUI clients (i.e. emacsclient
> -c) or in stand-alone Emacs instances that do not use Emacs server.

It sounds like your encoding, which Emacs uses to communicate with the
speller, is set up incorrectly.

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Thu, Jul 04, 2013 at 05:44:45AM +0300, Eli Zaretskii wrote:
> It sounds like your encoding, which Emacs uses to communicate with the
> speller, is set up incorrectly.

Uh? And why would that be affected by whether flyspell-mode is enabled
or not? Or by whether the accented letter is typed as part of a word
rather than at the beginning of it?

Everything of course is possible, but I don't get the rationale for your
explanation. Can you expand on it a little bit?

Also, I should note that it works fine with Emacs 23 (without changing
my encoding settings), whereas it breaks with 24.3.

Cheers.
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Agustin Martin]

On Thu, Jul 04, 2013 at 09:02:21AM +0200, Stefano Zacchiroli wrote:
> On Thu, Jul 04, 2013 at 05:44:45AM +0300, Eli Zaretskii wrote:
> > It sounds like your encoding, which Emacs uses to communicate with the
> > speller, is set up incorrectly.
> 
> Uh? And why would that be affected by whether flyspell-mode is enabled
> or not? Or by whether the accented letter is typed as part of a word
> rather than at the beginning of it?
> 
> Everything of course is possible, but I don't get the rationale for your
> explanation. Can you expand on it a little bit?
> 
> Also, I should note that it works fine with Emacs 23 (without changing
> my encoding settings), whereas it breaks with 24.3.

For the records, in Debian sid ispell.el and flyspell.el are exactly the
same for emacs23 and emacs24 packages (are provided externally by
dictionaries-common package), although I see that you reproduced this
also with "emacs -Q", which should use pristine values.

I could not reproduce this in my ISO8859-1 box. However, if I call Emacs
under the es_ES.UTF8 locale,

$ LC_ALL=es_ES.UTF8 emacs23
M-x server-start
$ LC_ALL=es_ES.UTF8 emacsclient.emacs23 -t

everytime I type an accented char in the GUI I see its UTF8 correspondence
in the client terminal, not the char. I am seeing this for Emacs23,
Emacs24 and emacs-snapshot, with or without "-Q", no need to fiddle with
`flyspell-mode'. This seems to not happen with GUI clients.

Regards,

-- 
Agustin

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Andreas Schwab]

Agustin Martin <agustin.martin@hispalinux.es> writes:

> everytime I type an accented char in the GUI I see its UTF8 correspondence
> in the client terminal, not the char.

So you didn't switch your terminal into UTF-8 mode.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Eli Zaretskii]

> Date: Thu, 4 Jul 2013 09:02:21 +0200
> From: Stefano Zacchiroli <zack@upsilon.cc>
> Cc: 14782@debbugs.gnu.org
> 
> On Thu, Jul 04, 2013 at 05:44:45AM +0300, Eli Zaretskii wrote:
> > It sounds like your encoding, which Emacs uses to communicate with the
> > speller, is set up incorrectly.
> 
> Uh? And why would that be affected by whether flyspell-mode is enabled
> or not? Or by whether the accented letter is typed as part of a word
> rather than at the beginning of it?

Sorry, I completely misunderstood your description.

What does "M-: (terminal-coding-system) RET" show on the frame created
by emacsclient?

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Thu, Jul 04, 2013 at 07:16:59PM +0300, Eli Zaretskii wrote:
> Sorry, I completely misunderstood your description.

No worries. On the contrary: thanks for your help thus far!

> What does "M-: (terminal-coding-system) RET" show on the frame created
> by emacsclient?

It returns "utf-8-unix" and does so independently of whether
flyspell-mode is on or not, and also of whether the client is GUI- or
console-based.

Cheers.
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Eli Zaretskii]

> Date: Thu, 4 Jul 2013 18:52:36 +0200
> From: Stefano Zacchiroli <zack@upsilon.cc>
> Cc: 14782@debbugs.gnu.org
> 
> > What does "M-: (terminal-coding-system) RET" show on the frame created
> > by emacsclient?
> 
> It returns "utf-8-unix" and does so independently of whether
> flyspell-mode is on or not, and also of whether the client is GUI- or
> console-based.

And what does Emacs show if you go to one of the garbled characters it
displays and type "C-u C-x ="?

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Thu, Jul 04, 2013 at 09:13:25PM +0300, Eli Zaretskii wrote:
> And what does Emacs show if you go to one of the garbled characters it
> displays and type "C-u C-x ="?

As a result of typing the accented "e" with flyspell-mode on I got two
garbled characters. On them the above commands returns, respectively:

             position: 13 of 15 (80%), column: 5
            character: Ã (displayed as Ã) (codepoint 195, #o303, #xc3)
    preferred charset: unicode (Unicode (ISO10646))
code point in charset: 0xC3
               script: latin
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), j:Japanese, l:Latin, v:Viet
             to input: type "C-x 8 RET HEX-CODEPOINT" or "C-x 8 RET NAME"
          buffer code: #xC3 #x83
            file code: not encodable by coding system undecided-unix
              display: terminal code #xC3 #x83

and:

             position: 14 of 15 (87%), column: 6
            character: � (displayed as �) (codepoint 4194217, #o17777651, #x3fffa9)
    preferred charset: eight-bit (Raw bytes 128-255)
code point in charset: 0xA9
               syntax: w 	which means: word
             category: L:Left-to-right (strong)
             to input: type "C-x 8 RET HEX-CODEPOINT" or "C-x 8 RET NAME"
          buffer code: #xA9
            file code: not encodable by coding system undecided-unix
              display: not encodable for terminal

Disabling flyspell-mode the character is not garbled anymore, and on the
accented "e" the result is:

             position: 6 of 15 (33%), column: 5
            character: é (displayed as é) (codepoint 233, #o351, #xe9)
    preferred charset: unicode (Unicode (ISO10646))
code point in charset: 0xE9
               script: latin
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), c:Chinese, j:Japanese, l:Latin, v:Viet
             to input: type "C-x 8 RET HEX-CODEPOINT" or "C-x 8 RET NAME"
          buffer code: #xC3 #xA9
            file code: not encodable by coding system undecided-unix
              display: terminal code #xC3 #xA9

Hope this helps,
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Eli Zaretskii]

> Date: Thu, 4 Jul 2013 21:20:02 +0200
> From: Stefano Zacchiroli <zack@upsilon.cc>
> Cc: 14782@debbugs.gnu.org
> 
> On Thu, Jul 04, 2013 at 09:13:25PM +0300, Eli Zaretskii wrote:
> > And what does Emacs show if you go to one of the garbled characters it
> > displays and type "C-u C-x ="?
> 
> As a result of typing the accented "e" with flyspell-mode on I got two
> garbled characters. On them the above commands returns, respectively:

Thanks.  How did you type those characters, exactly?  And what does
"M-: (keyboard-coding-system) RET" says in the frame produced by
emacsclient?

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefan Monnier]

>> And what does Emacs show if you go to one of the garbled characters it
>> displays and type "C-u C-x ="?
> As a result of typing the accented "e" with flyspell-mode on I got two
> garbled characters. On them the above commands returns, respectively:

Is this with a single frame, or multiple frames?  With a mix of GUI
frames and tty frames, maybe?  Other than flyspell, what other
modes/packages are in use?
Can you try and give a detailed recipe, ideally starting from "emacs -Q"?

> Disabling flyspell-mode the character is not garbled anymore, and on the
> accented "e" the result is:

I can't see anything obvious in flyspell which could cause that, so it
is probably some interaction between flyspell and something else (or
maybe it's just accidentally correlated with flyspell).


        Stefan

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Fri, Jul 05, 2013 at 02:16:37PM +0300, Eli Zaretskii wrote:
> Thanks.  How did you type those characters, exactly?

I use X11 compose key (which is mapped to my right-Alt) and compose
together the characters "e" and "'" (single straight quote), but that
should be transparent to Emacs.

I'd like to stress that AFAICT my general encoding settings work just
fine and that I've been using UTF-8 locales since many many years. As
evidence of that I can report also the following experiment (done in the
very same terminal where I can reproduce the problem with emacsclient):

  $ echo conférence > foo.txt

In the above, I use X11 compose key to obtain the accented "e" on the
console, and use bash redirection to create foo.txt. The obtained .txt
file contains the correct unicode code point encoded in UTF-8:

  $ od -c foo.txt
  0000000   c   o   n   f 303 251   r   e   n   c   e  \n
  0000014

FWIW emacsclients (both on the console and in the GUI) can *read* that
file just fine, showing the appropriate character. It is just at
*insert* time, and only if flyspell-mode is enabled, that I get
character garbling.

> And what does "M-: (keyboard-coding-system) RET" says in the frame
> produced by emacsclient?

It returns "utf-8-unix" in all possible combinations of console/GUI
client and flyspell-mode on/off.

Cheers.

PS I'll now work on a complete recipe to reproduce the problem starting
   from emacs -Q and post it to the bug log. But it will need X11
   compose key.
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Fri, Jul 05, 2013 at 07:42:59AM -0400, Stefan Monnier wrote:
> Is this with a single frame, or multiple frames?  With a mix of GUI
> frames and tty frames, maybe?  Other than flyspell, what other
> modes/packages are in use?
> Can you try and give a detailed recipe, ideally starting from "emacs -Q"?

In fact, that's pretty easy to reproduce here:

  $ emacs -Q                    # the usual main GUI frame appears
  M-x server-mode

then in a terminal:

  $ emacsclient -t foo.txt
  conférence                    # this works fine
  M-x flyspell-mode
  conférence                    # this gives garbled output

In both cases I wrote the word "conférence" using X composing, as
mentioned in my previous mail.

To answer your earlier question, I've tried will all sort of
GUI/console client combinations, ditto for combinations of --daemon
(to avoid starting the main GUI frame) and no --daemon, but that
doesn't seem to matter at all. In the above recipe I don't use
--daemon, but rather start server-mode from the main GUI frame. There
is no visible difference.

> > Disabling flyspell-mode the character is not garbled anymore, and on the
> > accented "e" the result is:
> 
> I can't see anything obvious in flyspell which could cause that, so it
> is probably some interaction between flyspell and something else (or
> maybe it's just accidentally correlated with flyspell).

Well, it might be, of course. But in the above example I use -Q and
only activate flyspell-mode. So if it's an interaction with other
modes, then it's something very basic that Emacs uses by default, such
as text-mode (or server-mode, but that would be pretty weird).

Cheers.
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefan Monnier]

> In fact, that's pretty easy to reproduce here:

>   $ emacs -Q                    # the usual main GUI frame appears
>   M-x server-mode

> then in a terminal:

>   $ emacsclient -t foo.txt
>   conférence                    # this works fine
>   M-x flyspell-mode
>   conférence                    # this gives garbled output

> In both cases I wrote the word "conférence" using X composing, as
> mentioned in my previous mail.

Great, I can reproduce it here, thank you.

> only activate flyspell-mode. So if it's an interaction with other
> modes, then it's something very basic that Emacs uses by default, such
> as text-mode (or server-mode, but that would be pretty weird).

Indeed.


        Stefan "not found it yet"

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefan Monnier]

> then in a terminal:

>   $ emacsclient -t foo.txt
>   conférence                    # this works fine
>   M-x flyspell-mode
>   conférence                    # this gives garbled output

The problem was in sit-for.
I installed the patch below which should fix it,


        Stefan


=== modified file 'lisp/subr.el'
--- lisp/subr.el	2013-07-04 09:39:36 +0000
+++ lisp/subr.el	2013-07-06 00:06:02 +0000
@@ -2200,7 +2196,10 @@
     (or nodisp (redisplay)))
    (t
     (or nodisp (redisplay))
-    (let ((read (read-event nil nil seconds)))
+    ;; FIXME: we should not read-event here at all, because it's much too
+    ;; difficult to reliably "undo" a read-event by pushing it onto
+    ;; unread-command-events.
+    (let ((read (read-event nil t seconds)))
       (or (null read)
 	  (progn
 	    ;; If last command was a prefix arg, e.g. C-u, push this event onto

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefano Zacchiroli]

On Fri, Jul 05, 2013 at 08:12:26PM -0400, Stefan Monnier wrote:
> The problem was in sit-for.
> I installed the patch below which should fix it,

I confirm that the patch fixes the problem for me.

Thanks a lot for (another) blazing-fast bug fix!
Cheers.
-- 
Stefano Zacchiroli  . . . . . . .  zack@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader  . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

bug#14782: 24.3.50; flyspell-mode garbles accents in terminal clients

[Stefan Monnier]

>> The problem was in sit-for.
>> I installed the patch below which should fix it,
> I confirm that the patch fixes the problem for me.

Great.

> Thanks a lot for (another) blazing-fast bug fix!

Among a sea of unfixed bugs ;-)


        Stef