bug#6920: 23.2; ESC passed out of order to post-read-conversion

bug#6920: 23.2; ESC passed out of order to post-read-conversion

[Ryan Johnson]

  This problem shows up running in terminal mode on a cygwin xterm,
apparently because it sends terminal escape sequences one character at a
time (where linux usually sends the whole thing at once). To reproduce,
run the following three elisp commands:

(define-coding-system 'utf-8-echo
   "Echoes all input to the message area"
   :coding-type 'utf-8
   :mnemonic ?U
   :ascii-compatible-p t
   :charset-list '(unicode)
   :post-read-conversion 'echo-conversion)
(defun echo-conversion (len)
   (let* ((p (point))
      (e (+ p len))
      (str (buffer-substring p e)))
     (message "%s" str)
     len))
(set-keyboard-coding-system 'utf-8-echo)

Then start using the left arrow key to navigate. The escape sequence is
'^[OD' and every few arrow presses garbled bits of the "OD" portion will
appear in the buffer and the point doesn't move properly. A
representative sample of the message buffer looks like this:

^[
O
D
^[
O
D
O
D
^[ [2 times]
O
D

The lossage buffer (below) shows how emacs becomes very confused as a
result, reporting 'ESC O D O D D O D D ESC' at one point. Defining a
similar coding system based on iso-8859-1 instead of utf-8 causes emacs
to crash. Unfortunately, cygwin-gdb doesn't trap the signal so I can't
produce a stack trace, but sometimes it gets stuck in an endless loop
inside malloc, which suggests some sort of memory corruption is the
culprit. As far as I know, 'ESC O D' is valid in both coding systems
so that shouldn't be the problem.

System info follows...

In GNU Emacs 23.2.1 (i686-pc-cygwin)
  of 2010-08-12 on host
configured using `configure  '--prefix=/home/Ryan/apps/emacs-23.2' 
'--without-xpm' '--without-png' '--without-gif''

Important settings:
   value of $LC_ALL: nil
   value of $LC_COLLATE: nil
   value of $LC_CTYPE: nil
   value of $LC_MESSAGES: nil
   value of $LC_MONETARY: nil
   value of $LC_NUMERIC: nil
   value of $LC_TIME: nil
   value of $LANG: C.UTF-8
   value of $XMODIFIERS: nil
   locale-coding-system: utf-8-unix
   default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
   tooltip-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent input:
ESC [ > 0 ; 2 6 2 ; 0 c ESC ] 1 1 ; r g b : 0 0 0 0
/ 0 0 0 0 / 0 0 0 0 ESC \ ESC O B ESC O B ESC O B ESC
O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B
ESC O B ESC O B ESC O B ESC O B ESC C-x ESC [ 1 ; 5
B ESC C-x ESC x s e t - k e y TAB RET u t f - 8 - e
c h o RET O D D ESC O D ESC O D ESC O D ESC O D O D
D ESC O D ESC O D ESC O D ESC O D ESC O D O D D O D
D ESC O D ESC O D ESC x r e p o r t - e m TAB RET

Recent messages:
t
-
e
m

^M
E
S
C
^M

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail regexp-opt ecomplete rfc822 mml
easymenu mml-sec password-cache mm-decode mm-bodies mm-encode mailcap
mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader
gnus-util netrc time-date mm-util mail-prsvr gmm-utils wid-edit
mailheader canlock sha1 hex-util hashcash mail-utils emacsbug tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer
select scroll-bar mldrag mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev loaddefs button minibuffer faces
cus-face files text-properties overlay md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote
make-network-process x multi-tty emacs)

bug#6920: 23.2; ESC passed out of order to post-read-conversion

[Stefan Kangas]

Ryan Johnson <ryanjohn@ece.cmu.edu> writes:

>  This problem shows up running in terminal mode on a cygwin xterm,
> apparently because it sends terminal escape sequences one character at a
> time (where linux usually sends the whole thing at once). To reproduce,
> run the following three elisp commands:
>
> (define-coding-system 'utf-8-echo
>   "Echoes all input to the message area"
>   :coding-type 'utf-8
>   :mnemonic ?U
>   :ascii-compatible-p t
>   :charset-list '(unicode)
>   :post-read-conversion 'echo-conversion)
> (defun echo-conversion (len)
>   (let* ((p (point))
>      (e (+ p len))
>      (str (buffer-substring p e)))
>     (message "%s" str)
>     len))
> (set-keyboard-coding-system 'utf-8-echo)
>
> Then start using the left arrow key to navigate. The escape sequence is
> '^[OD' and every few arrow presses garbled bits of the "OD" portion will
> appear in the buffer and the point doesn't move properly. A
> representative sample of the message buffer looks like this:
>
> ^[
> O
> D
> ^[
> O
> D
> O
> D
> ^[ [2 times]
> O
> D
>
> The lossage buffer (below) shows how emacs becomes very confused as a
> result, reporting 'ESC O D O D D O D D ESC' at one point. Defining a
> similar coding system based on iso-8859-1 instead of utf-8 causes emacs
> to crash. Unfortunately, cygwin-gdb doesn't trap the signal so I can't
> produce a stack trace, but sometimes it gets stuck in an endless loop
> inside malloc, which suggests some sort of memory corruption is the
> culprit. As far as I know, 'ESC O D' is valid in both coding systems
> so that shouldn't be the problem.
>
> System info follows...
>
> In GNU Emacs 23.2.1 (i686-pc-cygwin)
>  of 2010-08-12 on host
> configured using `configure  '--prefix=/home/Ryan/apps/emacs-23.2'
> '--without-xpm' '--without-png' '--without-gif''

This was reported 9 years ago but unfortunately never got a reply at the
time.  Can you still reproduce it on a modern version of Emacs, for
example the latest version 26.3?

Best regards,
Stefan Kangas

bug#6920: 23.2; ESC passed out of order to post-read-conversion

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:
> This was reported 9 years ago but unfortunately never got a reply at the
> time.  Can you still reproduce it on a modern version of Emacs, for
> example the latest version 26.3?

The email bounced, so I think it's unlikely that we'll make any
progress here and I'm closing this.

If anyone can still reproduce the original issue, please reopen the bug report.

Best regards,
Stefan Kangas