bug#17231: [PATCH] term.el: unreliable directory tracking

bug#17231: [PATCH] term.el: unreliable directory tracking

[Callum Cameron]

I've had mixed results with directory tracking in ansi-term. Sometimes
output like this appears in my terminal (offending line highlighted
with !):

> callum@Sulabesi:~/emacsBugTest/git/emacs/src$ ssh vm3.local
> Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.11.0-19-generic x86_64)
>
>  * Documentation:  https://help.ubuntu.com/
>
> Last login: Wed Apr  9 09:49:46 2014 from sulabesi.local
! nSiTu callum
> callum@VM3:~$

I put a call to (message "\"%s\"" str) at the start of
term-emulate-terminal to see what was going on. Here are the relevant
values of str:

> "^[AnSiTu callum"
> "^M
> ^[AnSiTh VM3^M
> ^[AnSiTc /home/callum^M
> callum@VM3:~$ "

The newline after the AnSiTu command is in the *next* string, so the
command doesn't match the regexp in
term-handle-ansi-terminal-messages. So the '^[A' gets eaten by the
escape-char handling code (and does nothing), and the rest of the
command gets printed as normal text: 'nSiTu callum'.

I haven't found a way to reproduce this reliably. It seems to happen
more often on slower machines, but happens occasionally even on fast
ones. In this case I was doing a login over SSH, but it happens in
local shells, too (although not as often). Here's the minimal
.bash_profile that was on vm3.local in this example. It sometimes
causes the bug for me (but might not for you!):

> if [ "${TERM}" = 'eterm-color' ]; then
>     printf '\eAnSiTu %s\n' "$(id -un)"
>     printf '\eAnSiTh %s\n' "$(hostname)"
>     printf '\eAnSiTc %s\n' "$(pwd)"
> fi

As you can see, this isn't even doing directory tracking, just setting
up the initial state when the shell starts - but it's enough to
trigger the bug. The order of the prints doesn't seem to matter; it's
always the first command that gets missed. It doesn't seem to be a
bash-specific problem, either, since I've had the same thing happen in
zsh, and it happens locally as well as over SSH. The problem is
present both in Emacs 24.3.1 from the Ubuntu package repositories, and
in trunk (as of commit 35adb6a463abe5e0539bc72082bdb0217a6c11d5 in the
git mirror).

If you don't experience the bug, you can force it manually by doing
(in bash):

> printf '\eAnSiTu %s' "$(id -un)"; sleep 1; printf '\n'

Everything here has been done in emacs -Q, then going straight to M-x
ansi-term.

The patch checks to see if there is an incomplete command at the end
of term-emulate-terminal's input string, and, if so, saves it so the
whole command can be processed when the next string arrives. I'm a
Lisp newbie, so I don't know if this is the best way to do it, and it
isn't foolproof - if the command gets split in the 'AnSiT' magic, it
won't get handled. But I've never seen this happen (I've only ever
seen the trailing newline get split off), and I've been using it for a
few weeks without problems, and without any noticeable slowdown.

Patch against the term.el in trunk.


*** old_term.el    2014-04-09 10:41:01.264492832 +0100
--- new_term.el    2014-04-09 10:27:56.784487049 +0100
***************
*** 607,612 ****
--- 607,615 ----
  the user command `term-send-input'.  `term-simple-send' just sends
  the string plus a newline.")

+ (defvar term-partial-ansi-terminal-message nil "Keep partial ansi
+ terminal messages for future processing.")
+
  (defcustom term-eol-on-send t
    "Non-nil means go to the end of the line before sending input.
  See `term-send-input'."
***************
*** 1097,1102 ****
--- 1100,1107 ----
    (make-local-variable 'ange-ftp-default-password)
    (make-local-variable 'ange-ftp-generate-anonymous-password)

+   (make-local-variable 'term-partial-ansi-terminal-message)
+
    ;; You may want to have different scroll-back sizes -mm
    (make-local-variable 'term-buffer-maximum-size)

***************
*** 2693,2698 ****
--- 2698,2708 ----
  ;;difference ;-) -mm

  (defun term-handle-ansi-terminal-messages (message)
+   ;; Handle stored partial message
+   (when term-partial-ansi-terminal-message
+     (setq message (concat term-partial-ansi-terminal-message message))
+     (setq term-partial-ansi-terminal-message nil))
+
    ;; Is there a command here?
    (while (string-match "\eAnSiT.+\n" message)
      ;; Extract the command code and the argument.
***************
*** 2745,2750 ****
--- 2755,2767 ----
        (setq ange-ftp-default-user nil)
        (setq ange-ftp-default-password nil)
        (setq ange-ftp-generate-anonymous-password nil)))))
+
+   ;; If there is a partial message at the end of the string, store it
+   ;; for future use.
+   (when (string-match "\eAnSiT.+$" message)
+     (setq term-partial-ansi-terminal-message (match-string 0 message))
+     (setq message (replace-match "" t t message)))
+
    message)



Here's the output from report-emacs-bug, too:

In GNU Emacs 24.4.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.20)
 of 2014-04-09 on Sulabesi
Windowing system distributor `The X.Org Foundation', version 11.0.11405000
System Description:    Linux Mint 16 Petra

Configured using:
 `configure --prefix=/home/callum/emacsBugTest/install'

Configured features:
XPM JPEG TIFF GIF PNG SOUND GSETTINGS NOTIFY GNUTLS FREETYPE XFT ZLIB

Important settings:
  value of $LC_CTYPE: en_GB.UTF-8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Term

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<menu> a n s i - t e r m <return> <return> s s h SPC
v m 3 . l o c a l <return> C-d <up> <return> C-d <up>
<return> C-d <up> <return> C-d <up> <return> <menu>
r e p o r t - e m a c s - b u g <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils tramp-cache
tramp-sh tramp tramp-compat auth-source eieio byte-opt bytecomp
byte-compile cconv eieio-core gnus-util mm-util mail-prsvr
password-cache tramp-loaddefs trampver shell pcomplete comint ansi-color
format-spec advice help-fns term disp-table easymenu ehelp ring
time-date tooltip electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode prog-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Memory information:
((conses 16 98260 9766)
 (symbols 48 20811 0)
 (miscs 40 47 107)
 (strings 32 20424 5213)
 (string-bytes 1 608496)
 (vectors 16 13344)
 (vector-slots 8 418292 5933)
 (floats 8 85 176)
 (intervals 56 279 285)
 (buffers 960 12)
 (heap 1024 32967 874))

bug#17231: [PATCH] term.el: unreliable directory tracking

[Noam Postavsky]

So this has been sitting for a while...

Callum Cameron <cjcameron7@gmail.com> writes:

> The patch checks to see if there is an incomplete command at the end
> of term-emulate-terminal's input string, and, if so, saves it so the
> whole command can be processed when the next string arrives. I'm a
> Lisp newbie, so I don't know if this is the best way to do it, and it
> isn't foolproof - if the command gets split in the 'AnSiT' magic, it
> won't get handled. But I've never seen this happen (I've only ever
> seen the trailing newline get split off), and I've been using it for a
> few weeks without problems, and without any noticeable slowdown.

The patch looks okay to me.  It's a bit unfortunate that it adds yet
another way to store partial strings; I have a patch which fixes things
in a more unified way, but my patch rewrites too much to go on the
release branch.

So I think it would be nice if this patch could go to emacs-26.  I
presume you don't have copyright assignment (so the patch should be
marked as a tiny change)?

> + (defvar term-partial-ansi-terminal-message nil "Keep partial ansi
> + terminal messages for future processing.")

And this newline in the middle of the docstring should be fixed.

bug#17231: [PATCH] term.el: unreliable directory tracking

[Noam Postavsky]

tags 17231 fixed
close 17231 26.1
quit

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> The patch looks okay to me.  It's a bit unfortunate that it adds yet
> another way to store partial strings; I have a patch which fixes things
> in a more unified way, but my patch rewrites too much to go on the
> release branch.
>
> So I think it would be nice if this patch could go to emacs-26.

Pushed to emacs-26 [1: 5472568a3c], and some tests in a followup commit
[2: 1d50c185f0].  I pushed my more invasive patch to master [3:
47019a521f].

[1: 5472568a3c]: 2018-01-18 22:17:07 -0500
  Handle split AnSiT messages for term.el (Bug#17231)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=5472568a3c2338856d25380012ee4398e024c806

[2: 1d50c185f0]: 2018-01-18 22:17:40 -0500
  Add tests for term.el
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=1d50c185f0c857bb1a85945314b522540071f796

[3: 47019a521f]: 2018-01-18 22:25:35 -0500
  Switch term.el to lexical binding, and clean up code a bit
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=47019a521f774fbd13441e178a6a82c9989b9912