bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[Michael Hoffman]

Steps to reproduce:

1. emacs -Q
2. M-x ansi-term RET RET ; run ansi-term with default shell
3. C-c C-j ; term-line-mode
4. git clone https://github.com/github/linguist.git ; needs to be a big enough repo for progress rewriting to cause a problem here--very small repos don't demonstrate it

The output looks like this

```
mhoffman@mhoffman2 ~
$ git clone https://github.com/github/linguist.git
Cloning into 'linguist'...
remote: Enumerating objects: 42098, done.
remote: Total 42098 (delta 0), reused 0 (delta 0), pack-reused 42098
Receiving objects: 100% (42098/42098), 38.14 MiB | 42.50 MiB/s, done.
Resolving deltas: 100% (23911/23911), done.

mhoffman@mhoffman2 ~
$ Resolving deltas:  80% (19129/23911)Resolving deltas:  51% (12195/23911)Resolving deltas:  50% (11956/23911)Resolving deltas:  49% (11717/23911)Resolving deltas:  48% (11478/23911)Resolving deltas:  47% (11239/23911)Resolving deltas:  44% (10521/23911)Resolving deltas:  39% (9326/23911)Resolving deltas:  30% (7174/23911)Resolving deltas:  18% (4304/23911)Resolving deltas:  11% (2631/23911)Receiving objects:  93% (39152/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  82% (34521/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  77% (32416/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  70% (29469/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  54% (22733/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  46% (19366/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  45% (18945/42098), 14.68 MiB | 29.35 MiB/sReceiving objects:  41% (17261/42098)Receiving objects:  26% (10946/42098)Receiving objects:  22% (9262/42098)Receiving objects:  20% (8420/42098)Receiving objects:  17% (7157/42098)Receiving objects:  16% (6736/42098)Receiving objects:   2% (842/42098)
```

Note all the junk that has piled up after the next prompt.

Expected final output is more like what occurs in term-char-mode instead:

```
mhoffman@mhoffman2 ~
$ git clone https://github.com/github/linguist.git
Cloning into 'linguist'...
remote: Enumerating objects: 42098, done.
remote: Total 42098 (delta 0), reused 0 (delta 0), pack-reused 42098
Receiving objects: 100% (42098/42098), 38.12 MiB | 39.04 MiB/s, done.
Resolving deltas: 100% (23913/23913), done.

mhoffman@mhoffman2 ~
$ 
```

I tried adding the following advice to debug this a bit.

```
(defadvice term-emulate-terminal
    (before dribble-str (proc str) activate)
  (message "term-emulate-terminal: `%s'" str))

(defadvice term-handle-ansi-escape
     (before dribble-str (proc params char) activate)
   (message "term-handle-ansi-escape(%s): `%s'" params char))

(defadvice term-erase-in-line
     (before dribble-str (kind) activate)
   (message "term-erase-in-line: %s" kind))
``

Results are below (I have replaced ^M and ^[ characters with the two-character strings):

```
term-emulate-terminal: ‘git clone https://github.com/github/linguist.git^M
^[[?2004l^M’
term-handle-ansi-escape((0)): ‘108’
term-emulate-terminal: ‘Cloning into 'linguist'...^M
’
term-emulate-terminal: ‘remote: Enumerating objects: 42098, done.^[[K^M
’
term-handle-ansi-escape((0)): ‘75’
term-erase-in-line: 0
term-emulate-terminal: ‘Receiving objects:   0% (1/42098)^MReceiving objects:   1% (421/42098)^MReceiving objects:   2% (842/42098)^M’
term-emulate-terminal: ‘Receiving objects:   3% (1263/42098)^MReceiving objects:   4% (1684/42098)^MReceiving objects:   5% (2105/42098)^MReceiving objects:   6% (2526/42098)^MReceiving objects:   7% (2947/42098)^MReceiving objects:   8% (3368/42098)^MReceiving objects:   9% (3789/42098)^MReceiving objects:  10% (4210/42098)^MReceiving objects:  11% (4631/42098)^MReceiving objects:  12% (5052/42098)^MReceiving objects:  13% (5473/42098)^MReceiving objects:  14% (5894/42098)^MReceiving objects:  15% (6315/42098)^M’
term-emulate-terminal: ‘Receiving objects:  16% (6736/42098)^MReceiving objects:  17% (7157/42098)^MReceiving objects:  18% (7578/42098)^M’
term-emulate-terminal: ‘Receiving objects:  19% (7999/42098)^M’
term-emulate-terminal: ‘Receiving objects:  20% (8420/42098)^M’
term-emulate-terminal: ‘Receiving objects:  21% (8841/42098)^MReceiving objects:  22% (9262/42098)^MReceiving objects:  23% (9683/42098)^M’
term-emulate-terminal: ‘Receiving objects:  24% (10104/42098)^MReceiving objects:  25% (10525/42098)^MReceiving objects:  26% (10946/42098)^M’
term-emulate-terminal: ‘Receiving objects:  27% (11367/42098)^MReceiving objects:  28% (11788/42098)^MReceiving objects:  29% (12209/42098)^MReceiving objects:  30% (12630/42098)^MReceiving objects:  31% (13051/42098)^MReceiving objects:  32% (13472/42098)^MReceiving objects:  33% (13893/42098)^MReceiving objects:  34% (14314/42098)^MReceiving objects:  35% (14735/42098)^MReceiving objects:  36% (15156/42098)^MReceiving objects:  37% (15577/42098)^MReceiving objects:  38% (15998/42098)^MReceiving objects:  39% (16419/42098)^MReceiving objects:  40% (16840/42098)^MReceiving objects:  41% (17261/42098)^M’
term-emulate-terminal: ‘Receiving objects:  42% (17682/42098)^MReceiving objects:  43% (18103/42098)^MReceiving objects:  44% (18524/42098)^MReceiving objects:  45% (18945/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  46% (19366/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  47% (19787/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  48% (20208/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  49% (20629/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  50% (21049/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  51% (21470/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  52% (21891/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  53% (22312/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  54% (22733/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  55% (23154/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  56% (23575/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  57% (23996/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  58% (24417/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  59% (24838/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  60% (25259/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  61% (25680/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  62% (26101/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  63% (26522/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  64% (26943/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  65% (27364/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  66% (27785/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  67% (28206/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  68% (28627/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  69% (29048/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  70% (29469/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  71% (29890/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  72% (30311/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  73% (30732/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  74% (31153/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  75% (31574/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  76% (31995/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  77% (32416/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  78% (32837/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  79% (33258/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  80% (33679/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  81% (34100/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  82% (34521/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  83% (34942/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  84% (35363/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  85% (35784/42098), 13.85 MiB | 27.68 MiB/s^M’
term-emulate-terminal: ‘Receiving objects:  86% (36205/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  87% (36626/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  88% (37047/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  89% (37468/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  90% (37889/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  91% (38310/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  92% (38731/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  93% (39152/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  94% (39573/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  95% (39994/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  96% (40415/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  97% (40836/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  98% (41257/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects:  99% (41678/42098), 13.85 MiB | 27.68 MiB/s^Mremote: Total 42098 (delta 0), reused 0 (delta 0), pack-reused 42098^[[K^M
’
term-handle-ansi-escape((0)): ‘75’
term-erase-in-line: 0
term-emulate-terminal: ‘Receiving objects: 100% (42098/42098), 13.85 MiB | 27.68 MiB/s^MReceiving objects: 100% (42098/42098), 38.13 MiB | 40.67 MiB/s, done.^M
Resolving deltas:   0% (0/23915)^MResolving deltas:   1% (240/23915)^MResolving deltas:   2% (479/23915)^MResolving deltas:   3% (718/23915)^MResolving deltas:   4% (957/23915)^MResolving deltas:   5% (1196/23915)^MResolving deltas:   6% (1435/23915)^MResolving deltas:   7% (1675/23915)^MResolving deltas:   8% (1914/23915)^MResolving deltas:   9% (2153/23915)^MResolving deltas:  10% (2392/23915)^MResolving deltas:  11% (2631/23915)^MResolving deltas:  12% (2870/23915)^MResolving deltas:  13% (3109/23915)^MResolving deltas:  14% (3349/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  15% (3588/23915)^MResolving deltas:  16% (3827/23915)^MResolving deltas:  17% (4066/23915)^MResolving deltas:  18% (4305/23915)^MResolving deltas:  19% (4544/23915)^MResolving deltas:  20% (4783/23915)^MResolving deltas:  21% (5023/23915)^MResolving deltas:  22% (5262/23915)^MResolving deltas:  23% (5501/23915)^MResolving deltas:  24% (5740/23915)^MResolving deltas:  25% (5979/23915)^MResolving deltas:  26% (6218/23915)^MResolving deltas:  27% (6458/23915)^MResolving deltas:  28% (6697/23915)^MResolving deltas:  29% (6936/23915)^MResolving deltas:  30% (7175/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  31% (7414/23915)^MResolving deltas:  32% (7653/23915)^MResolving deltas:  33% (7892/23915)^MResolving deltas:  34% (8132/23915)^MResolving deltas:  35% (8371/23915)^MResolving deltas:  36% (8610/23915)^MResolving deltas:  37% (8849/23915)^MResolving deltas:  38% (9088/23915)^MResolving deltas:  39% (9327/23915)^MResolving deltas:  40% (9566/23915)^MResolving deltas:  41% (9806/23915)^MResolving deltas:  42% (10045/23915)^MResolving deltas:  43% (10284/23915)^MResolving deltas:  44% (10523/23915)^MResolving deltas:  45% (10762/23915)^MResolving deltas:  46% (11001/23915)^MResolving deltas:  47% (11241/23915)^MResolving deltas:  48% (11480/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  49% (11719/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  50% (11958/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  51% (12197/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  52% (12436/23915)^MResolving deltas:  53% (12675/23915)^MResolving deltas:  54% (12915/23915)^MResolving deltas:  55% (13154/23915)^MResolving deltas:  56% (13393/23915)^MResolving deltas:  57% (13632/23915)^MResolving deltas:  58% (13871/23915)^MResolving deltas:  59% (14110/23915)^MResolving deltas:  60% (14349/23915)^MResolving deltas:  61% (14589/23915)^MResolving deltas:  62% (14828/23915)^MResolving deltas:  63% (15067/23915)^MResolving deltas:  64% (15306/23915)^MResolving deltas:  65% (15545/23915)^MResolving deltas:  66% (15784/23915)^MResolving deltas:  67% (16024/23915)^MResolving deltas:  68% (16263/23915)^MResolving deltas:  69% (16502/23915)^MResolving deltas:  70% (16741/23915)^MResolving deltas:  71% (16980/23915)^MResolving deltas:  72% (17219/23915)^MResolving deltas:  73% (17458/23915)^MResolving deltas:  74% (17698/23915)^MResolving deltas:  75% (17937/23915)^MResolving deltas:  76% (18176/23915)^MResolving deltas:  77% (18415/23915)^MResolving deltas:  78% (18654/23915)^MResolving deltas:  79% (18893/23915)^MResolving deltas:  80% (19132/23915)^M’
term-emulate-terminal: ‘Resolving deltas:  81% (19372/23915)^MResolving deltas:  82% (19611/23915)^MResolving deltas:  83% (19850/23915)^MResolving deltas:  84% (20089/23915)^MResolving deltas:  85% (20328/23915)^MResolving deltas:  86% (20567/23915)^MResolving deltas:  87% (20807/23915)^MResolving deltas:  88% (21046/23915)^MResolving deltas:  89% (21285/23915)^MResolving deltas:  90% (21524/23915)^MResolving deltas:  91% (21763/23915)^MResolving deltas:  92% (22002/23915)^MResolving deltas:  93% (22241/23915)^MResolving deltas:  94% (22481/23915)^MResolving deltas:  95% (22720/23915)^MResolving deltas:  96% (22959/23915)^MResolving deltas:  97% (23198/23915)^MResolving deltas:  98% (23437/23915)^MResolving deltas:  99% (23676/23915)^MResolving deltas: 100% (23915/23915)^MResolving deltas: 100% (23915/23915), done.^M
’
term-emulate-terminal: ‘\x1a//home/mhoffman^M
^[[?2004h^M^M
^[AnSiTu mhoffman^M
^[AnSiTc ~^M
^[AnSiTh mhoffman2.REDACTED^M
^[[32mmhoffman@mhoffman2 ^[[33m~^[[35m^[[36m^[[0m^M^M
$ ’
term-handle-ansi-escape((0)): ‘104’
term-handle-ansi-escape((32)): ‘109’
term-handle-ansi-escape((33)): ‘109’
term-handle-ansi-escape((35)): ‘109’
term-handle-ansi-escape((36)): ‘109’
term-handle-ansi-escape((0)): ‘109’
Mark set [2 times]
```

It looks like the term-termcap-format advertises ^M for the cr capability which "move[s] the cursor to the beginning of the line it is on" but this doesn't work. https://www.gnu.org/software/termutils/manual/termcap-1.3/html_mono/termcap.html 

In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
 of 2022-01-24, modified by Debian built on lgw01-amd64-048
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Ubuntu 22.04.1 LTS

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-cairo
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/emacs-NbbgEv/emacs-27.1+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD
JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Term

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
gnus-util rmail rmail-loaddefs text-property-search seq byte-opt gv
bytecomp byte-compile cconv mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils help-mode novice time-date
subr-x cl-loaddefs cl-lib term disp-table easymenu comint ansi-color
ehelp ring tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 118271 9462)
 (symbols 48 6851 1)
 (strings 32 17828 2074)
 (string-bytes 1 582159)
 (vectors 16 10720)
 (vector-slots 8 138689 10086)
 (floats 8 21 44)
 (intervals 56 16919 4)
 (buffers 1000 13))

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[Michael Hoffman]

This line of bash is a better minimum example for this that doesn't require going over the network:

printf "line %d" 0; for i in {1..500}; do tput cr; printf "line %d" "$i"; done

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[Eli Zaretskii]

> Date: Wed, 21 Dec 2022 21:58:40 +0000
> From: "Michael Hoffman" <emacs-hoffman@snkmail.com>
> 
> This line of bash is a better minimum example for this that doesn't require going over the network:
> 
> printf "line %d" 0; for i in {1..500}; do tput cr; printf "line %d" "$i"; done

Funny thing is, "M-x term" does produce the expected results, and I
cannot find what makes "M-x ansi-term" behave differently, as it's
supposed to be almist the same as "M-x term".

Miha, could you perhaps look into this issue?

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2084 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 21 Dec 2022 21:58:40 +0000
>> From: "Michael Hoffman" <emacs-hoffman@snkmail.com>
>> 
>> This line of bash is a better minimum example for this that doesn't require going over the network:
>> 
>> printf "line %d" 0; for i in {1..500}; do tput cr; printf "line %d" "$i"; done
>
> Funny thing is, "M-x term" does produce the expected results, and I
> cannot find what makes "M-x ansi-term" behave differently, as it's
> supposed to be almist the same as "M-x term".
>
> Miha, could you perhaps look into this issue?

A more reliable example would be this bash line:

    printf 'foo \015'; sleep 2; printf 'bar'

In char mode, it writes foo and overwrites it with bar, which is
expected. But in line mode, it pushes foo after the process mark. I
could reproduce the issue in both M-x term and M-x ansi-term. The issue
happens due this code in function term-emulate-terminal:

    ;; If the buffer is in line mode, and there is a partial
    ;; input line, save the line (by narrowing to leave it
    ;; outside the restriction ) until we're done with output.
    (when (and (> (point-max) (process-mark proc))
           (term-in-line-mode))
      (narrow-to-region (point-min) (process-mark proc)))

The idea is to let the user edit his partial input during a long-running
command. But term.el assumes that, in line mode, all text after process
mark is user input, it doesn't distinguish between actual user input and
process output that happens to be behind process mark.

This is also the reason why a lot of full-screen TUI programs such as
"htop" don't work correctly in line mode even if they do in char mode.

Two possible ideas to solve this:

- Introduce a new marker to separate user input from process output.

- Use text properties to distinguish user input from process output.
  This is what comint.el does, it marks process output with 'field' =
  'output'.

Hope this helps. Unfortunately I can't promise to be able work on any
solution at the moment.

Best regards.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[Eli Zaretskii]

> From: <miha@kamnitnik.top>
> Cc: 60244@debbugs.gnu.org
> Date: Sat, 24 Dec 2022 16:02:01 +0100
> 
> A more reliable example would be this bash line:
> 
>     printf 'foo \015'; sleep 2; printf 'bar'
> 
> In char mode, it writes foo and overwrites it with bar, which is
> expected. But in line mode, it pushes foo after the process mark. I
> could reproduce the issue in both M-x term and M-x ansi-term. The issue
> happens due this code in function term-emulate-terminal:
> 
>     ;; If the buffer is in line mode, and there is a partial
>     ;; input line, save the line (by narrowing to leave it
>     ;; outside the restriction ) until we're done with output.
>     (when (and (> (point-max) (process-mark proc))
>            (term-in-line-mode))
>       (narrow-to-region (point-min) (process-mark proc)))
> 
> The idea is to let the user edit his partial input during a long-running
> command. But term.el assumes that, in line mode, all text after process
> mark is user input, it doesn't distinguish between actual user input and
> process output that happens to be behind process mark.
> 
> This is also the reason why a lot of full-screen TUI programs such as
> "htop" don't work correctly in line mode even if they do in char mode.
> 
> Two possible ideas to solve this:
> 
> - Introduce a new marker to separate user input from process output.
> 
> - Use text properties to distinguish user input from process output.
>   This is what comint.el does, it marks process output with 'field' =
>   'output'.
> 
> Hope this helps. Unfortunately I can't promise to be able work on any
> solution at the moment.

Thanks for the analysis.  This is AFAIU an old issue, so fixing it is
not urgent.  I hope Someone(TM) will implement one of your solutions
at some point.

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[Michael Hoffman]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

Thanks so much for looking into this.

On Sat, Dec 24, 2022 at 10:01 AM miha wrote:

> Two possible ideas to solve this:
>
> - Introduce a new marker to separate user input from process output.
>
> - Use text properties to distinguish user input from process output.
>   This is what comint.el does, it marks process output with 'field' =
>   'output'.
>

Is one of these options preferred? The second idea sounds more robust, and
also it is nice that the example of comint.el exists for it. But perhaps I
am naive about what they require.

[-- Attachment #2: Type: text/html, Size: 917 bytes --]

bug#60244: 27.1; term-line-mode works poorly with git progress rewriting

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

"Michael Hoffman" <emacs-hoffman@snkmail.com> writes:

> Thanks so much for looking into this.
>
> On Sat, Dec 24, 2022 at 10:01 AM miha wrote:
>
>     Two possible ideas to solve this:
>
>     - Introduce a new marker to separate user input from process output.
>
>     - Use text properties to distinguish user input from process output.
>       This is what comint.el does, it marks process output with 'field' =
>       'output'.
>
> Is one of these options preferred? The second idea sounds more robust, and also it is nice that the example of comint.el
> exists for it. But perhaps I am naive about what they require.

I prefer the second option. While it is probably more work to implement
than the first one, it opens up possibilities of more robust input
navigation and multi-line input editing in term-line-mode, just like in
comint.el with 'comint-use-prompt-regexp' = nil.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]