bug#66469: 30.0.50; Fields in eshell output regress behavior when yanking output into new input

bug#66469: 30.0.50; Fields in eshell output regress behavior when yanking output into new input

[Jules Tamagnan]

Essentially the issue is that when killing eshell output and yanking it
into a new input line one cannot navigate as expected. Thankfully this
behavior is extremely easy to reproduce from `emacs -Q'. For example:

$ ls
blah1 blah2 blah3

Now if I kill this line and yank it into the next output line like:

$ blah1 blah2 blah3

From here I can go to the beginning of the line such that my cursor is
before `blah1' and after the `$ ' and I can start adding a new
command. Lets run `ls':

$ ls blah1 blah2 blah3

If I go to the end of the line now and then go back to the beginning of
the line I do not end up before `ls'. Instead I end up before
`blah1'. No amount of C-a or M-b will allow me to go to the beginning of
the line. Many other text navigation features seem impaired as well.


In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.38, cairo version 1.16.0)
Repository revision: fc7c1f1e93e48ea17657da247590439bff831798
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: NixOS 23.11 (Tapir)

Configured using:
 'configure
 --prefix=/nix/store/5gi0vqvvvq0fljqdh7p6gx33js25b6zr-emacs-git-20230919.0
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 --with-cairo --with-native-compilation --with-tree-sitter
 --with-xwidgets'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM XWIDGETS
GTK3 ZLIB

Important settings:
  value of $LC_ALL: C.UTF-8
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Eshell

Minor modes in effect:
  eshell-prompt-mode: t
  eshell-pred-mode: t
  eshell-hist-mode: t
  eshell-cmpl-mode: t
  eshell-proc-mode: t
  eshell-arg-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils compile comp comp-cstr warnings
icons rx cl-seq cl-macs cl-extra help-mode em-unix em-term term
disp-table shell subr-x ehelp em-script em-prompt text-property-search
em-pred em-ls em-hist em-glob em-extpipe em-cmpl em-dirs em-basic
em-banner em-alias esh-mode esh-var eshell esh-cmd generator cl-loaddefs
cl-lib esh-ext esh-opt esh-proc esh-io byte-opt gv bytecomp byte-compile
esh-arg pcomplete comint ansi-osc ansi-color ring esh-module esh-groups
esh-util files-x rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads xwidget-internal dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
gtk x-toolkit xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 118397 18051) (symbols 48 9690 0) (strings 32 28757 1282)
 (string-bytes 1 981841) (vectors 16 22506)
 (vector-slots 8 436941 13182) (floats 8 33 36) (intervals 56 478 0)
 (buffers 992 13))

bug#66469: 30.0.50; Fields in eshell output regress behavior when yanking output into new input

[Jim Porter]

[-- Attachment #1: Type: text/plain, Size: 340 bytes --]

On 10/11/2023 11:01 AM, Jules Tamagnan wrote:
> Essentially the issue is that when killing eshell output and yanking it
> into a new input line one cannot navigate as expected. Thankfully this
> behavior is extremely easy to reproduce from `emacs -Q'.
Thanks for the report. Could you try the attached patch to verify that 
it fixes things?

[-- Attachment #2: 0001-Fix-behavior-of-Eshell-prompt-when-yanking-output-in.patch --]
[-- Type: text/plain, Size: 2989 bytes --]

From e1e0bce5b3a2fcdaf966e450d86a2fb4b22376a7 Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Wed, 11 Oct 2023 11:38:27 -0700
Subject: [PATCH] Fix behavior of Eshell prompt when yanking output into it

* lisp/eshell/esh-util.el (eshell--unmark-string-as-output): New
function...

* lisp/eshell/esh-mode.el (eshell-mode): ... use it.

* test/lisp/eshell/eshell-tests.el (eshell-test/yank-output): New test
(bug#66469).
---
 lisp/eshell/esh-mode.el          |  3 +++
 lisp/eshell/esh-util.el          |  8 ++++++++
 test/lisp/eshell/eshell-tests.el | 19 +++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index 2b560afb92c..9d2cd1e67eb 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -361,6 +361,9 @@ eshell-mode
   (setq-local eshell-last-output-end (point-marker))
   (setq-local eshell-last-output-block-begin (point))
 
+  (add-function :filter-return (local 'filter-buffer-substring-function)
+                #'eshell--unmark-string-as-output)
+
   (let ((modules-list (copy-sequence eshell-modules-list)))
     (setq-local eshell-modules-list modules-list))
 
diff --git a/lisp/eshell/esh-util.el b/lisp/eshell/esh-util.el
index 4c251a29269..ca2f775318a 100644
--- a/lisp/eshell/esh-util.el
+++ b/lisp/eshell/esh-util.el
@@ -234,6 +234,14 @@ eshell--mark-yanked-as-output
                 (eshell--mark-as-output start1 end1)))))
     (add-hook 'after-change-functions hook nil t)))
 
+(defun eshell--unmark-string-as-output (string)
+  "Unmark STRING as Eshell output."
+  (remove-list-of-text-properties
+   0 (length string)
+   '(rear-nonsticky front-sticky field insert-in-front-hooks)
+   string)
+  string)
+
 (defun eshell-find-delimiter
   (open close &optional bound reverse-p backslash-p)
   "From point, find the CLOSE delimiter corresponding to OPEN.
diff --git a/test/lisp/eshell/eshell-tests.el b/test/lisp/eshell/eshell-tests.el
index b02e5fca592..d2ef44ae507 100644
--- a/test/lisp/eshell/eshell-tests.el
+++ b/test/lisp/eshell/eshell-tests.el
@@ -195,6 +195,25 @@ eshell-test/get-old-input/run-output
    (eshell-send-input)
    (eshell-match-output "(\"hello\" \"there\")")))
 
+(ert-deftest eshell-test/yank-output ()
+  "Test that yanking a line of output into the next prompt works (bug#66469)."
+  (with-temp-eshell
+   (eshell-insert-command "echo hello")
+   ;; Go to the output and kill the line of text.
+   (forward-line -1)
+   (kill-line)
+   ;; Go to the last prompt and yank the previous output.
+   (goto-char (point-max))
+   (yank)
+   ;; Go to the beginning of the prompt and add some text.
+   (move-beginning-of-line 1)
+   (insert-and-inherit "echo ")
+   ;; Make sure when we go to the beginning of the line, we go to the
+   ;; right spot (before the "echo").
+   (move-end-of-line 1)
+   (move-beginning-of-line 1)
+   (should (looking-at "echo hello"))))
+
 (provide 'eshell-tests)
 
 ;;; eshell-tests.el ends here
-- 
2.25.1

bug#66469: 30.0.50; Fields in eshell output regress behavior when yanking output into new input

[Jim Porter]

Version: 30.1

On 10/11/2023 11:41 AM, Jim Porter wrote:
> Thanks for the report. Could you try the attached patch to verify that 
> it fixes things?

Confirmed off-list that this fixes things, so merging as 60f69528533 and 
closing this now.