bug#67259: 29.1; shell/term behavior different from other terminal emulators

bug#67259: 29.1; shell/term behavior different from other terminal emulators

[Matt]

I'm helping to maintain Org mode's shell interaction.  The following
was reported to the Org list.

Emacs M-x shell and M-x term executing in term-line-mode both produce
a different result than expected as compared to terminal emulators
like xterm or xfce4-terminal running Bash.  For non-Emacs terminal
emulators, "bar" is echoed.  M-x shell and M-x term executing in
term-line-mode do *not* echo "bar".

To reproduce with M-x shell:

1. emacs -Q
2. M-x shell
3. Copy the following two lines:

ssh localhost "echo foo>foo_file"
echo "bar"

4. Paste them into a non-Emacs terminal emulator (tested with xterm or xfce4-terminal) and press return
5. After entering the correct password, "bar" is echoed
6. Paste them into the buffer associated with M-x shell
7. After entering the correct password, "bar" is NOT echoed

To reproduce with M-x term:

1. emacs -Q
2. M-x term
3. C-c C-j to switch to term-line-mode
4. Copy the following two lines:

ssh localhost "echo foo>foo_file"
echo "bar"

5. Paste them into the buffer associated with M-x term
6. After entering the correct password, "bar" is NOT echoed

In GNU Emacs 29.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37,
cairo version 1.16.0)
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Guix System

Configured using:
 'configure
 CONFIG_SHELL=/gnu/store/v9p25q9l5nnaixkhpap5rnymmwbhf9rp-bash-minimal-5.1.16/bin/bash
 SHELL=/gnu/store/v9p25q9l5nnaixkhpap5rnymmwbhf9rp-bash-minimal-5.1.16/bin/bash
 --prefix=/gnu/store/xjrhyapm3zwgpmq5baz6m9kavz287jjj-emacs-29.1
 --enable-fast-install --with-cairo --with-modules
 --with-native-compilation=aot --disable-build-details'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM
GTK3 ZLIB

Important settings:
  value of $EMACSLOADPATH: /home/ahab/.guix-profile/share/emacs/site-lisp:/gnu/store/xjrhyapm3zwgpmq5baz6m9kavz287jjj-emacs-29.1/share/emacs/29.1/lisp
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
comp comp-cstr warnings icons subr-x rx cl-seq cl-macs gv cl-extra
help-mode bytecomp byte-compile cl-lib sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 79918 9356)
 (symbols 48 7446 0)
 (strings 32 20277 1733)
 (string-bytes 1 627375)
 (vectors 16 16608)
 (vector-slots 8 334511 19434)
 (floats 8 27 46)
 (intervals 56 235 0)
 (buffers 984 11))
   

bug#67259: 29.1; shell/term behavior different from other terminal emulators

[Ihor Radchenko]

Matt <matt@excalamus.com> writes:

> To reproduce with M-x shell:
>
> 1. emacs -Q
> 2. M-x shell
> 3. Copy the following two lines:
>
> ssh localhost "echo foo>foo_file"
> echo "bar"
>
> 4. Paste them into a non-Emacs terminal emulator (tested with xterm or xfce4-terminal) and press return
> 5. After entering the correct password, "bar" is echoed
> 6. Paste them into the buffer associated with M-x shell
> 7. After entering the correct password, "bar" is NOT echoed

Bump!

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#67259: 29.1; shell/term behavior different from other terminal emulators

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>> To reproduce with M-x shell:
>>
>> 1. emacs -Q
>> 2. M-x shell
>> 3. Copy the following two lines:
>>
>> ssh localhost "echo foo>foo_file"
>> echo "bar"
>>
>> 4. Paste them into a non-Emacs terminal emulator (tested with xterm or xfce4-terminal) and press return
>> 5. After entering the correct password, "bar" is echoed
>> 6. Paste them into the buffer associated with M-x shell
>> 7. After entering the correct password, "bar" is NOT echoed
>
> Bump!

And another bump. It has been over 3 months...

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#67259: 29.1; shell/term behavior different from other terminal emulators

[Max Nikulin]

On 18/11/2023 16:55, Matt wrote:
> I'm helping to maintain Org mode's shell interaction.  The following
> was reported to the Org list.

When I noticed a discussion of dash bug causing difference in comparison 
to bash for the provided input, I have realized that Org babel shell 
issue and this emacs bugs are unrelated. This one is specific to 
interactive sessions.

> Emacs M-x shell and M-x term executing in term-line-mode both produce
> a different result

Likely `shell' and `term' issues should be treated separately and this 
bug should be cloned.

>  than expected as compared to terminal emulators
> like xterm or xfce4-terminal running Bash.  For non-Emacs terminal
> emulators, "bar" is echoed.  M-x shell and M-x term executing in
> term-line-mode do *not* echo "bar".
> 
> To reproduce with M-x shell:
> 
> 1. emacs -Q
> 2. M-x shell
> 3. Copy the following two lines:
> 
> ssh localhost "echo foo>foo_file"
> echo "bar"

I think the following allows to eliminate ssh and to demonstrate the 
issue with purely local commands. (BASH on macOS might be too old)

     fakessh() {
         bash -c 'read -t 0.2 -r; printf "fakessh read: %s\n" "$REPLY"';
     }

     fakessh
     echo next

result:

     fakessh read: echo next

instead of

    fakessh read:
    next

likely expected by users because it is behavior of xterm&Co.

since `shell' relies on specific terminal type, to get the same result 
on pasting whole snippet at once, do in xterm or a similar application

     TERM=dumb bash

With default TERM modern terminal applications and shells have bracketed 
paste enabled. It is a security measure that allows users to review 
pasted commands before executing them

<https://security.stackexchange.com/questions/39118/how-can-i-protect-myself-from-this-kind-of-clipboard-abuse>

When bash is running outside of Emacs, it is possible to paste multiple 
commands into an editor, try C-x C-e in bash prompt.

In my opinion, `shell' either should be documented as unsafe with 
warnings in docstring and manual or some workaround should be 
implemented, e.g. saving paste text into a temporary file and executing 
them.

> To reproduce with M-x term:
> 
> 1. emacs -Q
> 2. M-x term
> 3. C-c C-j to switch to term-line-mode

Notice that `term' is not affected if `term-line-mode' is not activated.

> 4. Copy the following two lines:
> 
> ssh localhost "echo foo>foo_file"
> echo "bar"

Another option to break xterm that might be closer to `term-line-mode':
Create a custom inputrc file, e.g./tmp/disable-paste.inputrc

     $include /etc/inputrc
     set enable-bracketed-paste off

and run e.g. in xterm

      INPUTRC=/tmp/disable-paste.inputrc bash

The result pasting the bunch of commands at once is

     fakessh read: echo next

I think, input from `term-line-mode' should be treated more closely to 
bracketed paste.

As a final remark, be careful with scripts running commands like ssh or 
ffmpeg that optionally read stdin. Be explicit with your intentions and 
either do

     fakessh </dev/null
     echo next

or

     fakessh <<"EOF"
     echo next
     EOF

depending on desired result.
<https://mywiki.wooledge.org/BashFAQ/089>
"I'm reading a file line by line and running ssh or ffmpeg, only the 
first line gets processed!"