bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

I believe Bash 5.0 breaks TRAMP sudo in
tramp-sh-handle-file-name-all-completions.

Using Bash 5.0 and (find-file "/sudo::/") results in an error "Process
has died".  Setting tramp-verbose to 9 shows that it dies around here:

13:56:22.449785 tramp-maybe-send-script (5) # Sending script
‘tramp_perl_file_name_all_completions’...
13:56:22.449920 tramp-send-command (6) #
tramp_perl_file_name_all_completions () {
\perl -e '
opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
@files = readdir(d); closedir(d);
foreach $f (@files) {
 if (-d "$ARGV[0]/$f") {
  print "$f/\n";
 }
 else {
  print "$f\n";
 }
}
print "ok\n"
' "$1" 2>/dev/null
} 2>/dev/null; echo tramp_exit_status $?
13:56:25.428280 tramp-maybe-send-script (5) # Sending script
‘tramp_perl_file_name_all_completions’...failed

Note that I cannot reproduce this from emacs -Q because out of the box,
TRAMP sudo does not seem to call
tramp-sh-handle-file-name-all-completions and I don't have time right
now to figure out what setting enables that code path.

Switching my /bin/sh to Dash instead of Bash 5.0 immediately fixed the
problem, which strongly led me to suspect some subtle change in Bash 5.0
broke this.

In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-07-05 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 MODULES THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> I believe Bash 5.0 breaks TRAMP sudo in
> tramp-sh-handle-file-name-all-completions.
>
> Using Bash 5.0 and (find-file "/sudo::/") results in an error "Process
> has died".  Setting tramp-verbose to 9 shows that it dies around here:

Unfortunately, I have no bash 5.0 available. Could you pls eval the
following lines in a bash 5.0 terminal:

--8<---------------cut here---------------start------------->8---
tramp_perl_file_name_all_completions () {
\perl -e '
opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
@files = readdir(d); closedir(d);
foreach $f (@files) {
 if (-d "$ARGV[0]/$f") {
  print "$f/\n";
 }
 else {
  print "$f\n";
 }
}
print "ok\n"
' "$1" 2>/dev/null
}; echo tramp_exit_status $?
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

On Mon, Jan 28, 2019 at 8:19 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Allen Li <darkfeline@felesatra.moe> writes:
>
> Hi Allen,
>
> > I believe Bash 5.0 breaks TRAMP sudo in
> > tramp-sh-handle-file-name-all-completions.
> >
> > Using Bash 5.0 and (find-file "/sudo::/") results in an error "Process
> > has died".  Setting tramp-verbose to 9 shows that it dies around here:
>
> Unfortunately, I have no bash 5.0 available. Could you pls eval the
> following lines in a bash 5.0 terminal:
>
> --8<---------------cut here---------------start------------->8---
> tramp_perl_file_name_all_completions () {
> \perl -e '
> opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
> @files = readdir(d); closedir(d);
> foreach $f (@files) {
>  if (-d "$ARGV[0]/$f") {
>   print "$f/\n";
>  }
>  else {
>   print "$f\n";
>  }
> }
> print "ok\n"
> ' "$1" 2>/dev/null
> }; echo tramp_exit_status $?
> --8<---------------cut here---------------end--------------->8---

I get tramp_exit_status 0 as expected, but I think
tramp_perl_file_name_all_completions may be a red herring, since when
I tried using Edebug on tramp-sh-handle-file-name-all-completions, I
get the "Process has died" error after some time irrespective of my
code stepping.

>
> Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> I get tramp_exit_status 0 as expected, but I think
> tramp_perl_file_name_all_completions may be a red herring, since when
> I tried using Edebug on tramp-sh-handle-file-name-all-completions, I
> get the "Process has died" error after some time irrespective of my
> code stepping.

So could you pls call this shell function, once it has been declared:

# tramp_perl_file_name_all_completions `pwd`

Instead of `pwd` you could use any directory path.

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

On Mon, Jan 28, 2019 at 9:12 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Allen Li <darkfeline@felesatra.moe> writes:
>
> Hi Allen,
>
> > I get tramp_exit_status 0 as expected, but I think
> > tramp_perl_file_name_all_completions may be a red herring, since when
> > I tried using Edebug on tramp-sh-handle-file-name-all-completions, I
> > get the "Process has died" error after some time irrespective of my
> > code stepping.
>
> So could you pls call this shell function, once it has been declared:
>
> # tramp_perl_file_name_all_completions `pwd`

Running that didn't yield any insights. in both normal bash and bash
--posix.  I also tried sending it via stdin instead of an interactive
shell, all with no luck.

I tried monkey patching tramp-wait-for-regexp to see what the exit status is.

(defun tramp-wait-for-regexp (proc timeout regexp)
  "Wait for a REGEXP to appear from process PROC within TIMEOUT seconds.
Expects the output of PROC to be sent to the current buffer.  Returns
the string that matched, or nil.  Waits indefinitely if TIMEOUT is
nil."
  (with-current-buffer (process-buffer proc)
    (let ((found (tramp-check-for-regexp proc regexp)))
      (cond (timeout
     (with-timeout (timeout)
       (while (not found)
(tramp-accept-process-output proc 1)
(unless (process-live-p proc)
                   (message "@@@@@@@@@@@ DEAD %d" (process-exit-status proc))
                   (debug)
   (tramp-error-with-buffer
    nil proc 'file-error "Process has died"))
(setq found (tramp-check-for-regexp proc regexp)))))
    (t
     (while (not found)
       (tramp-accept-process-output proc 1)
       (unless (process-live-p proc)
                 (message "@@@@@@@@@@@ DEAD %d" (process-exit-status proc))
                 (debug)
(tramp-error-with-buffer
  nil proc 'file-error "Process has died"))
       (setq found (tramp-check-for-regexp proc regexp)))))
      (tramp-message proc 6 "\n%s" (buffer-string))
      (when (not found)
(if timeout
    (tramp-error
     proc 'file-error "[[Regexp `%s' not found in %d secs]]"
     regexp timeout)
  (tramp-error proc 'file-error "[[Regexp `%s' not found]]" regexp)))
      found)))

In my Messages buffer, I found

@@@@@@@@@@@ DEAD 139

A search tells me that this is SIGSEGV.

The corresponding backtrace:

  (if (process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died"))
  (while (not found) (tramp-accept-process-output proc 1) (if
(process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died")) (setq found (tramp-check-for-regexp
proc regexp)))
  (cond (timeout (let ((-with-timeout-value- (catch 'timeout (let*
((-with-timeout-timer- (run-with-timer timeout nil (function (lambda
nil (throw 'timeout 'timeout))))) (with-timeout-timers (cons
-with-timeout-timer- with-timeout-timers))) (unwind-protect (progn
(while (not found) (tramp-accept-process-output proc 1) (if
(process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died")) (setq found (tramp-check-for-regexp
proc regexp)))) (cancel-timer -with-timeout-timer-)))))) (if (eq
-with-timeout-value- 'timeout) (progn) -with-timeout-value-))) (t
(while (not found) (tramp-accept-process-output proc 1) (if
(process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died")) (setq found (tramp-check-for-regexp
proc regexp)))))
  (let ((found (tramp-check-for-regexp proc regexp))) (cond (timeout
(let ((-with-timeout-value- (catch 'timeout (let*
((-with-timeout-timer- (run-with-timer timeout nil (function (lambda
nil (throw 'timeout 'timeout))))) (with-timeout-timers (cons
-with-timeout-timer- with-timeout-timers))) (unwind-protect (progn
(while (not found) (tramp-accept-process-output proc 1) (if
(process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died")) (setq found (tramp-check-for-regexp
proc regexp)))) (cancel-timer -with-timeout-timer-)))))) (if (eq
-with-timeout-value- 'timeout) (progn) -with-timeout-value-))) (t
(while (not found) (tramp-accept-process-output proc 1) (if
(process-live-p proc) nil (message "@@@@@@@@@@@ DEAD %d"
(process-exit-status proc)) (debug) (tramp-error-with-buffer nil proc
'file-error "Process has died")) (setq found (tramp-check-for-regexp
proc regexp))))) (tramp-message proc 6 "\n%s" (buffer-string)) (if
(not found) (progn (if timeout (tramp-error proc 'file-error "[[Regexp
`%s' not found in %d secs]]" regexp timeout) (tramp-error proc
'file-error "[[Regexp `%s' not found]]" regexp)))) found)
  (save-current-buffer (set-buffer (process-buffer proc)) (let ((found
(tramp-check-for-regexp proc regexp))) (cond (timeout (let
((-with-timeout-value- (catch 'timeout (let* ((-with-timeout-timer-
(run-with-timer timeout nil (function (lambda nil (throw 'timeout
'timeout))))) (with-timeout-timers (cons -with-timeout-timer-
with-timeout-timers))) (unwind-protect (progn (while (not found)
(tramp-accept-process-output proc 1) (if (process-live-p proc) nil
(message "@@@@@@@@@@@ DEAD %d" (process-exit-status proc)) (debug)
(tramp-error-with-buffer nil proc 'file-error "Process has died"))
(setq found (tramp-check-for-regexp proc regexp)))) (cancel-timer
-with-timeout-timer-)))))) (if (eq -with-timeout-value- 'timeout)
(progn) -with-timeout-value-))) (t (while (not found)
(tramp-accept-process-output proc 1) (if (process-live-p proc) nil
(message "@@@@@@@@@@@ DEAD %d" (process-exit-status proc)) (debug)
(tramp-error-with-buffer nil proc 'file-error "Process has died"))
(setq found (tramp-check-for-regexp proc regexp))))) (tramp-message
proc 6 "\n%s" (buffer-string)) (if (not found) (progn (if timeout
(tramp-error proc 'file-error "[[Regexp `%s' not found in %d secs]]"
regexp timeout) (tramp-error proc 'file-error "[[Regexp `%s' not
found]]" regexp)))) found))
  tramp-wait-for-regexp(#<process *tramp/sudo root@cielnosurge*> nil
"\\(^\\|\0\\)[^#$\n]*///e7c116e27b816fdfc1b43f2c9e5a951b#\\$\\(\033[[0-9]+n\\)?\015?$")
  tramp-wait-for-output(#<process *tramp/sudo root@cielnosurge*>)
  tramp-send-command((tramp-file-name "sudo" #("root" 0 4
(tramp-default t)) nil "cielnosurge" nil "/" nil)
"tramp_perl_file_name_all_completions () {\n\\perl -e '\nopendir(d,
$ARGV[0]) || die(\"$ARGV[0]: $!\\nfail\\n\");\n@files = readdir(d);
closedir(d);\nforeach $f (@files) {\n if (-d \"$ARGV[0]/$f\") {\n
print \"$f/\\n\";\n }\n else {\n  print \"$f\\n\";\n }\n}\nprint
\"ok\\n\"\n' \"$1\" 2>/dev/null\n} 2>/dev/null; echo tramp_exit_status
$?")
  tramp-send-command-and-check((tramp-file-name "sudo" #("root" 0 4
(tramp-default t)) nil "cielnosurge" nil "/" nil)
"tramp_perl_file_name_all_completions () {\n\\perl -e '\nopendir(d,
$ARGV[0]) || die(\"$ARGV[0]: $!\\nfail\\n\");\n@files = readdir(d);
closedir(d);\nforeach $f (@files) {\n if (-d \"$ARGV[0]/$f\") {\n
print \"$f/\\n\";\n }\n else {\n  print \"$f\\n\";\n }\n}\nprint
\"ok\\n\"\n' \"$1\" 2>/dev/null\n}")
  tramp-barf-unless-okay((tramp-file-name "sudo" #("root" 0 4
(tramp-default t)) nil "cielnosurge" nil "/" nil)
"tramp_perl_file_name_all_completions () {\n\\perl -e '\nopendir(d,
$ARGV[0]) || die(\"$ARGV[0]: $!\\nfail\\n\");\n@files = readdir(d);
closedir(d);\nforeach $f (@files) {\n if (-d \"$ARGV[0]/$f\") {\n
print \"$f/\\n\";\n }\n else {\n  print \"$f\\n\";\n }\n}\nprint
\"ok\\n\"\n' \"$1\" 2>/dev/null\n}" "Script %s sending failed"
"tramp_perl_file_name_all_completions")
  tramp-maybe-send-script((tramp-file-name "sudo" #("root" 0 4
(tramp-default t)) nil "cielnosurge" nil "/" nil) "%s -e '\nopendir(d,
$ARGV[0]) || die(\"$ARGV[0]: $!\\nfail\\n\");\n@files = readdir(d);
closedir(d);\nforeach $f (@files) {\n if (-d \"$ARGV[0]/$f\") {\n
print \"$f/\\n\";\n }\n else {\n  print \"$f\\n\";\n }\n}\nprint
\"ok\\n\"\n' \"$1\" 2>/dev/null"
"tramp_perl_file_name_all_completions")
  tramp-sh-handle-file-name-all-completions(""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)))
  apply(tramp-sh-handle-file-name-all-completions (""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t))))
  tramp-sh-file-name-handler(file-name-all-completions ""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)))
  apply(tramp-sh-file-name-handler file-name-all-completions (""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t))))
  tramp-file-name-handler(file-name-all-completions ""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)))
  file-name-all-completions("" #("/sudo:root@cielnosurge:/" 6 10
(tramp-default t)))
  tramp-handle-file-name-completion("" #("/sudo:root@cielnosurge:/" 6
10 (tramp-default t)) file-exists-p)
  apply(tramp-handle-file-name-completion (""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)) file-exists-p))
  tramp-sh-file-name-handler(file-name-completion ""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)) file-exists-p)
  apply(tramp-sh-file-name-handler file-name-completion (""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)) file-exists-p))
  tramp-file-name-handler(file-name-completion ""
#("/sudo:root@cielnosurge:/" 6 10 (tramp-default t)) file-exists-p)

>
> Instead of `pwd` you could use any directory path.
>
> Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> I tried monkey patching tramp-wait-for-regexp to see what the exit status is.

> In my Messages buffer, I found
>
> @@@@@@@@@@@ DEAD 139
>
> A search tells me that this is SIGSEGV.
>
> The corresponding backtrace:

Thanks for the test. Unfortunately, it doesn't show why this happens.

Maybe we could get some more messages from the shell. Could you, pls,
apply the following patch:


[-- Attachment #2: Type: text/plain, Size: 412 bytes --]

diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
index c304fcb..46f467d 100644
--- a/lisp/tramp-sh.el
+++ b/lisp/tramp-sh.el
@@ -658,7 +658,7 @@ foreach $f (@files) {
  }
 }
 print \"ok\\n\"
-' \"$1\" 2>/dev/null"
+' \"$1\""
   "Perl script to produce output suitable for use with
 `file-name-all-completions' on the remote file system.  Escape
 sequence %s is replaced with name of Perl binary.  This string is

[-- Attachment #3: Type: text/plain, Size: 132 bytes --]


Recompile tramp-sh.el, and rerun the test with tramp-verbose 10. Your
patch isn't needed in this scenario.

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

On Tue, Jan 29, 2019 at 10:04 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Allen Li <darkfeline@felesatra.moe> writes:
>
> Hi Allen,
>
> > I tried monkey patching tramp-wait-for-regexp to see what the exit status is.
>
> > In my Messages buffer, I found
> >
> > @@@@@@@@@@@ DEAD 139
> >
> > A search tells me that this is SIGSEGV.
> >
> > The corresponding backtrace:
>
> Thanks for the test. Unfortunately, it doesn't show why this happens.
>
> Maybe we could get some more messages from the shell. Could you, pls,
> apply the following patch:
>
> diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
> index c304fcb..46f467d 100644
> --- a/lisp/tramp-sh.el
> +++ b/lisp/tramp-sh.el
> @@ -658,7 +658,7 @@ foreach $f (@files) {
>   }
>  }
>  print \"ok\\n\"
> -' \"$1\" 2>/dev/null"
> +' \"$1\""
>    "Perl script to produce output suitable for use with
>  `file-name-all-completions' on the remote file system.  Escape
>  sequence %s is replaced with name of Perl binary.  This string is
>
> Recompile tramp-sh.el, and rerun the test with tramp-verbose 10. Your
> patch isn't needed in this scenario.

I took the liberty of removing the 2>/dev/null in
tramp-send-command-and-check as well, but
I don't think there's a lot of info.

16:13:19.685224 tramp-send-string (10) # \perl -e 'print "Hello
";' ; echo tramp_exit_status $?
16:13:19.685305 tramp-get-connection-property (7) # process-buffer nil
16:13:19.685403 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.685479 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.686923 tramp-accept-process-output (10) # *tramp/sudo
root@cielnosurge* run t
Hello
16:13:19.687030 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.687115 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.687232 tramp-accept-process-output (10) # *tramp/sudo
root@cielnosurge* run t
Hello
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.687323 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.687399 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.687486 tramp-wait-for-regexp (6) #
Hello
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.687604 tramp-get-connection-property (7) # process-buffer nil
16:13:19.687708 tramp-get-connection-property (7) # perl-file-spec undef
16:13:19.687793 tramp-get-connection-property (7) # process-name nil
16:13:19.687877 tramp-get-connection-property (7) # process-name nil
16:13:19.687959 tramp-get-connection-property (7) # process-buffer nil
16:13:19.688044 tramp-get-connection-property (7) # last-cmd-time
(23638 12831 685113 887000)
16:13:19.688133 tramp-get-connection-property (7) # process-name nil
16:13:19.688215 tramp-get-connection-property (7) # remote-echo nil
16:13:19.688305 tramp-send-command (6) # \perl -e 'use File::Spec;' ;
echo tramp_exit_status $?
16:13:19.688429 tramp-get-connection-property (7) # process-name nil
16:13:19.688533 tramp-get-connection-property (7) # chunksize 0
16:13:19.688638 tramp-set-connection-property (7) # last-cmd-time
(23638 12831 688608 245000)
16:13:19.688755 tramp-send-string (10) # \perl -e 'use File::Spec;' ;
echo tramp_exit_status $?
16:13:19.688844 tramp-get-connection-property (7) # process-buffer nil
16:13:19.688948 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.689027 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.696952 tramp-accept-process-output (10) # *tramp/sudo
root@cielnosurge* run t
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.697071 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.697154 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.697241 tramp-wait-for-regexp (6) #
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.697361 tramp-get-connection-property (7) # process-buffer nil
16:13:19.697462 tramp-set-connection-property (7) # perl-file-spec t
16:13:19.697544 tramp-get-connection-property (7) # perl-cwd-realpath undef
16:13:19.697629 tramp-get-connection-property (7) # process-name nil
16:13:19.697713 tramp-get-connection-property (7) # process-name nil
16:13:19.697793 tramp-get-connection-property (7) # process-buffer nil
16:13:19.697879 tramp-get-connection-property (7) # last-cmd-time
(23638 12831 688608 245000)
16:13:19.697965 tramp-get-connection-property (7) # process-name nil
16:13:19.698047 tramp-get-connection-property (7) # remote-echo nil
16:13:19.698126 tramp-send-command (6) # \perl -e 'use Cwd
"realpath";' ; echo tramp_exit_status $?
16:13:19.698207 tramp-get-connection-property (7) # process-name nil
16:13:19.698287 tramp-get-connection-property (7) # chunksize 0
16:13:19.698372 tramp-set-connection-property (7) # last-cmd-time
(23638 12831 698346 135000)
16:13:19.698453 tramp-send-string (10) # \perl -e 'use Cwd
"realpath";' ; echo tramp_exit_status $?
16:13:19.698532 tramp-get-connection-property (7) # process-buffer nil
16:13:19.698627 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.698703 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.703296 tramp-accept-process-output (10) # *tramp/sudo
root@cielnosurge* run t
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.703413 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.703494 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.703578 tramp-wait-for-regexp (6) #
tramp_exit_status 0
///33fa052e8933ed0d43e05e92dd2ca7f3#$
16:13:19.703698 tramp-get-connection-property (7) # process-buffer nil
16:13:19.703799 tramp-set-connection-property (7) # perl-cwd-realpath t
16:13:19.703880 tramp-set-connection-property (7) # perl \perl
16:13:19.703962 tramp-get-connection-property (7) # process-name nil
16:13:19.704044 tramp-get-connection-property (7) # scripts nil
16:13:19.704123 tramp-maybe-send-script (5) # Sending script
‘tramp_perl_file_name_all_completions’...
16:13:19.704213 tramp-get-connection-property (7) # perl \perl
16:13:19.704290 tramp-get-connection-property (7) # perl \perl
16:13:19.704378 tramp-get-connection-property (7) # process-name nil
16:13:19.704457 tramp-get-connection-property (7) # process-name nil
16:13:19.704536 tramp-get-connection-property (7) # process-buffer nil
16:13:19.704619 tramp-get-connection-property (7) # last-cmd-time
(23638 12831 698346 135000)
16:13:19.704707 tramp-get-connection-property (7) # process-name nil
16:13:19.704790 tramp-get-connection-property (7) # remote-echo nil
16:13:19.704866 tramp-send-command (6) #
tramp_perl_file_name_all_completions () {
\perl -e '
opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
@files = readdir(d); closedir(d);
foreach $f (@files) {
 if (-d "$ARGV[0]/$f") {
  print "$f/\n";
 }
 else {
  print "$f\n";
 }
}
print "ok\n"
' "$1"
} ; echo tramp_exit_status $?
16:13:19.704957 tramp-get-connection-property (7) # process-name nil
16:13:19.705087 tramp-get-connection-property (7) # chunksize 0
16:13:19.705193 tramp-set-connection-property (7) # last-cmd-time
(23638 12831 705148 363000)
16:13:19.705294 tramp-send-string (10) #
tramp_perl_file_name_all_completions () {
\perl -e '
opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
@files = readdir(d); closedir(d);
foreach $f (@files) {
 if (-d "$ARGV[0]/$f") {
  print "$f/\n";
 }
 else {
  print "$f\n";
 }
}
print "ok\n"
' "$1"
} ; echo tramp_exit_status $?
16:13:19.705404 tramp-get-connection-property (7) # process-buffer nil
16:13:19.705518 tramp-get-connection-property (7) # check-remote-echo nil
16:13:19.705597 tramp-get-connection-property (7) # check-remote-echo nil
16:13:21.402934 tramp-maybe-send-script (5) # Sending script
‘tramp_perl_file_name_all_completions’...failed


>
> Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> I took the liberty of removing the 2>/dev/null in
> tramp-send-command-and-check as well, but
> I don't think there's a lot of info.

> 16:13:19.704866 tramp-send-command (6) #
> tramp_perl_file_name_all_completions () {
> \perl -e '
> opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
> @files = readdir(d); closedir(d);
> foreach $f (@files) {
>  if (-d "$ARGV[0]/$f") {
>   print "$f/\n";
>  }
>  else {
>   print "$f\n";
>  }
> }
> print "ok\n"
> ' "$1"
> } ; echo tramp_exit_status $?
> 16:13:19.704957 tramp-get-connection-property (7) # process-name nil
> 16:13:19.705087 tramp-get-connection-property (7) # chunksize 0
> 16:13:19.705193 tramp-set-connection-property (7) # last-cmd-time
> (23638 12831 705148 363000)
> 16:13:19.705294 tramp-send-string (10) #
> tramp_perl_file_name_all_completions () {
> \perl -e '
> opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
> @files = readdir(d); closedir(d);
> foreach $f (@files) {
>  if (-d "$ARGV[0]/$f") {
>   print "$f/\n";
>  }
>  else {
>   print "$f\n";
>  }
> }
> print "ok\n"
> ' "$1"
> } ; echo tramp_exit_status $?
> 16:13:19.705404 tramp-get-connection-property (7) # process-buffer nil
> 16:13:19.705518 tramp-get-connection-property (7) # check-remote-echo nil
> 16:13:19.705597 tramp-get-connection-property (7) # check-remote-echo nil
> 16:13:21.402934 tramp-maybe-send-script (5) # Sending script
> ‘tramp_perl_file_name_all_completions’...failed

Yes, there's even no trace 10 output about the result. Hmm, that's very
strange.

Could you, pls, send the contents of the *tramp/sudo root@cielnosurge* buffer?

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]

I finally traced the problem down to setting tramp-histfile-override to t.

Apparently, Bash 5.0 doesn't like HISTSIZE=0.  Changing it to
HISTSIZE=1 fixes the error.

Attached patch demonstrating workaround.

I couldn't reproduce it by hand however: env HISTFILE=''
HISTFILESIZE=0 HISTSIZE=0 bash </tmp/tmp.sh

Since it's a SIGSEGV, presumably the stars have to be perfectly
aligned for the error to happen.

Reproduce:

1. emacs -Q
2. (setq tramp-histfile-override t)
3. C-x C-f /sudo::/ TAB <enter sudo password> RET

Versions:

"GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-07-05"
GNU bash, version 5.0.0(1)-release (x86_64-pc-linux-gnu)


On Mon, Feb 4, 2019 at 11:53 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Allen Li <darkfeline@felesatra.moe> writes:
>
> Hi Allen,
>
> > I took the liberty of removing the 2>/dev/null in
> > tramp-send-command-and-check as well, but
> > I don't think there's a lot of info.
>
> > 16:13:19.704866 tramp-send-command (6) #
> > tramp_perl_file_name_all_completions () {
> > \perl -e '
> > opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
> > @files = readdir(d); closedir(d);
> > foreach $f (@files) {
> >  if (-d "$ARGV[0]/$f") {
> >   print "$f/\n";
> >  }
> >  else {
> >   print "$f\n";
> >  }
> > }
> > print "ok\n"
> > ' "$1"
> > } ; echo tramp_exit_status $?
> > 16:13:19.704957 tramp-get-connection-property (7) # process-name nil
> > 16:13:19.705087 tramp-get-connection-property (7) # chunksize 0
> > 16:13:19.705193 tramp-set-connection-property (7) # last-cmd-time
> > (23638 12831 705148 363000)
> > 16:13:19.705294 tramp-send-string (10) #
> > tramp_perl_file_name_all_completions () {
> > \perl -e '
> > opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");
> > @files = readdir(d); closedir(d);
> > foreach $f (@files) {
> >  if (-d "$ARGV[0]/$f") {
> >   print "$f/\n";
> >  }
> >  else {
> >   print "$f\n";
> >  }
> > }
> > print "ok\n"
> > ' "$1"
> > } ; echo tramp_exit_status $?
> > 16:13:19.705404 tramp-get-connection-property (7) # process-buffer nil
> > 16:13:19.705518 tramp-get-connection-property (7) # check-remote-echo nil
> > 16:13:19.705597 tramp-get-connection-property (7) # check-remote-echo nil
> > 16:13:21.402934 tramp-maybe-send-script (5) # Sending script
> > ‘tramp_perl_file_name_all_completions’...failed
>
> Yes, there's even no trace 10 output about the result. Hmm, that's very
> strange.
>
> Could you, pls, send the contents of the *tramp/sudo root@cielnosurge* buffer?
>
> Best regards, Michael.

[-- Attachment #2: 0001-Workaround-34192.patch --]
[-- Type: text/x-patch, Size: 768 bytes --]

From c46cfe1b1bf4a7f218eea15012f7ce9b04524c6c Mon Sep 17 00:00:00 2001
From: Allen Li <darkfeline@felesatra.moe>
Date: Wed, 6 Feb 2019 22:56:12 -0800
Subject: [PATCH] Workaround 34192

---
 lisp/net/tramp-sh.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
index 404fae9197..88968c83d6 100644
--- a/lisp/net/tramp-sh.el
+++ b/lisp/net/tramp-sh.el
@@ -4075,7 +4075,7 @@ tramp-open-shell
 		(format "HISTFILE=%s"
 			(tramp-shell-quote-argument tramp-histfile-override))
 	      (if tramp-histfile-override
-		  "HISTFILE='' HISTFILESIZE=0 HISTSIZE=0"
+		  "HISTFILE='' HISTFILESIZE=0 HISTSIZE=1"
 		""))
 	    (tramp-shell-quote-argument tramp-end-of-output)
 	    shell (or extra-args ""))
-- 
2.20.1

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> I finally traced the problem down to setting tramp-histfile-override to t.
>
> Apparently, Bash 5.0 doesn't like HISTSIZE=0.  Changing it to
> HISTSIZE=1 fixes the error.

Is this new in bash 5.0? I have installed GNU bash 4.4.19, and its
info manual says:

--8<---------------cut here---------------start------------->8---
'HISTSIZE'
     The maximum number of commands to remember on the history list.  If
     the value is 0, commands are not saved in the history list.
     Numeric values less than zero result in every command being saved
     on the history list (there is no limit).  The shell sets the
     default value to 500 after reading any startup files.
--8<---------------cut here---------------end--------------->8---

Hmm, there's exactly the same text in
https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz/bash-5.0/doc/bash.info

Before changing Tramp, I'd like to understand what's up.

> Reproduce:
>
> 1. emacs -Q
> 2. (setq tramp-histfile-override t)
> 3. C-x C-f /sudo::/ TAB <enter sudo password> RET

Do you have some special settings in your remote .profile,
.bash_profile, .bash_login, or .bashrc?  Could you test with moving
these files away?

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

On Thu, Feb 7, 2019 at 9:27 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Allen Li <darkfeline@felesatra.moe> writes:
>
> Hi Allen,
>
> > I finally traced the problem down to setting tramp-histfile-override to t.
> >
> > Apparently, Bash 5.0 doesn't like HISTSIZE=0.  Changing it to
> > HISTSIZE=1 fixes the error.
>
> Is this new in bash 5.0? I have installed GNU bash 4.4.19, and its
> info manual says:
>
> --8<---------------cut here---------------start------------->8---
> 'HISTSIZE'
>      The maximum number of commands to remember on the history list.  If
>      the value is 0, commands are not saved in the history list.
>      Numeric values less than zero result in every command being saved
>      on the history list (there is no limit).  The shell sets the
>      default value to 500 after reading any startup files.
> --8<---------------cut here---------------end--------------->8---
>
> Hmm, there's exactly the same text in
> https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz/bash-5.0/doc/bash.info
>
> Before changing Tramp, I'd like to understand what's up.

To be clear, I'm not suggesting any change in Emacs to fix this.  All
signs point to this being an obscure bug in Bash.  I may take this
upstream (but I don't think it's worth the effort as there's a very
simple workaround of using dash for /bin/sh).

>
> > Reproduce:
> >
> > 1. emacs -Q
> > 2. (setq tramp-histfile-override t)
> > 3. C-x C-f /sudo::/ TAB <enter sudo password> RET
>
> Do you have some special settings in your remote .profile,
> .bash_profile, .bash_login, or .bashrc?  Could you test with moving
> these files away?

I don't think so, as TRAMP starts /bin/sh without anything that should
load any of those files.  I may try at some point, but again since
there's a simple workaround I'm hitting the limit of how much effort I
want to put into this.

Perhaps more people will run into this bug once Bash 5 pushes out to
the larger distros, or the bug will get fixed inadvertently.  I don't
have any opinions on whether to keep this bug open or close in the
meantime.

Thanks for your help so far.

>
> Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

>> Hi Allen,

Hi Allen,

>> > Reproduce:
>> >
>> > 1. emacs -Q
>> > 2. (setq tramp-histfile-override t)
>> > 3. C-x C-f /sudo::/ TAB <enter sudo password> RET
>>
>> Do you have some special settings in your remote .profile,
>> .bash_profile, .bash_login, or .bashrc?  Could you test with moving
>> these files away?
>
> I don't think so, as TRAMP starts /bin/sh without anything that should
> load any of those files.  I may try at some point, but again since
> there's a simple workaround I'm hitting the limit of how much effort I
> want to put into this.

I believe bash loads (some of) them automatically. Therefore, bash
knows the parameters --noprofile and --norc.

It would be very helpful for investigation, if you could move these
remote files away temporarily, and check whether the error persists.

> Perhaps more people will run into this bug once Bash 5 pushes out to
> the larger distros, or the bug will get fixed inadvertently.  I don't
> have any opinions on whether to keep this bug open or close in the
> meantime.

If it is reproducible with bash 5.0, I would like to add a not to the
Tramp manuals. At least.

Thanks for your support in testing.

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Allen Li]

On Fri, Feb 8, 2019 at 10:44 AM Michael Albinus <michael.albinus@gmx.de> wrote:
> It would be very helpful for investigation, if you could move these
> remote files away temporarily, and check whether the error persists.
>
> > Perhaps more people will run into this bug once Bash 5 pushes out to
> > the larger distros, or the bug will get fixed inadvertently.  I don't
> > have any opinions on whether to keep this bug open or close in the
> > meantime.
>
> If it is reproducible with bash 5.0, I would like to add a not to the
> Tramp manuals. At least.
>
> Thanks for your support in testing.

Reproducing in a container, anyone else on Arch Linux should be able
to follow this.

1. mkdir ~/container
2. sudo pacstrap -i -c ~/container base --ignore linux --ignore
linux-firmware sudo emacs
3. sudo systemd-nspawn -b -D ~/container
4. Login with root
5. useradd -m bob
6. passwd bob
bob
7. visudo
bob ALL=(ALL) ALL
8. su bob
9. cd ~

1. emacs -Q
2. (setq tramp-histfile-override t)
3. C-x C-f /sudo::/ TAB bob RET

(Exit container with C-] C-] C-])

Versions as of now:
"GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-07-05"
GNU bash, version 5.0.0(1)-release (x86_64-pc-linux-gnu)

>
> Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> Reproducing in a container, anyone else on Arch Linux should be able
> to follow this.

Thanks. I've downloaded bash 5.0 from the ftp server, compiled and
installed it. In a bash shell, I could reproduce the problem, playing
the commands Tramp has emitted.

Well, I wrote a bash bug report. Let's see what they'll answer.

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Allen,

>> Reproducing in a container, anyone else on Arch Linux should be able
>> to follow this.
>
> Thanks. I've downloaded bash 5.0 from the ftp server, compiled and
> installed it. In a bash shell, I could reproduce the problem, playing
> the commands Tramp has emitted.
>
> Well, I wrote a bash bug report. Let's see what they'll answer.

I've got an answer from the bash maintainer. He confirms the problem,
and he has sent also a patch. I've tested the patch; it fixes the
problem in my environment with bash 5.0.

Will forward that message. So we have to wait for a new bash 5.0.x
version which fixes this. In the meantime, I'll write a warning in the
Tramp doc.

Thanks again for your testing, and best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

Emacs 26.2

Hi Allen,

> I've got an answer from the bash maintainer. He confirms the problem,
> and he has sent also a patch. I've tested the patch; it fixes the
> problem in my environment with bash 5.0.
>
> Will forward that message. So we have to wait for a new bash 5.0.x
> version which fixes this. In the meantime, I'll write a warning in the
> Tramp doc.

I've documented this in the Tramp manual, pushed to the emacs-26
branch. Since there's nothing left on Emacs side we could do, I'm
closing the bug.

Best regards, Michael.

bug#34192: 26.1; Bash 5.0 breaks TRAMP sudo

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 191 bytes --]


-------------------- Start of forwarded message --------------------
Subject: Re: bash 5.0 dies with HISTSIZE=0
From: Chet Ramey <chet.ramey@case.edu>
Date: Wed, 20 Feb 2019 16:23:23 -0500


[-- Attachment #2.1: Type: text/plain, Size: 1052 bytes --]

On 2/20/19 8:10 AM, Michael Albinus wrote:

> Bash Version: 5.0
> Patch Level: 0
> Release Status: release
> 
> Description:
> This is an error report from an Emacs/Tramp user. With bash 5.0, bash
> 5.0 dies when HISTSIZE is set to 0. Setting it to 1 seems to work. Using
> bash 4.4 works as well. <See https://debbugs.gnu.org/34192> for details.
> 
> Repeat-By:
> The following is an extract from what Tramp does inside Emacs. I could
> reproduce it in a a shell:
> 
> 
> # env TERM='dumb' INSIDE_EMACS='27.0.50,tramp:2.4.2-pre' ENV='' HISTFILE='' HISTFILESIZE=0 HISTSIZE=0 PROMPT_COMMAND='' PS1=\#\$\  PS2='' PS3='' /home/albinus/src/bash-5.0/bash -norc -noprofile
> #$ tramp_perl_file_name_all_completions () {
> \perl -e '
> opendir(d, $ARGV[0]) || die("$ARGV[0]: $!\nfail\n");

Thanks for the report. Try the attached patch and see if that fixes the
crash.

Chet


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/

[-- Attachment #2.2: history-zero-length.patch --]
[-- Type: text/plain, Size: 1704 bytes --]

*** ../bash-5.0-patched/bashhist.c	2018-07-05 22:41:14.000000000 -0400
--- bashhist.c	2019-02-20 16:20:04.000000000 -0500
***************
*** 561,573 ****
    if (!history_expansion_inhibited && history_expansion && history_expansion_p (line))
      {
        /* If we are expanding the second or later line of a multi-line
  	 command, decrease history_length so references to history expansions
  	 in these lines refer to the previous history entry and not the
  	 current command. */
        if (history_length > 0 && command_oriented_history && current_command_first_line_saved && current_command_line_count > 1)
          history_length--;
        expanded = history_expand (line, &history_value);
        if (history_length >= 0 && command_oriented_history && current_command_first_line_saved && current_command_line_count > 1)
!         history_length++;
  
        if (expanded)
--- 561,576 ----
    if (!history_expansion_inhibited && history_expansion && history_expansion_p (line))
      {
+       int old_len;
+ 
        /* If we are expanding the second or later line of a multi-line
  	 command, decrease history_length so references to history expansions
  	 in these lines refer to the previous history entry and not the
  	 current command. */
+       old_len = history_length;
        if (history_length > 0 && command_oriented_history && current_command_first_line_saved && current_command_line_count > 1)
          history_length--;
        expanded = history_expand (line, &history_value);
        if (history_length >= 0 && command_oriented_history && current_command_first_line_saved && current_command_line_count > 1)
!         history_length = old_len;
  
        if (expanded)

[-- Attachment #3: Type: text/plain, Size: 67 bytes --]

-------------------- End of forwarded message --------------------