bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

[-- Attachment #1: Type: text/plain, Size: 1817 bytes --]

One oddity of Eshell is that even when you're connected to a remote host 
(usually you just "cd" into a remote directory using Tramp syntax), 
absolute file names are still on your *local* host when running any Lisp 
commands. However, running *external* commands (programs on the remote 
host), absolute file names are on that remote host.

When you think about how it's implemented, this makes sense: Lisp 
commands always run in the local Emacs process, but external programs 
run on the remote. So naturally, "absolute" file names are relative to a 
different host in either case. This wouldn't be so bad except that it's 
not always obvious when you're running a Lisp command or not. Eshell 
provides Lisp implementations of some common commands, like "cat", but 
it also transparently falls back to the external program if it doesn't 
understand some option. This results in it being pretty hard to tell 
what's going to happen when you run a command.

There's an "elecslash" module for Eshell that helps with this, but it 
can't tell when you have a Lisp command that will actually fallback to 
the external program when you run it.

Instead, the attached patch provides a new way to handle this: if you 
enable 'eshell-connection-local-file-names', then "normal" absolute file 
names like "/foo/bar" or "~/user" are evaluated relative to the current 
remote connection (if any). Eshell does this by expanding the file name 
to a full remote name like "/ssh:remote:/foo/bar". If these strings get 
sent to an external program, Eshell will unexpand them back to a 
host-local name (it will make sure that the remote host is correct, too).

You can also keep Eshell from performing this expansion on a 
case-by-case basis by quoting the file name (like "this", 'this', or 
/:this) or escaping the leading / or ~.

[-- Attachment #2: 0001-Mark-all-backslash-escaped-characters-in-Eshell-as-e.patch --]
[-- Type: text/plain, Size: 8420 bytes --]

From dd5a423c1d9f2c033e44abe5dae3f5a3a312802b Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Sat, 2 Sep 2023 22:29:22 -0700
Subject: [PATCH 1/2] Mark all backslash-escaped characters in Eshell as
 'escaped'

* lisp/eshell/esh-arg.el (eshell-parse-backslash): Mark all
backslash-escaped characters with the 'escaped' property, even if
they're non-special.

* test/lisp/eshell/esh-arg-tests.el
(esh-arg-test/escape/backslash-nonspecial)
(esh-arg-test/escape/backslash-nonspecial-unicode)
(esh-arg-test/escape/backslash-special)
(esh-arg-test/escape/backslash-newline)
(esh-arg-test/escape/backslash-newline-conditional)
(esh-arg-test/escape-quoted/backslash-nonspecial)
(esh-arg-test/escape-quoted/backslash-special)
(esh-arg-test/escape-quoted/backslash-newline): Rename tests, and
check string properties.
(esh-arg-test/escape-quoted/basic)
(esh-arg-test/escape-single-quoted/basic)
(esh-arg-test/escape-single-quoted/single-quote): New tests.
---
 lisp/eshell/esh-arg.el            |   7 +-
 test/lisp/eshell/esh-arg-tests.el | 124 +++++++++++++++++-------------
 2 files changed, 74 insertions(+), 57 deletions(-)

diff --git a/lisp/eshell/esh-arg.el b/lisp/eshell/esh-arg.el
index 78cf28d785a..b62f0893e89 100644
--- a/lisp/eshell/esh-arg.el
+++ b/lisp/eshell/esh-arg.el
@@ -477,15 +477,14 @@ eshell-parse-backslash
        ;; multiple lines.
        ((eq (char-before) ?\n)
         'eshell-empty-token)
-       ((memq (char-before) special-chars)
-        (list 'eshell-escape-arg (char-to-string (char-before))))
        ;; If the char is in a quote, backslash only has special
        ;; meaning if it is escaping a special char.  Otherwise, the
        ;; result is the literal string "\c".
-       (eshell-current-quoted
+       ((and eshell-current-quoted
+             (not (memq (char-before) special-chars)))
         (concat "\\" (char-to-string (char-before))))
        (t
-        (char-to-string (char-before)))))))
+        (list 'eshell-escape-arg (char-to-string (char-before))))))))
 
 (defun eshell-parse-literal-quote ()
   "Parse a literally quoted string.  Nothing has special meaning!"
diff --git a/test/lisp/eshell/esh-arg-tests.el b/test/lisp/eshell/esh-arg-tests.el
index b748c5ab4c0..8c139cee589 100644
--- a/test/lisp/eshell/esh-arg-tests.el
+++ b/test/lisp/eshell/esh-arg-tests.el
@@ -36,42 +36,40 @@ eshell-test-value
 
 ;;; Tests:
 
-(ert-deftest esh-arg-test/escape/nonspecial ()
-  "Test that \"\\c\" and \"c\" are equivalent when \"c\" is not a
-special character."
-  (with-temp-eshell
-   (eshell-match-command-output "echo he\\llo"
-                                "hello\n")))
-
-(ert-deftest esh-arg-test/escape/nonspecial-unicode ()
-  "Test that \"\\c\" and \"c\" are equivalent when \"c\" is a
-unicode character (unicode characters are nonspecial by
-definition)."
-  (with-temp-eshell
-   (eshell-match-command-output "echo Vid\\éos"
-                                "Vidéos\n")))
-
-(ert-deftest esh-arg-test/escape/special ()
-  "Test that the backslash is not preserved for escaped special
-chars."
-  (with-temp-eshell
-   (eshell-match-command-output "echo he\\\\llo"
-                                ;; Backslashes are doubled for regexp.
-                                "he\\\\llo\n")))
-
-(ert-deftest esh-arg-test/escape/newline ()
-  "Test that an escaped newline is equivalent to the empty string."
-  (with-temp-eshell
-   (eshell-match-command-output "echo hi\\\nthere"
-                                "hithere\n")))
-
-(ert-deftest esh-arg-test/escape/trailing-newline ()
-  "Test that an escaped newline is equivalent to the empty string."
+(ert-deftest esh-arg-test/escape/backslash-nonspecial ()
+  "Test that \"\\c\" expands to \"c\" when \"c\" is not a special character.
+It should mark \"c\" as being escaped, though."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo he\\llo")
+           #("hello" 2 3 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape/backslash-nonspecial-unicode ()
+  "Test that \"\\c\" expands to \"c\" when \"c\" is a Unicode character.
+Unicode characters are nonspecial by definition.  As above, this
+would mark \"c\" as escaped."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo Vid\\éos")
+           #("Vidéos" 3 4 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape/backslash-special ()
+  "Test that the backslash is removed for escaped special characters."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo he\\\\llo")
+           #("he\\llo" 2 3 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape/backslash-newline ()
+  "Test that an escaped newline expands to the empty string."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo hi\\\nthere")
+           "hithere")))
+
+(ert-deftest esh-arg-test/escape/trailing-backslash-newline ()
+  "Test that an escaped newline expands to the empty string."
   (with-temp-eshell
    (eshell-match-command-output "echo hi\\\n"
                                 "hi\n")))
 
-(ert-deftest esh-arg-test/escape/newline-conditional ()
+(ert-deftest esh-arg-test/escape/backslash-newline-conditional ()
   "Test invocation of an if/else statement using line continuations."
   (let ((eshell-test-value t))
     (eshell-command-result-equal
@@ -82,27 +80,47 @@ esh-arg-test/escape/newline-conditional
      "if $eshell-test-value \\\n{echo yes} \\\n{echo no}"
      "no")))
 
-(ert-deftest esh-arg-test/escape-quoted/nonspecial ()
-  "Test that the backslash is preserved for escaped nonspecial
-chars."
-  (with-temp-eshell
-   (eshell-match-command-output "echo \"h\\i\""
-                                ;; Backslashes are doubled for regexp.
-                                "h\\\\i\n")))
-
-(ert-deftest esh-arg-test/escape-quoted/special ()
-  "Test that the backslash is not preserved for escaped special
-chars."
-  (with-temp-eshell
-   (eshell-match-command-output "echo \"\\\"hi\\\\\""
-                                ;; Backslashes are doubled for regexp.
-                                "\\\"hi\\\\\n")))
-
-(ert-deftest esh-arg-test/escape-quoted/newline ()
-  "Test that an escaped newline is equivalent to the empty string."
-  (with-temp-eshell
-   (eshell-match-command-output "echo \"hi\\\nthere\""
-                                "hithere\n")))
+(ert-deftest esh-arg-test/escape-quoted/basic ()
+  "Test that double-quoted text is marked as escaped."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo \"hi\"")
+           #("hi" 0 2 (escaped t))))
+  (should (equal-including-properties
+           (eshell-test-command-result "echo \"hi\"there")
+           #("hithere" 0 2 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape-quoted/backslash-nonspecial ()
+  "Test that in double-quotes, \"\\\" is preserved before nonspecial chars."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo \"h\\i\"")
+           #("h\\i" 0 3 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape-quoted/backslash-special ()
+  "Test that in double-quotes, \"\\\" is not preserved before special chars."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo \"\\\"hi\\\\\"")
+           #("\"hi\\" 0 4 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape-quoted/backslash-newline ()
+  "Test that in double-quotes, an escaped newline expands to the empty string."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo \"hi\\\nthere\"")
+           #("hithere" 0 7 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape-single-quoted/basic ()
+  "Test that single-quoted text is marked as escaped."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo 'hi'")
+           #("hi" 0 2 (escaped t))))
+  (should (equal-including-properties
+           (eshell-test-command-result "echo 'hi'there")
+           #("hithere" 0 2 (escaped t)))))
+
+(ert-deftest esh-arg-test/escape-single-quoted/single-quote ()
+  "Test that a doubled single-quote inside single-quotes is one single-quote."
+  (should (equal-including-properties
+           (eshell-test-command-result "echo 'it''s me'")
+           #("it's me" 0 7 (escaped t)))))
 
 (ert-deftest esh-arg-test/special-reference/default ()
   "Test that \"#<buf>\" refers to the buffer \"buf\"."
-- 
2.25.1


[-- Attachment #3: 0002-Let-Eshell-expand-absolute-file-names-via-the-curren.patch --]
[-- Type: text/plain, Size: 17997 bytes --]

From 9675e111f774c6e8748cf7226d970d16dbfa7f1f Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Sat, 9 Sep 2023 15:41:53 -0700
Subject: [PATCH 2/2] Let Eshell expand absolute file names via the current
 remote connection

* lisp/eshell/em-dirs.el (eshell-connection-local-file-names): New
option.
(eshell-parse-absolute-file): New function...
(eshell-dirs-initialize): ... use it.
(eshell-expand-absolute-file): New function...
(eshell-parse-user-reference): ... use it.
(eshell-file-external-name): New function.

* lisp/eshell/esh-ext.el (eshell--file-external-names): New function...
(eshell-external-command): ... use it.

* test/lisp/eshell/em-dirs-tests.el (tramp): Require.
(em-dirs-test/expand-user-reference/remote)
(em-dirs-test/expand-absolute-file/local)
(em-dirs-test/expand-absolute-file/remote)
(em-dirs-test/expand-absolute-file/quoted): New tests.

* test/lisp/eshell/esh-ext-tests.el (em-dirs): Require.
(em-ext-test/unexpand-remote-file/local)
(em-ext-test/unexpand-absolute-file/remote): New tests.

* test/lisp/eshell/eshell-tests-helpers.el
(eshell-command-result-equal): New argument IGNORE-ERRORS.

* doc/misc/eshell.texi (Remote Access): Document
'eshell-connection-local-file-names'.

* etc/NEWS: Announce this change.
---
 doc/misc/eshell.texi                     | 37 ++++++++++++--
 etc/NEWS                                 | 25 +++++----
 lisp/eshell/em-dirs.el                   | 64 ++++++++++++++++++++++++
 lisp/eshell/esh-ext.el                   | 16 +++++-
 test/lisp/eshell/em-dirs-tests.el        | 48 ++++++++++++++++++
 test/lisp/eshell/esh-ext-tests.el        | 43 ++++++++++++++++
 test/lisp/eshell/eshell-tests-helpers.el | 15 +++---
 7 files changed, 229 insertions(+), 19 deletions(-)

diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index 30c85da795b..bbd570923f7 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -1527,9 +1527,40 @@ Remote Access
 built-in commands or Lisp functions from a remote directory, you must
 be careful about specifying absolute file names: @samp{cat
 /var/output.log} will always print the contents of your @emph{local}
-@file{/var/output.log}, even from a remote directory.  If you find
-this behavior annoying, you can enable the optional electric forward
-slash module (@pxref{Electric forward slash}).
+@file{/var/output.log}, even from a remote directory.
+
+@vindex eshell-connection-local-file-names
+If you find this behavior annoying, you can customize Eshell to help you
+refer to remote files when in a remote directory.  By setting
+@code{eshell-connection-local-file-names} to @code{t}, you can make
+Eshell expand absolute file names like @file{/var/output.log} or
+@file{~user} within the remote connection assocated with the current
+directory:
+
+@example
+/ssh:user@@remote:~ $ concat "log is at " /var/output.log
+log is at /ssh:user@@remote:/var/output.log
+@end example
+
+When using this option, you can avoid this expansion by
+quoting or escaping the initial @samp{/} or @samp{~}, or by explicitly
+typing the connection.  To explicitly refer to a @emph{local} file, you
+can quote the file name:
+
+@example
+/ssh:user@@remote:/etc $ cd ~
+/ssh:user@@remote:~ $ cd /:~
+~ $
+@end example
+
+When this option is enabled, Eshell will also ``unexpand'' any remote
+file names (e.g.@: @samp{/ssh:user@@remote:file.txt}) or before sending
+them to external commands, so those commands only see @file{file.txt} as
+they expect.
+
+If you instead prefer to keep the default logic but make it easier to
+type the full remote file names, you can enable the optional electric
+forward slash module (@pxref{Electric forward slash}).
 
 @vindex eshell-explicit-remote-commands
 When running commands, you can also make them explicitly remote by
diff --git a/etc/NEWS b/etc/NEWS
index 456f9b8f8b8..4c4b807b4f3 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -854,6 +854,22 @@ using this new option.  (Or set 'display-buffer-alist' directly.)
 
 ** Eshell
 
++++
+*** Eshell commands can now be explicitly-remote (or local).
+By prefixing a command name in Eshell with a remote identifier, like
+"/ssh:user@remote:whoami", you can now run commands on a particular
+host no matter your current directory.  Likewise, you can run a
+command on your local system no matter your current directory via
+"/:whoami".  For more information, see the "(eshell) Remote Access"
+node in the Eshell manual.
+
++++
+*** New option to expand absolute file names via the current remote connection.
+By setting 'eshell-connection-local-file-names' to a non-nil value,
+Eshell will expand absolute file names like "/foo/bar" or "~/user"
+within the current remote connection.  See "(eshell) Remote Access" for
+more details.
+
 +++
 *** New builtin Eshell command 'compile'.
 This command runs another command, sending its output to a compilation
@@ -909,15 +925,6 @@ or get a sublist of elements 2 through 4 with '$my-list[2..5]'.  For
 more information, see the "(eshell) Dollars Expansion" node in the
 Eshell manual.
 
-+++
-*** Eshell commands can now be explicitly-remote (or local).
-By prefixing a command name in Eshell with a remote identifier, like
-"/ssh:user@remote:whoami", you can now run commands on a particular
-host no matter your current directory.  Likewise, you can run a
-command on your local system no matter your current directory via
-"/:whoami".  For more information, see the "(eshell) Remote Access"
-node in the Eshell manual.
-
 +++
 *** Eshell's '$UID' and '$GID' variables are now connection-aware.
 Now, when expanding '$UID' or '$GID' in a remote directory, the value
diff --git a/lisp/eshell/em-dirs.el b/lisp/eshell/em-dirs.el
index 07063afc286..8642dee1855 100644
--- a/lisp/eshell/em-dirs.el
+++ b/lisp/eshell/em-dirs.el
@@ -65,6 +65,18 @@ eshell-dirs-load-hook
   :version "24.1"			; removed eshell-dirs-initialize
   :type 'hook)
 
+(defcustom eshell-connection-local-file-names nil
+  "If non-nil, expand absolute file name within the current remote connection.
+When this option is enabled, Eshell expands absolute file names like
+\"/foo/bar\" or \"~user\" within the remote connection associated with
+the current working directory.
+
+Additionally, when this option is enabled, Eshell will
+\"unexpand\" remote or quoted file names before passing them to
+external commands (see `eshell-file-external-name')."
+  :version "30.1"
+  :type 'boolean)
+
 (defcustom eshell-pwd-convert-function (if (eshell-under-windows-p)
 					   #'expand-file-name
 					 #'identity)
@@ -205,6 +217,9 @@ eshell-dirs-initialize
 
   (add-hook 'eshell-parse-argument-hook
 	    #'eshell-parse-user-reference nil t)
+  (when eshell-connection-local-file-names
+    (add-hook 'eshell-parse-argument-hook
+	      #'eshell-parse-absolute-file nil t))
   (if (eshell-under-windows-p)
       (add-hook 'eshell-parse-argument-hook
 		#'eshell-parse-drive-letter nil t))
@@ -267,9 +282,58 @@ eshell-parse-user-reference
     ;; Apply this modifier fairly early so it happens before things
     ;; like glob expansion.
     (add-hook 'eshell-current-modifiers #'eshell-expand-user-reference -50)
+    (when eshell-connection-local-file-names
+      (add-hook 'eshell-current-modifiers #'eshell-expand-absolute-file -40))
+    (forward-char)
+    (char-to-string (char-before))))
+
+(defun eshell-expand-absolute-file (file)
+  "Expand an absolute FILE like \"/foo/bar\" to be on the current remote host.
+This treats quoted file names like \"/:/foo/bar\" literally."
+  (if (or (file-name-quoted-p file t)
+          (file-remote-p file)
+          ;; Don't expand virtual targets; otherwise, we'd fail to
+          ;; find them later in `eshell-get-target'.
+          (assoc file eshell-virtual-targets))
+      file
+    (concat (file-remote-p default-directory) file)))
+
+(defun eshell-parse-absolute-file ()
+  "An argument beginning with / is a filename to be expanded."
+  (when (and (not eshell-current-argument)
+             (not eshell-current-quoted)
+             (eq (char-after) ?/))
+    (add-hook 'eshell-current-modifiers #'eshell-expand-absolute-file -50)
     (forward-char)
     (char-to-string (char-before))))
 
+(defun eshell-file-external-name (file)
+  "Simplify FILE so that external commands can understand it.
+This returns the unquoted local part of the file name.  If the
+remote connection associated with FILE doesn't match
+`default-directory', signal an error.
+
+When FILE doesn't start with a \"/\", or the leading \"/\" is
+escaped, just return FILE as-is."
+  (if (or (not (eq (aref file 0) ?/))
+          (get-text-property 0 'escaped file))
+      file
+    ;; Check that the file and cwd connections are the same.
+    (let ((file-connection (file-remote-p file))
+          (cwd-connection (file-remote-p default-directory)))
+      (cond
+       ((equal file-connection cwd-connection)) ; It's ok!
+       ((not file-connection)
+        (error "`%s' is local, but current directory is remote (`%s')"
+               file cwd-connection))
+       ((not cwd-connection)
+        (error "`%s' is remote, but current directory is local"
+               file))
+       (t
+        (error "`%s' does not match current connection `%s'"
+               file cwd-connection))))
+    (file-name-unquote (file-local-name file) t)))
+
 (defun eshell-parse-drive-letter ()
   "An argument beginning with X:[^/] is a drive letter reference."
   (when (and (not eshell-current-argument)
diff --git a/lisp/eshell/esh-ext.el b/lisp/eshell/esh-ext.el
index 44861c222b8..635114bcea4 100644
--- a/lisp/eshell/esh-ext.el
+++ b/lisp/eshell/esh-ext.el
@@ -229,9 +229,23 @@ eshell-remote-command
       (error "%s: not a remote command" command))
     (eshell-external-command command-localname args)))
 
+(defun eshell--file-external-names (args)
+  "Simplify ARGS so that external commands can understand any file names.
+If `eshell-connection-local-file-names' is nil or the `eshell-dirs'
+module is disabled, just return ARGS unchanged."
+  (declare-function eshell-file-external-name "em-dirs" (file))
+  (defvar eshell-connection-local-file-names)
+  (if (and (eshell-using-module 'eshell-dirs)
+           (bound-and-true-p eshell-connection-local-file-names))
+      (mapcar (lambda (arg)
+                (if (stringp arg) (eshell-file-external-name arg) arg))
+              args)
+    args))
+
 (defun eshell-external-command (command args)
   "Insert output from an external COMMAND, using ARGS."
-  (setq args (eshell-stringify-list (flatten-tree args)))
+  (setq args (eshell-stringify-list
+              (eshell--file-external-names (flatten-tree args))))
   (let ((interp (eshell-find-interpreter
 		 command
 		 args
diff --git a/test/lisp/eshell/em-dirs-tests.el b/test/lisp/eshell/em-dirs-tests.el
index 9789e519f4c..328cec2994d 100644
--- a/test/lisp/eshell/em-dirs-tests.el
+++ b/test/lisp/eshell/em-dirs-tests.el
@@ -23,6 +23,7 @@
 
 ;;; Code:
 
+(require 'tramp)
 (require 'ert)
 (require 'esh-mode)
 (require 'eshell)
@@ -112,12 +113,59 @@ em-dirs-test/expand-user-reference/local
    (format "echo ~%s" user-login-name)
    (expand-file-name (format "~%s" user-login-name))))
 
+(ert-deftest em-dirs-test/expand-user-reference/remote ()
+  "Test expansion of \"~USER\" references in remote directories."
+  (skip-unless (eshell-tests-remote-accessible-p))
+  (let* ((default-directory ert-remote-temporary-file-directory)
+         (remote (file-remote-p default-directory)))
+    (let ((eshell-connection-local-file-names t))
+      (eshell-command-result-equal
+       "echo ~"
+       (concat remote (expand-file-name "~")))
+      (eshell-command-result-equal
+       (format "echo ~%s" user-login-name)
+       (concat remote (expand-file-name (format "~%s" user-login-name)))))
+    (let ((eshell-connection-local-file-names nil))
+      (eshell-command-result-equal "echo ~" (expand-file-name "~"))
+      (eshell-command-result-equal
+       (format "echo ~%s" user-login-name)
+       (expand-file-name (format "~%s" user-login-name))))))
+
 (ert-deftest em-dirs-test/expand-user-reference/quoted ()
   "Test that a quoted \"~\" isn't expanded."
   (eshell-command-result-equal "echo \\~" "~")
   (eshell-command-result-equal "echo \"~\"" "~")
   (eshell-command-result-equal "echo '~'" "~"))
 
+(ert-deftest em-dirs-test/expand-absolute-file/local ()
+  "Test \"expansion\" of absolute files in local directories.
+This should always be a no-op."
+  (let ((eshell-connection-local-file-names t))
+    (eshell-command-result-equal "echo /bin/foo" "/bin/foo"))
+  (let ((eshell-connection-local-file-names nil))
+    (eshell-command-result-equal "echo /bin/foo" "/bin/foo")))
+
+(ert-deftest em-dirs-test/expand-absolute-file/remote ()
+  "Test expansion of absolute files in remote directories.
+This should be a file name on the remote host when
+`eshell-connection-local-file-names' is non-nil."
+  (skip-unless (eshell-tests-remote-accessible-p))
+  (let* ((default-directory ert-remote-temporary-file-directory)
+         (remote (file-remote-p default-directory)))
+    (let ((eshell-connection-local-file-names t))
+      (eshell-command-result-equal "echo /bin/foo" (concat remote "/bin/foo")))
+    (let ((eshell-connection-local-file-names nil))
+      (eshell-command-result-equal "echo /bin/foo" "/bin/foo"))))
+
+(ert-deftest em-dirs-test/expand-absolute-file/quoted ()
+  "Test that a quoted \"/\" for an absolute file isn't expanded."
+  (skip-unless (eshell-tests-remote-accessible-p))
+  (let ((default-directory ert-remote-temporary-file-directory)
+        (eshell-connection-local-file-names t))
+    (eshell-command-result-equal "echo \\/bin/foo" "/bin/foo")
+    (eshell-command-result-equal "echo \"/bin/foo\"" "/bin/foo")
+    (eshell-command-result-equal "echo '/bin/foo'" "/bin/foo")))
+
 \f
 ;; `cd'
 
diff --git a/test/lisp/eshell/esh-ext-tests.el b/test/lisp/eshell/esh-ext-tests.el
index 8abbd74f737..aac13c3d8ea 100644
--- a/test/lisp/eshell/esh-ext-tests.el
+++ b/test/lisp/eshell/esh-ext-tests.el
@@ -27,6 +27,7 @@
 (require 'ert)
 (require 'esh-mode)
 (require 'esh-ext)
+(require 'em-dirs)
 (require 'eshell)
 
 (require 'eshell-tests-helpers
@@ -74,6 +75,48 @@ esh-ext-test/addpath/set-locally
      (eshell-match-command-output "echo $PATH"
                                   (concat original-path "\n")))))
 
+(ert-deftest em-ext-test/unexpand-remote-file/local ()
+  "Test unexpansion of remote file names with a local, external command."
+  (skip-unless (and (eshell-tests-remote-accessible-p)
+                    (executable-find "echo")))
+  (let* ((default-directory ert-remote-temporary-file-directory)
+         (remote (file-remote-p default-directory)))
+    (let ((eshell-connection-local-file-names t))
+      (eshell-command-result-equal "*echo /bin/foo" "/bin/foo\n")
+      (eshell-command-result-equal (format "*echo %s/bin/foo" remote)
+                                   "/bin/foo\n")
+      (should-error (eshell-command-result-equal
+                     "*echo /ssh:nowhere.invalid:/bin/foo"
+                     "/bin/foo\n" t)))
+    (let ((eshell-connection-local-file-names nil))
+      (eshell-command-result-equal "*echo /bin/foo" "/bin/foo\n")
+      (eshell-command-result-equal (format "*echo %s/bin/foo" remote)
+                                   (concat remote "/bin/foo\n"))
+      (eshell-command-result-equal
+       "*echo /ssh:nowhere.invalid:/bin/foo"
+       "/ssh:nowhere.invalid:/bin/foo\n"))))
+
+(ert-deftest em-ext-test/unexpand-absolute-file/remote ()
+  "Test unexpansion of remote file names with a remote, external command."
+  (skip-unless (and (eshell-tests-remote-accessible-p)
+                    (executable-find "echo")))
+  (let* ((default-directory ert-remote-temporary-file-directory)
+         (remote (file-remote-p default-directory)))
+    (let ((eshell-connection-local-file-names t))
+      (eshell-command-result-equal "*echo /bin/foo" "/bin/foo\n")
+      (eshell-command-result-equal (format "*echo %s/bin/foo" remote)
+                                   "/bin/foo\n")
+      (should-error (eshell-command-result-equal
+                     "*echo /ssh:nowhere.invalid:/bin/foo"
+                     "/bin/foo\n" t)))
+    (let ((eshell-connection-local-file-names nil))
+      (eshell-command-result-equal "*echo /bin/foo" "/bin/foo\n")
+      (eshell-command-result-equal (format "*echo %s/bin/foo" remote)
+                                   (concat remote "/bin/foo\n"))
+      (eshell-command-result-equal
+       "*echo /ssh:nowhere.invalid:/bin/foo"
+       "/ssh:nowhere.invalid:/bin/foo\n"))))
+
 (ert-deftest esh-ext-test/explicitly-remote-command ()
   "Test that an explicitly-remote command is remote no matter the current dir."
   (skip-unless (and (eshell-tests-remote-accessible-p)
diff --git a/test/lisp/eshell/eshell-tests-helpers.el b/test/lisp/eshell/eshell-tests-helpers.el
index 652146fefcc..9db13cc617f 100644
--- a/test/lisp/eshell/eshell-tests-helpers.el
+++ b/test/lisp/eshell/eshell-tests-helpers.el
@@ -180,13 +180,16 @@ eshell-command-result--equal-explainer
 (put 'eshell-command-result--equal 'ert-explainer
      #'eshell-command-result--equal-explainer)
 
-(defun eshell-command-result-equal (command result)
-  "Execute COMMAND non-interactively and compare it to RESULT."
+(defun eshell-command-result-equal (command result &optional ignore-errors)
+  "Execute COMMAND non-interactively and compare it to RESULT.
+If IGNORE-ERRORS is non-nil, ignore any errors signaled when
+inserting the command."
   (ert-info (#'eshell-get-debug-logs :prefix "Command logs: ")
-    (should (eshell-command-result--equal
-             command
-             (eshell-test-command-result command)
-             result))))
+    (let ((debug-on-error (and (not ignore-errors) debug-on-error)))
+      (should (eshell-command-result--equal
+               command
+               (eshell-test-command-result command)
+               result)))))
 
 (provide 'eshell-tests-helpers)
 
-- 
2.25.1

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Sun, 5 May 2024 13:58:55 -0700
> From: Jim Porter <jporterbugs@gmail.com>
> 
> One oddity of Eshell is that even when you're connected to a remote host 
> (usually you just "cd" into a remote directory using Tramp syntax), 
> absolute file names are still on your *local* host when running any Lisp 
> commands. However, running *external* commands (programs on the remote 
> host), absolute file names are on that remote host.
> 
> When you think about how it's implemented, this makes sense: Lisp 
> commands always run in the local Emacs process, but external programs 
> run on the remote. So naturally, "absolute" file names are relative to a 
> different host in either case. This wouldn't be so bad except that it's 
> not always obvious when you're running a Lisp command or not. Eshell 
> provides Lisp implementations of some common commands, like "cat", but 
> it also transparently falls back to the external program if it doesn't 
> understand some option. This results in it being pretty hard to tell 
> what's going to happen when you run a command.
> 
> There's an "elecslash" module for Eshell that helps with this, but it 
> can't tell when you have a Lisp command that will actually fallback to 
> the external program when you run it.
> 
> Instead, the attached patch provides a new way to handle this: if you 
> enable 'eshell-connection-local-file-names', then "normal" absolute file 
> names like "/foo/bar" or "~/user" are evaluated relative to the current 
> remote connection (if any). Eshell does this by expanding the file name 
> to a full remote name like "/ssh:remote:/foo/bar". If these strings get 
> sent to an external program, Eshell will unexpand them back to a 
> host-local name (it will make sure that the remote host is correct, too).
> 
> You can also keep Eshell from performing this expansion on a 
> case-by-case basis by quoting the file name (like "this", 'this', or 
> /:this) or escaping the leading / or ~.

I don't understand how would this work conceptually.  Suppose I want
to run a command on a remote host, but pipe the results into a command
that runs locally -- how will Eshell know which file name to interpret
relative to which directory, and how can the user indicate which is
which unequivocally?

IOW, I fear that this problem cannot be solved in principle in a
shell-like application, and so trying to solve it will only cause us a
terrible complexity mess.  What am I missing?

Thanks.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/6/2024 4:14 AM, Eli Zaretskii wrote:
> I don't understand how would this work conceptually.  Suppose I want
> to run a command on a remote host, but pipe the results into a command
> that runs locally -- how will Eshell know which file name to interpret
> relative to which directory, and how can the user indicate which is
> which unequivocally?

Thanks for taking a look. I'll try to explain this in more detail, since 
it's a fairly subtle interaction, especially if you don't use Tramp + 
Eshell heavily. (With the benefit of hindsight, we might have chosen not 
to handle remote access in Eshell the way it does, but it's now one of 
the big features users mention when they describe what they like about 
it. This patch is my best attempt at smoothing some of the existing 
rough edges here so that remote access in Eshell doesn't feel so 
capricious.)

Anyway...

File names are expanded according to the current working directory when 
you enter your command, so unless you explicitly type out the host in a 
file name, it's treated as belonging to the host associated with cwd. 
(If it helps, this is sort of like how "~" is expanded in regular 
shells. The shell expands it early on when parsing your input, so "sudo 
echo ~" outputs *your* homedir, not root's.)

Here are some examples with the new option enabled (note that "*" before 
a program name means "always execute the external program on the host"):

   ##### 1. Change to root
~ $ cd /sudo::
/sudo:root@host:~ # pwd; *pwd
/sudo:root@host:/root
/root

   ##### 2. Change to an absolute directory, stay as root
/sudo:root@host:~ # cd /etc; pwd; *pwd
/sudo:root@host:/etc
/etc

   ##### 3. Change to the home directory, stay as root
/sudo:root@host:~ # cd ~; pwd; *pwd
/sudo:root@host:/root
/root

   ##### 4. Write the expanded "~/foo.txt" to the *local* "~/bar.txt".
   ##### Using "/:" quoting lets you explicitly name a local file
/sudo:root@host:~ # *echo ~/foo.txt > /:~/bar.txt
/sudo:root@host:~ # cat bar.txt
/bin/cat: bar.txt: No such file or directory

   ##### 5. Change to the *local* home directory, stop being root
/sudo:root@host:~ # cd /:~; pwd; *pwd
/home/jim
/home/jim

   ##### 6. "bar.txt" ended up here
~ $ cat bar.txt
['-c', '/root/foo.txt']

In the last line above, note that the value we wrote to our file is just 
the local part. There's no Tramp remote host part here because Python 
(or other any other external program) wouldn't understand that syntax. 
Eshell strips off the remote part for you, unless you escape the 
argument, e.g. by surrounding it with quotes. (When doing this 
unexpanding, Eshell also makes sure that the remote host of the file 
name matches the host where the program will run.)

> IOW, I fear that this problem cannot be solved in principle in a
> shell-like application, and so trying to solve it will only cause us a
> terrible complexity mess.  What am I missing?

With this option *disabled* (the default), there are some problems that 
(in my opinion) make working with remote file names in Eshell even more 
complex. For example, suppose I'm on a remote host, and want to change 
to my home directory on that remote. There's not an easy way to do that:

   ##### 3b. Change to the home directory; stop being root(!)
/sudo:root@host:~ # cd ~; pwd; *pwd
/home/jim
/home/jim

Or suppose my cwd is /ssh:user@remote:/somedir. If I run "cat 
/etc/hosts", Eshell will print my local hosts file. But what if I run 
"cat -n /etc/hosts"? Eshell's "cat" implementation doesn't understand 
"-n", so it calls the "cat" on "remote". Now it prints the remote hosts 
file. You can only predict which hosts files you'll get if you know 
exactly which flags Eshell's "cat" implementation supports.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Mon, 6 May 2024 11:13:20 -0700
> Cc: 70792@debbugs.gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> File names are expanded according to the current working directory when 
> you enter your command, so unless you explicitly type out the host in a 
> file name, it's treated as belonging to the host associated with cwd. 

I know this, but we are explicitly talking about _absolute_ file
names, which normally trivially expand to themselves.  _That_ is the
problem which I was talking about: you seem to propose a feature where
an absolute file name is sometimes expanded not to itself, but to a
remote file name.

>    ##### 1. Change to root
> ~ $ cd /sudo::
> /sudo:root@host:~ # pwd; *pwd
> /sudo:root@host:/root
> /root
> 
>    ##### 2. Change to an absolute directory, stay as root
> /sudo:root@host:~ # cd /etc; pwd; *pwd
> /sudo:root@host:/etc
> /etc

So you are saying that to chdir to the _local_ /etc I must quote it as
in "/:/etc", or somesuch?  If not, how do I chdir to a local directory
by its absolute file name?

>    ##### 3. Change to the home directory, stay as root
> /sudo:root@host:~ # cd ~; pwd; *pwd
> /sudo:root@host:/root
> /root

Likewise here: how to chdir to the _local_ home directory? quote it?

>    ##### 4. Write the expanded "~/foo.txt" to the *local* "~/bar.txt".
>    ##### Using "/:" quoting lets you explicitly name a local file
> /sudo:root@host:~ # *echo ~/foo.txt > /:~/bar.txt
> /sudo:root@host:~ # cat bar.txt
> /bin/cat: bar.txt: No such file or directory
> 
>    ##### 5. Change to the *local* home directory, stop being root
> /sudo:root@host:~ # cd /:~; pwd; *pwd
> /home/jim
> /home/jim

That's awful!  Completely un-natural, let alone a lot of typing!

Also, am I still able to specify remote file names when my default
directory is local?  Or do I have to chdir to a remote directory
first?

>    ##### 6. "bar.txt" ended up here
> ~ $ cat bar.txt
> ['-c', '/root/foo.txt']

Is "/root/foo.txt" a local or remote file name?  (I know you used
*echo, but the file bar.text has no memory of that.)

> In the last line above, note that the value we wrote to our file is just 
> the local part.

And that's considered a feature??

> With this option *disabled* (the default), there are some problems that 
> (in my opinion) make working with remote file names in Eshell even more 
> complex. For example, suppose I'm on a remote host, and want to change 
> to my home directory on that remote. There's not an easy way to do that:

The simplest solution is to introduce a special command for that, so
that the user could tell Eshell explicitly whether he/she wants to
consider him/herself on the local or the remote host.  Similar to what
we did with rlogin or ssh.

> Or suppose my cwd is /ssh:user@remote:/somedir. If I run "cat 
> /etc/hosts", Eshell will print my local hosts file. But what if I run 
> "cat -n /etc/hosts"? Eshell's "cat" implementation doesn't understand 
> "-n", so it calls the "cat" on "remote". Now it prints the remote hosts 
> file. You can only predict which hosts files you'll get if you know 
> exactly which flags Eshell's "cat" implementation supports.

If Eshel knew that I consider myself on the remote, it could have
modified the logic accordingly, to DTRT.  Without such an explicit
knowledge, we are _guessing_, and our guesses are bound to be wrong
sometimes.

Bottom line: instead of trying to improve our guesswork, I suggest to
explore the possibility of adding new command(s) that will tell Eshell
exactly what the user means wrt local/remote operation.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

Before I respond to the initial points, I wanted to emphasize one part 
first: my patch is intended to make Eshell behave more like other shells 
and to simplify how users can perform "common" tasks. By "common", I 
mean specifically that, when you've started a remote "session" (by 
cd'ing into a remote host), normal filenames all refer to something on 
that host, just like if you used SSH on a terminal. To refer to 
something on another host, you use remote file name syntax (or /:quoted 
file name syntax to explicitly refer to a local file[1]).

Also, even with this option, absolutely nothing changes about how Eshell 
works when the current directory is local.

On 5/6/2024 11:43 AM, Eli Zaretskii wrote:
> I know this, but we are explicitly talking about _absolute_ file
> names, which normally trivially expand to themselves.  _That_ is the
> problem which I was talking about: you seem to propose a feature where
> an absolute file name is sometimes expanded not to itself, but to a
> remote file name.

Correct. Eshell's transparent remote access forces us to consider a 
question that other shells don't have: how do I refer to a file that's 
absolute *on the current connection*?

Currently, for Lisp-based commands, you have to type the the full remote 
part of the file name again, like "/ssh:user@remote:/some/file". For 
external commands, you have to type just the local part, like 
"/some/file". For commands that could be *either* Lisp-based or external 
(this includes most Eshell built-ins), there's no way to do this. 
(Unless you know exactly how Eshell implements things.)

>>     ##### 2. Change to an absolute directory, stay as root
>> /sudo:root@host:~ # cd /etc; pwd; *pwd
>> /sudo:root@host:/etc
>> /etc
> 
> So you are saying that to chdir to the _local_ /etc I must quote it as
> in "/:/etc", or somesuch?  If not, how do I chdir to a local directory
> by its absolute file name?

If your current working directory is local, "cd /etc" is enough (since 
the current host is the local one). If your current working directory is 
remote, any of the following would change back to the local /etc:

   cd /:/etc
   cd \/etc
   cd "/etc"
   cd '/etc'

>>     ##### 3. Change to the home directory, stay as root
>> /sudo:root@host:~ # cd ~; pwd; *pwd
>> /sudo:root@host:/root
>> /root
> 
> Likewise here: how to chdir to the _local_ home directory? quote it?

If your cwd is local, just "cd ~" is enough. If cwd is remote then you'd 
use "cd /:~".

>>     ##### 4. Write the expanded "~/foo.txt" to the *local* "~/bar.txt".
>>     ##### Using "/:" quoting lets you explicitly name a local file
>> /sudo:root@host:~ # *echo ~/foo.txt > /:~/bar.txt
>> /sudo:root@host:~ # cat bar.txt
>> /bin/cat: bar.txt: No such file or directory
>>
>>     ##### 5. Change to the *local* home directory, stop being root
>> /sudo:root@host:~ # cd /:~; pwd; *pwd
>> /home/jim
>> /home/jim
> 
> That's awful!  Completely un-natural, let alone a lot of typing!

It's only two extra characters compared to the equivalent command that 
all happens on a single host (which I think would be the more-common 
scenario):

   ##### 4b. Write the expanded "~/foo.txt" to the remote "~/bar.txt".
   ##### Using "/:" quoting lets you explicitly name a local file
/sudo:root@host:~ # *echo ~/foo.txt > ~/bar.txt
/sudo:root@host:~ # cat bar.txt
['-c', '/root/foo.txt']

The example I chose is somewhat contrived, of course. I just wanted to 
show off how you can mix local and remote expanded file names in a 
single command for this case.

> Also, am I still able to specify remote file names when my default
> directory is local?  Or do I have to chdir to a remote directory
> first?

Yes. Just type the full remote name, like "/ssh:user@remote:/file".

>>     ##### 6. "bar.txt" ended up here
>> ~ $ cat bar.txt
>> ['-c', '/root/foo.txt']
> 
> Is "/root/foo.txt" a local or remote file name?  (I know you used
> *echo, but the file bar.text has no memory of that.)

At this point, it's just text, so *technically* it's neither. However, 
that text was created from the local portion of a remote file name, so 
the string is local to the remote host where Python was executed. (In 
the example, I used "sudo", so I suppose local/remote are misnomers, but 
the same reasoning applies if you used "ssh".)

>> In the last line above, note that the value we wrote to our file is just
>> the local part.
> 
> And that's considered a feature??

As mentioned above, this is a contrived example to show the lifecycle of 
these names, but yes. The Python command I used just shows off the 
internals. For a more practical example, suppose I want to write the 
word counts of a remote file into a local file. This might actually come 
up in practice: if the remote file is large, I don't want to copy the 
whole thing locally first. With this new option, I could do the following:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > /:~/counts.txt

Today, you could do this like so:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > ~/counts.txt

That looks a bit simpler at a glance (no "/:"), but now there's a 
problem lurking: "~" points to the remote homedir in the first case, and 
the local homedir in the second.

Now suppose a slightly different case. What if I'm in a remote directory 
and want to write this summary file to my remote homedir? With this new 
option, I could type the following:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > ~/counts.txt

Today, you'd need to do this:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > 
/ssh:user@remote:~/counts.txt

Or this:

   /ssh:user@remote:/somedir $ cd /ssh:user@remote:~
   /ssh:user@remote:~ $ *wc file.txt > counts.txt

In both of the "today" cases, you need to type "/ssh:user@remote:~" 
somewhere, which is the thing I'd consider to be unnatural and a lot of 
typing.

>> With this option *disabled* (the default), there are some problems that
>> (in my opinion) make working with remote file names in Eshell even more
>> complex. For example, suppose I'm on a remote host, and want to change
>> to my home directory on that remote. There's not an easy way to do that:
> 
> The simplest solution is to introduce a special command for that, so
> that the user could tell Eshell explicitly whether he/she wants to
> consider him/herself on the local or the remote host.  Similar to what
> we did with rlogin or ssh.

Eshell users already have a command for explicitly moving between local 
and remote hosts: it's just "cd".

I'm not familiar with what we do about rlogin and ssh though. Where 
would I find that info? If someone else has come up with a better way to 
handle a similar scenario, I'd be happy to take a look.

>> Or suppose my cwd is /ssh:user@remote:/somedir. If I run "cat
>> /etc/hosts", Eshell will print my local hosts file. But what if I run
>> "cat -n /etc/hosts"? Eshell's "cat" implementation doesn't understand
>> "-n", so it calls the "cat" on "remote". Now it prints the remote hosts
>> file. You can only predict which hosts files you'll get if you know
>> exactly which flags Eshell's "cat" implementation supports.
> 
> If Eshel knew that I consider myself on the remote, it could have
> modified the logic accordingly, to DTRT.  Without such an explicit
> knowledge, we are _guessing_, and our guesses are bound to be wrong
> sometimes.

Unless I'm misunderstanding what you mean here, Eshell *does* know that 
you consider yourself on the remote. You previously cd'ed into a remote 
directory, expressing your intention to Eshell explicitly. With my 
patch, there's no guesswork here. At least the way I interpret your 
message, this patch does exactly what you suggest: because Eshell knows 
that you consider yourself on the remote (you cd'ed into it) and that 
you chose this new option, it knows that "/foo/bar" should refer to a 
file on the remote system, ensuring that your commands work the same no 
matter whether they're Lisp-based or external programs.

Without using this option, I don't think there's a way to DTRT in 
general. Currently, the string "/foo/bar" is just that, a string. It 
carries no information about the host Emacs should look on. Existing 
commands (whether Lisp-based or external) will just look on the host 
where the process is running. Conceptually, my patch adds annotations to 
these strings so that we can determine authoritatively which host they 
belong to.

[1] Eshell has some code to handle this syntax, but it's built on the 
one of the intended uses for quoted file names. From "Quoted File Names" 
in the Emacs manual: "For example, you can quote a local file name which 
appears remote, to prevent it from being treated as a remote file name."

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/6/2024 1:05 PM, Jim Porter wrote:
> Before I respond to the initial points, I wanted to emphasize one part 
> first: my patch is intended to make Eshell behave more like other shells 
> and to simplify how users can perform "common" tasks. By "common", I 
> mean specifically that, when you've started a remote "session" (by 
> cd'ing into a remote host), normal filenames all refer to something on 
> that host, just like if you used SSH on a terminal. To refer to 
> something on another host, you use remote file name syntax (or /:quoted 
> file name syntax to explicitly refer to a local file[1]).
> 
> Also, even with this option, absolutely nothing changes about how Eshell 
> works when the current directory is local.

One final point I'd like to make here is that this new (opt-in) behavior 
to treat "absolute" file names relative to the current remote connection 
already has precedent in Eshell. It's how Eshell looks for *commands* 
(as far as I know, it's always been this way). That means that 
"/bin/whoami" is the local whoami if the cwd is local, but it's the 
remote whoami is the cwd is remote:

   ~ $ /bin/whoami
   jim
   ~ $ cd /sudo::
   /sudo:root@host:~ # /bin/whoami
   root

By enabling this new option, Eshell will treat arguments much the same 
as it treats command names.

(Of course, I'm happy to explain any part of this in more detail, and to 
add more documentation to the manual for whatever outcome we can agree 
to. The behavior around Tramp + Eshell in general is non-obvious, hence 
my hesitance to submit this patch without thinking it over for a *long* 
time. As the diff says, I finished the initial version of this last 
September and have spent much of the intervening time considering 
whether this behavior makes sense and how/if it could go wrong.)

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Mon, 6 May 2024 13:05:28 -0700
> Cc: 70792@debbugs.gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> Currently, for Lisp-based commands, you have to type the the full remote 
> part of the file name again, like "/ssh:user@remote:/some/file". For 
> external commands, you have to type just the local part, like 
> "/some/file". For commands that could be *either* Lisp-based or external 
> (this includes most Eshell built-ins), there's no way to do this. 
> (Unless you know exactly how Eshell implements things.)

I think asking the user to specify the full "/ssh:..." file name for
remote files makes this easier, because Eshell could then
transparently handle also the commands which can be external or
internal.  That's one confusing problem down.

> > So you are saying that to chdir to the _local_ /etc I must quote it as
> > in "/:/etc", or somesuch?  If not, how do I chdir to a local directory
> > by its absolute file name?
> 
> If your current working directory is local, "cd /etc" is enough (since 
> the current host is the local one). If your current working directory is 
> remote, any of the following would change back to the local /etc:
> 
>    cd /:/etc
>    cd \/etc
>    cd "/etc"
>    cd '/etc'

The first form I could maybe agree to.  But the others are standard
shell quoting, so I think repurposing them for "escape to local file
names" is not a good idea, since quoting is used in shell commands for
other reasons, notably for quoting special characters.  We need to
allow user to quote file names without implying the name is local.

So I think the last 3 examples should not mean local file names.

> >>     ##### 5. Change to the *local* home directory, stop being root
> >> /sudo:root@host:~ # cd /:~; pwd; *pwd
> >> /home/jim
> >> /home/jim
> > 
> > That's awful!  Completely un-natural, let alone a lot of typing!
> 
> It's only two extra characters

Two extra characters _per_file_name_!

> > The simplest solution is to introduce a special command for that, so
> > that the user could tell Eshell explicitly whether he/she wants to
> > consider him/herself on the local or the remote host.  Similar to what
> > we did with rlogin or ssh.
> 
> Eshell users already have a command for explicitly moving between local 
> and remote hosts: it's just "cd".

Then why do we need to change anything at all?

My suggestion is to introduce a notion of "state", either local or
remote, and make it so that the "state" defines the semantics of file
names without the Tramp "/METHOD:..." prefix: such names _always_ mean
files on the host specified by the "state".  In particular, quoted
file names also specify files according to the "state"; they do not
"escape" to the other host.

This matches what you get from a normal shell: if you login to a
remote host, _all_ the file names are from the remote host.

It is okay to use the value of default-directory as the "state", but
we should use it consistently.  the only way of "escaping" from the
current host is by using fully-qualified file names that specify the
host explicitly.

The advantage of this is twofold: (a) users will always know what do
file names refer to, and (b) Eshell will always know that, and will be
able to DTRT in all such cases by converting the file names the user
typed to the appropriate form under the hood as needed.

> I'm not familiar with what we do about rlogin and ssh though. Where 
> would I find that info? If someone else has come up with a better way to 
> handle a similar scenario, I'd be happy to take a look.

See above.  The advantage of those is that they never allow you to mix
file names on different hosts except explicitly.

> > If Eshel knew that I consider myself on the remote, it could have
> > modified the logic accordingly, to DTRT.  Without such an explicit
> > knowledge, we are _guessing_, and our guesses are bound to be wrong
> > sometimes.
> 
> Unless I'm misunderstanding what you mean here, Eshell *does* know that 
> you consider yourself on the remote.

Then why is external vs internal commands an issue? why cannot Eshell
DTRT by invoking the command which can handle the file names the user
typed?

> Without using this option, I don't think there's a way to DTRT in 
> general.

My point is that we should try to find such a way.  Personally, I
don't think it's a problem to find it.

> Currently, the string "/foo/bar" is just that, a string. It 
> carries no information about the host Emacs should look on.

The "state" should supply the missing information, and the
"/METHOD:..." notation should allow the user to override what the
"state" says implicitly.

> Existing commands (whether Lisp-based or external) will just look on
> the host where the process is running.

Some kind of dispatch should look at the "state" before it runs the
command, and decide which variety of the command to run.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

(Replying a bit out of order to make the structure of my responses clearer.)

On 5/7/2024 4:55 AM, Eli Zaretskii wrote:
>> Eshell users already have a command for explicitly moving between local
>> and remote hosts: it's just "cd".
> 
> Then why do we need to change anything at all?
> 
> My suggestion is to introduce a notion of "state", either local or
> remote, and make it so that the "state" defines the semantics of file
> names without the Tramp "/METHOD:..." prefix: such names _always_ mean
> files on the host specified by the "state".  In particular, quoted
> file names also specify files according to the "state"; they do not
> "escape" to the other host.
> 
> This matches what you get from a normal shell: if you login to a
> remote host, _all_ the file names are from the remote host.
> 
> It is okay to use the value of default-directory as the "state", but
> we should use it consistently.  the only way of "escaping" from the
> current host is by using fully-qualified file names that specify the
> host explicitly.

I think we must be close to seeing things the same way, since your 
explanation is almost exactly how I proposed to fix this initially. In 
retrospect, I should probably have spent a bit more space introducing 
the issue and explaining why I did what I did; most of my initial 
explanation was simply on *how* the patch works.

So I'll try to provide a summary of this problem from a user POV to make 
sure we're on the same page:

----- SUMMARY -----

Currently, Eshell has a problem: when connected to a remote host (i.e. 
cd'ed into a remote dir), sometimes an absolute file name like 
"/foo/bar" means "a file on the current host", and sometimes it means "a 
file on the *local* host". This new option lets you change that so an 
absolute file name *always* means "a file on the current host"; this 
matches the behavior of normal shells. If you want to refer to a file on 
a different host, you instead type the fully-qualified remote file name 
("/ssh:remote:/foo/bar"); this always means "/foo/bar on remote".

----- END SUMMARY -----

There's just one open question with this: if I'm on a remote system, how 
do I type the fully-qualified *local* file name? I propose using "/:" as 
the prefix to mean "always look on the local host", so "/:/etc/foo.log" 
is a local file name no matter what. For this case, I'm open to other 
spellings, so long as we have *some* way to fully-qualify a local file name.

>> Currently, for Lisp-based commands, you have to type the the full remote 
>> part of the file name again, like "/ssh:user@remote:/some/file". For 
>> external commands, you have to type just the local part, like 
>> "/some/file". For commands that could be *either* Lisp-based or external 
>> (this includes most Eshell built-ins), there's no way to do this. 
>> (Unless you know exactly how Eshell implements things.)
> 
> I think asking the user to specify the full "/ssh:..." file name for
> remote files makes this easier, because Eshell could then
> transparently handle also the commands which can be external or
> internal.  That's one confusing problem down.

Based on your longer explanation above, I think we agree entirely here.

>>> So you are saying that to chdir to the _local_ /etc I must quote it as
>>> in "/:/etc", or somesuch?  If not, how do I chdir to a local directory
>>> by its absolute file name?
>>
>> If your current working directory is local, "cd /etc" is enough (since 
>> the current host is the local one). If your current working directory is 
>> remote, any of the following would change back to the local /etc:
>>
>>    cd /:/etc
>>    cd \/etc
>>    cd "/etc"
>>    cd '/etc'
> 
> The first form I could maybe agree to.  But the others are standard
> shell quoting, so I think repurposing them for "escape to local file
> names" is not a good idea, since quoting is used in shell commands for
> other reasons, notably for quoting special characters.  We need to
> allow user to quote file names without implying the name is local.

That's fine by me; I could remove options 2-4 from this list (so that 
they mean "/etc on the current host"). I had chosen this set of options 
to match the escaping logic for "~" in Eshell. However, I think I agree 
that that's too invasive: a user might want to write a file name with 
spaces "/something/like this.txt", and would be surprised if that 
escaped to a local filename.

>>>>     ##### 5. Change to the *local* home directory, stop being root
>>>> /sudo:root@host:~ # cd /:~; pwd; *pwd
>>>> /home/jim
>>>> /home/jim
>>>
>>> That's awful!  Completely un-natural, let alone a lot of typing!
>>
>> It's only two extra characters
> 
> Two extra characters _per_file_name_!

Note that these extra characters *only* come up if you're connected to a 
remote host and need to explicitly refer to a file on your local system. 
With this new option enabled, the "normal" spelling of "~/blah" would be 
on the current host, just like normal shells. The "/:" is there to 
explicitly override that behavior by fully-qualifying the name. (This 
makes the common case - everything on the same host - easier to type. 
You just type the normal, unqualified name.)

>> Currently, the string "/foo/bar" is just that, a string. It
>> carries no information about the host Emacs should look on.
> 
> The "state" should supply the missing information, and the
> "/METHOD:..." notation should allow the user to override what the
> "state" says implicitly.

Agreed. With this option, an unqualified file name will refer to the 
"state" (current remote host) to resolve the name.

>> Existing commands (whether Lisp-based or external) will just look on
>> the host where the process is running.
> 
> Some kind of dispatch should look at the "state" before it runs the
> command, and decide which variety of the command to run.

Eshell already has a lot of logic for determining whether to run a Lisp 
implementation of a command or an external program, so I don't think we 
can change that without breaking many things. However, once Eshell has 
determined the variety of the command, it can provide file names in the 
correct syntax for that variety. (In other words, Lisp code will see the 
fully-qualified file name[1], and external commands will see the 
unqualified file name.)

[1] It might be wise to let Lisp-based commands request unqualified file 
names too; Eshell already does similar things for determining when to 
turn a string like "123" into a Lisp number.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Tue, 7 May 2024 11:54:26 -0700
> Cc: 70792@debbugs.gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> There's just one open question with this: if I'm on a remote system, how 
> do I type the fully-qualified *local* file name? I propose using "/:" as 
> the prefix to mean "always look on the local host", so "/:/etc/foo.log" 
> is a local file name no matter what. For this case, I'm open to other 
> spellings, so long as we have *some* way to fully-qualify a local file name.

I think "/:" quoting should not change the host of the file name.
That's because the user might need this quoting for file names on the
remote host.

If the user wants to specify a local file name while default-directory
is remote, the user can use the normal Tramp "/METHOD:..." notation.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/8/2024 6:20 AM, Eli Zaretskii wrote:
> I think "/:" quoting should not change the host of the file name.
> That's because the user might need this quoting for file names on the
> remote host.

Not to say we *should* do this, but if we kept the "/:" syntax of my 
patch, a user could still /:-quote a remote file name in Eshell by using 
the fully-qualified name like: "/ssh:user@remote:/:/blah".

I can construct an argument for why using /: this way in Eshell would 
make sense, but maybe it's just needlessly "clever"...

(As a note, Eshell already uses /:-quoting to mean "on the local host" 
in one spot: for the command to run. However, I added that for Emacs 30, 
so we can still change it without worrying about compatibility issues. 
See the manual here for more info: 
<https://git.savannah.gnu.org/cgit/emacs.git/tree/doc/misc/eshell.texi#n1534>.)

> If the user wants to specify a local file name while default-directory
> is remote, the user can use the normal Tramp "/METHOD:..." notation.

How about a new "local" method? Then users would type 
"/local::~/some-file.txt". That's more typing, but it's also more clear, 
and doesn't repurpose an existing syntax used elsewhere in Emacs.

I don't think the extra typing is *too* bad, since cross-host file names 
are probably a lot rarer than regular intra-host ones. Most likely, 
users will use cross-host file names primarily when cd'ing to a 
different host.

If I go this route, I'm not sure whether it would be better to make 
"local" a real file name handler available everywhere in Emacs despite 
only being useful for Eshell, or if Eshell should just strip out the 
"/local::" prefix before sending it to other parts of Emacs. I'm leaning 
towards the former though, since the latter seems like a hack that could 
have unforeseen consequences.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Wed, 8 May 2024 09:13:59 -0700
> Cc: 70792@debbugs.gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 5/8/2024 6:20 AM, Eli Zaretskii wrote:
> > I think "/:" quoting should not change the host of the file name.
> > That's because the user might need this quoting for file names on the
> > remote host.
> 
> Not to say we *should* do this, but if we kept the "/:" syntax of my 
> patch, a user could still /:-quote a remote file name in Eshell by using 
> the fully-qualified name like: "/ssh:user@remote:/:/blah".
> 
> I can construct an argument for why using /: this way in Eshell would 
> make sense, but maybe it's just needlessly "clever"...

We do need the ability to quote like "/ssh:user@remote:/:/blah".
That's why I think quoting should not "escape to local".  Another
reason that I think "/:" quoting should not escape to local is that
"/:" is for protecting file names from being interpreted as
referencing another host; a simple name like "/foo/bar/baz", when
quoted as "/:/foo/bar/baz", should resolve to itself.

> (As a note, Eshell already uses /:-quoting to mean "on the local host" 
> in one spot: for the command to run. However, I added that for Emacs 30, 
> so we can still change it without worrying about compatibility issues. 
> See the manual here for more info: 
> <https://git.savannah.gnu.org/cgit/emacs.git/tree/doc/misc/eshell.texi#n1534>.)

I think we should remove that before we release Emacs 30.  It's wrong
to interpret quoting this way.

> > If the user wants to specify a local file name while default-directory
> > is remote, the user can use the normal Tramp "/METHOD:..." notation.
> 
> How about a new "local" method? Then users would type 
> "/local::~/some-file.txt". That's more typing, but it's also more clear, 
> and doesn't repurpose an existing syntax used elsewhere in Emacs.

Don't we already have that with "/localhost:" or somesuch?

> If I go this route, I'm not sure whether it would be better to make 
> "local" a real file name handler available everywhere in Emacs despite 
> only being useful for Eshell, or if Eshell should just strip out the 
> "/local::" prefix before sending it to other parts of Emacs. I'm leaning 
> towards the former though, since the latter seems like a hack that could 
> have unforeseen consequences.

I'd like Michael's opinion on this, since we will be "invading" the
Tramp methods space.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/8/2024 11:32 AM, Eli Zaretskii wrote:
>> (As a note, Eshell already uses /:-quoting to mean "on the local host"
>> in one spot: for the command to run. However, I added that for Emacs 30,
>> so we can still change it without worrying about compatibility issues.
>> See the manual here for more info:
>> <https://git.savannah.gnu.org/cgit/emacs.git/tree/doc/misc/eshell.texi#n1534>.)
> 
> I think we should remove that before we release Emacs 30.  It's wrong
> to interpret quoting this way.

Ok, whatever we decide for this bug, I'll apply the same syntax over 
there too.

>>> If the user wants to specify a local file name while default-directory
>>> is remote, the user can use the normal Tramp "/METHOD:..." notation.
>>
>> How about a new "local" method? Then users would type
>> "/local::~/some-file.txt". That's more typing, but it's also more clear,
>> and doesn't repurpose an existing syntax used elsewhere in Emacs.
> 
> Don't we already have that with "/localhost:" or somesuch?

I don't see one like that. It shouldn't be too difficult to add though 
(so long as Michael is ok with it).

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

Hi,

>> How about a new "local" method? Then users would type
>> "/local::~/some-file.txt". That's more typing, but it's also more clear,
>> and doesn't repurpose an existing syntax used elsewhere in Emacs.
>
> Don't we already have that with "/localhost:" or somesuch?
>
>> If I go this route, I'm not sure whether it would be better to make
>> "local" a real file name handler available everywhere in Emacs despite
>> only being useful for Eshell, or if Eshell should just strip out the
>> "/local::" prefix before sending it to other parts of Emacs. I'm leaning
>> towards the former though, since the latter seems like a hack that could
>> have unforeseen consequences.

There is no public Tramp syntax for "/local:" or "/localhost:" or alike
in Tramp. Tramp uses internally, in `tramp-null-hop', the "local" method
in order to cache properties of the local machine. But this isn't
intended to be a public API.

> I'd like Michael's opinion on this, since we will be "invading" the
> Tramp methods space.

I'm kind of undecided. We might use this method internally for Eshell,
as "/local::/path/to/file", but it should be clear that it won't be an
official Tramp method. There might be problems, because all methods the
basic Tramp functions work for must be registered in `tramp-methods'.

And there might be other problems if a user changes the Tramp
syntax. The so-called simplified syntax doesn't use explicit method
strings, see (info "(tramp) Change file name syntax") .

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Jim Porter <jporterbugs@gmail.com>,  70792@debbugs.gnu.org
> Date: Thu, 09 May 2024 20:14:07 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> There is no public Tramp syntax for "/local:" or "/localhost:" or alike
> in Tramp. Tramp uses internally, in `tramp-null-hop', the "local" method
> in order to cache properties of the local machine. But this isn't
> intended to be a public API.
> 
> > I'd like Michael's opinion on this, since we will be "invading" the
> > Tramp methods space.
> 
> I'm kind of undecided. We might use this method internally for Eshell,
> as "/local::/path/to/file", but it should be clear that it won't be an
> official Tramp method. There might be problems, because all methods the
> basic Tramp functions work for must be registered in `tramp-methods'.
> 
> And there might be other problems if a user changes the Tramp
> syntax. The so-called simplified syntax doesn't use explicit method
> strings, see (info "(tramp) Change file name syntax") .

What would you suggest as an alternative?

Since there's no official Tramp method for the local host, we could
invent one now.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Michael Albinus <michael.albinus@gmx.de>
>> Cc: Jim Porter <jporterbugs@gmail.com>,  70792@debbugs.gnu.org
>> Date: Thu, 09 May 2024 20:14:07 +0200
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> There is no public Tramp syntax for "/local:" or "/localhost:" or alike
>> in Tramp. Tramp uses internally, in `tramp-null-hop', the "local" method
>> in order to cache properties of the local machine. But this isn't
>> intended to be a public API.
>>
>> > I'd like Michael's opinion on this, since we will be "invading" the
>> > Tramp methods space.
>>
>> I'm kind of undecided. We might use this method internally for Eshell,
>> as "/local::/path/to/file", but it should be clear that it won't be an
>> official Tramp method. There might be problems, because all methods the
>> basic Tramp functions work for must be registered in `tramp-methods'.
>>
>> And there might be other problems if a user changes the Tramp
>> syntax. The so-called simplified syntax doesn't use explicit method
>> strings, see (info "(tramp) Change file name syntax") .
>
> What would you suggest as an alternative?
>
> Since there's no official Tramp method for the local host, we could
> invent one now.

We could start with "/local::". I'm simply not so optimistic like Jim,
that it will be easy. Likely, se need a full Tramp backend for this, say
tramp-local.el. The respective file name handler should remove the
"/local::" prefix from the arguments, call the original file operation,
and add the "/local::" prefix on the results where appropriate.

And somehow, I'd like to check that this is called from Eshell only. Am
I paranoid? Will people use it on their own?

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/9/2024 12:10 PM, Michael Albinus via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>> Since there's no official Tramp method for the local host, we could
>> invent one now.
> 
> We could start with "/local::". I'm simply not so optimistic like Jim,
> that it will be easy. Likely, se need a full Tramp backend for this, say
> tramp-local.el. The respective file name handler should remove the
> "/local::" prefix from the arguments, call the original file operation,
> and add the "/local::" prefix on the results where appropriate.
> 
> And somehow, I'd like to check that this is called from Eshell only. Am
> I paranoid? Will people use it on their own?

Maybe the better solution would simply be for "/local:"[1] to be an 
Eshell-only syntax that DTRT within Eshell. There shouldn't be any 
conflict so long as Tramp promises not to add a "local" method. :)

Now, it's possible that this could conflict with Tramp's alternate file 
name syntax, but a) this feature will be disabled by default to avoid 
automatically changing the meaning of existing Eshell forms, and b) 
Eshell could look at Tramp's syntax and change the spelling for 
"/local:" to avoid collisions if we wanted to avoid this.

(If in the future, we came up with an Emacs-wide use case for this, it 
should be possible to upgrade "/local:" to a full magic file name. I'm 
not sure why we'd want this though.)

[1] Note the single colon, though I'm not dead-set on this. If it's just 
an Eshell notation and not part of Tramp, we don't necessarily need to 
follow Tramp syntax. Of course, if we wanted to keep open the 
possibility that this becomes a new Tramp method, maybe the "::" is the 
more forward-thinking route.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jim Porter <jporterbugs@gmail.com> writes:

Hi Jim,

>> We could start with "/local::". I'm simply not so optimistic like
>> Jim,
>> that it will be easy. Likely, se need a full Tramp backend for this, say
>> tramp-local.el. The respective file name handler should remove the
>> "/local::" prefix from the arguments, call the original file operation,
>> and add the "/local::" prefix on the results where appropriate.
>> And somehow, I'd like to check that this is called from Eshell
>> only. Am
>> I paranoid? Will people use it on their own?
>
> Maybe the better solution would simply be for "/local:"[1] to be an
> Eshell-only syntax that DTRT within Eshell. There shouldn't be any
> conflict so long as Tramp promises not to add a "local" method. :)

Promised. Tramp uses just a tramp-file-name structure in tramp-null-hop,
that's it.

> Now, it's possible that this could conflict with Tramp's alternate
> file name syntax, but a) this feature will be disabled by default to
> avoid automatically changing the meaning of existing Eshell forms, and
> b) Eshell could look at Tramp's syntax and change the spelling for
> "/local:" to avoid collisions if we wanted to avoid this.

OK.

> (If in the future, we came up with an Emacs-wide use case for this, it
> should be possible to upgrade "/local:" to a full magic file name. I'm
> not sure why we'd want this though.)
>
> [1] Note the single colon, though I'm not dead-set on this. If it's
> just an Eshell notation and not part of Tramp, we don't necessarily
> need to follow Tramp syntax. Of course, if we wanted to keep open the
> possibility that this becomes a new Tramp method, maybe the "::" is
> the more forward-thinking route.

Anyway, even if you implement this completely in Eshell, I recommend to
write a file-name-handler for this. Potential 70+ primitive file
operations cannot be handled w/o such a basis, I fear.

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/9/2024 3:15 PM, Michael Albinus via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> Promised. Tramp uses just a tramp-file-name structure in tramp-null-hop,
> that's it.

Thanks for the promise. :)

>> (If in the future, we came up with an Emacs-wide use case for this, it
>> should be possible to upgrade "/local:" to a full magic file name. I'm
>> not sure why we'd want this though.)
>>
>> [1] Note the single colon, though I'm not dead-set on this. If it's
>> just an Eshell notation and not part of Tramp, we don't necessarily
>> need to follow Tramp syntax. Of course, if we wanted to keep open the
>> possibility that this becomes a new Tramp method, maybe the "::" is
>> the more forward-thinking route.
> 
> Anyway, even if you implement this completely in Eshell, I recommend to
> write a file-name-handler for this. Potential 70+ primitive file
> operations cannot be handled w/o such a basis, I fear.

My initial plan here was to have Eshell treat "/local:" similarly to any 
other Eshell-specific syntax (e.g. $VARIABLE expansions): Eshell would 
parse it, convert it to a regular filename, and then no other code would 
ever see the "/local:".

That might not actually work in practice though (I haven't written the 
code yet to prove it), so another option would be to do like you say and 
add a real file-name-handler within Eshell buffers for this.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Date: Thu, 9 May 2024 13:30:57 -0700
> Cc: 70792@debbugs.gnu.org
> From: Jim Porter <jporterbugs@gmail.com>
> 
> > We could start with "/local::". I'm simply not so optimistic like Jim,
> > that it will be easy. Likely, se need a full Tramp backend for this, say
> > tramp-local.el. The respective file name handler should remove the
> > "/local::" prefix from the arguments, call the original file operation,
> > and add the "/local::" prefix on the results where appropriate.
> > 
> > And somehow, I'd like to check that this is called from Eshell only. Am
> > I paranoid? Will people use it on their own?
> 
> Maybe the better solution would simply be for "/local:"[1] to be an 
> Eshell-only syntax that DTRT within Eshell. There shouldn't be any 
> conflict so long as Tramp promises not to add a "local" method. :)

I agree that it's a better solution: since the problem is Eshell-only,
having the solution in Eshell has several advantages.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]

On 5/9/2024 10:45 PM, Eli Zaretskii wrote:
>> Date: Thu, 9 May 2024 13:30:57 -0700
>> Cc: 70792@debbugs.gnu.org
>> From: Jim Porter <jporterbugs@gmail.com>
>>
>> Maybe the better solution would simply be for "/local:"[1] to be an
>> Eshell-only syntax that DTRT within Eshell. There shouldn't be any
>> conflict so long as Tramp promises not to add a "local" method. :)
> 
> I agree that it's a better solution: since the problem is Eshell-only,
> having the solution in Eshell has several advantages.

Here's a patch for the first part of this: it just changes the Eshell 
syntax for "run this command on localhost even if cwd is remote" to use 
"/local:program" instead of "/:program". Now, users can instead use "/:" 
in an Eshell command name to *disable* any remote file handlers.

For example "/:/ssh:user@remote:/foo" now means, "Run the program 'foo' 
on the current host, located in the directory '/ssh:user@remote:/'."

Next, I'll work on updating my original patch for this bug to use the 
new syntax (possibly with a new way of handling "/local:" if I need it). 
I wanted to get this ready first though so that Eshell is in a 
release-ready state for the upcoming Emacs 30, even if I don't finish 
the rest of this in time.

[-- Attachment #2: 0001-Use-local-prefix-in-Eshell-to-run-local-commands-whe.patch --]
[-- Type: text/plain, Size: 9229 bytes --]

From 30ae237516590e53efbe64bada58a1499c5999c5 Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Fri, 10 May 2024 12:22:52 -0700
Subject: [PATCH] Use "/local:" prefix in Eshell to run local commands when cwd
 is remote

* lisp/eshell/esh-ext.el (eshell-explicit-remote-commands)
(eshell-explicit-command): Update docstrings.
(eshell--local-prefix): New constant.
(eshell-handle-remote-command): Remove.
(eshell-quoted-file-command): New function...
(eshell-ext-initialize): ... add it as a hook.
(eshell-remote-command): Support running commands on localhost.
(eshell-connection-local-command): Rename from
'eshell-external-command'.
(eshell-external-command): New implementation calling
'eshell-remote-command' or 'eshell-connection-local-command' as
appropriate.

* test/lisp/eshell/esh-ext-tests.el
(esh-ext-test/explicitly-local-command): Update test.

* doc/misc/eshell.texi (Remote Access): Update documentation.

* etc/NEWS: Update announcement.
---
 doc/misc/eshell.texi              | 17 +++++---
 etc/NEWS                          |  2 +-
 lisp/eshell/esh-ext.el            | 65 ++++++++++++++++++-------------
 test/lisp/eshell/esh-ext-tests.el |  2 +-
 4 files changed, 50 insertions(+), 36 deletions(-)

diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index 30c85da795b..8cb73d4077b 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -1535,13 +1535,18 @@ Remote Access
 When running commands, you can also make them explicitly remote by
 prefixing the command name with a remote identifier, e.g.@:
 @samp{/ssh:user@@remote:whoami}.  This runs the command @code{whoami}
-over the SSH connection for @code{user@@remote}, no matter your
-current directory.  If you want to explicitly run a @emph{local}
-command even when in a remote directory, you can prefix the command
-name with @kbd{/:}, like @samp{/:whoami}.  In either case, you can
+over the SSH connection for @code{user@@remote}, no matter your current
+directory.  If you want to explicitly run a command on your @emph{local}
+machine even when in a remote directory, you can prefix the command name
+with @kbd{/local:}, like @samp{/local:whoami}.  In either case, you can
 also specify the absolute path to the program, e.g.@:
-@samp{/ssh:user@@remote:/usr/bin/whoami}.  To disable this syntax, set
-the option @code{eshell-explicit-remote-commands} to @code{nil}.
+@samp{/ssh:user@@remote:/usr/bin/whoami}.  If you need to refer to a
+program whose file name would be interpreted as an explicitly-remote
+command, you can use @kbd{/:} to quote the name, e.g.@:
+@samp{/:/ssh:user@@remote:whoami} (@pxref{Quoted File Names,,, emacs,
+The GNU Emacs Manual}).  To disable explicity-remote commands entirely,
+you can set the option @code{eshell-explicit-remote-commands} to
+@code{nil}.
 
 @node History
 @section History
diff --git a/etc/NEWS b/etc/NEWS
index bd68cd6d751..3178a9c3032 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -915,7 +915,7 @@ By prefixing a command name in Eshell with a remote identifier, like
 "/ssh:user@remote:whoami", you can now run commands on a particular
 host no matter your current directory.  Likewise, you can run a
 command on your local system no matter your current directory via
-"/:whoami".  For more information, see the "(eshell) Remote Access"
+"/local:whoami".  For more information, see the "(eshell) Remote Access"
 node in the Eshell manual.
 
 +++
diff --git a/lisp/eshell/esh-ext.el b/lisp/eshell/esh-ext.el
index b4fce7a82a2..c1f2d6d14ac 100644
--- a/lisp/eshell/esh-ext.el
+++ b/lisp/eshell/esh-ext.el
@@ -167,23 +167,23 @@ eshell-explicit-command-char
 (defcustom eshell-explicit-remote-commands t
   "If non-nil, support explicitly-remote commands.
 These are commands with a full remote file name, such as
-\"/ssh:host:whoami\".  If this is enabled, you can also run
-explicitly-local commands by using a quoted file name, like
-\"/:whoami\"."
+\"/ssh:host:whoami\".  If this is enabled, you can also explicitly run
+commands on your local host by using the \"/local:\" prefix, like
+\"/local:whoami\"."
   :type 'boolean
   :group 'eshell-ext)
 
 ;;; Functions:
 
+(defconst eshell--local-prefix "/local:")
+
 (defun eshell-ext-initialize ()     ;Called from `eshell-mode' via intern-soft!
   "Initialize the external command handling code."
-  (add-hook 'eshell-named-command-hook #'eshell-explicit-command nil t)
-  (when eshell-explicit-remote-commands
-    (add-hook 'eshell-named-command-hook
-              #'eshell-handle-remote-command nil t)))
+  (add-hook 'eshell-named-command-hook #'eshell-quoted-file-command nil t)
+  (add-hook 'eshell-named-command-hook #'eshell-explicit-command nil t))
 
 (defun eshell-explicit-command (command args)
-  "If a command name begins with `*', call it externally always.
+  "If a command name begins with \"*\", always call it externally.
 This bypasses all Lisp functions and aliases."
   (when (and (> (length command) 1)
 	     (eq (aref command 0) eshell-explicit-command-char))
@@ -194,39 +194,35 @@ eshell-explicit-command
 	(error "%s: external command not found"
 	       (substring command 1))))))
 
-(defun eshell-handle-remote-command (command args)
-  "Handle remote (or quoted) COMMAND names, using ARGS.
-This calls the appropriate function for commands that aren't on
-the connection associated with `default-directory'.  (See
-`eshell-explicit-remote-commands'.)"
-  (if (file-name-quoted-p command)
-      (let ((default-directory (if (file-remote-p default-directory)
-                                   (expand-file-name "~")
-                                 default-directory)))
-        (eshell-external-command (file-name-unquote command) args))
-    (when (file-remote-p command)
-      (eshell-remote-command command args))))
+(defun eshell-quoted-file-command (command args)
+  "If a command name begins with \"/:\", always call it externally.
+Similar to `eshell-explicit-command', this bypasses all Lisp functions
+and aliases, but it also ignores file name handlers."
+  (when (file-name-quoted-p command)
+    (eshell-external-command (file-name-unquote command) args)))
 
 (defun eshell-remote-command (command args)
   "Insert output from a remote COMMAND, using ARGS.
-A remote command is something that executes on a different machine.
-An external command simply means external to Emacs."
+A \"remote\" command in Eshell is something that executes on a different
+machine.  If COMMAND is a remote file name, run it on the host for that
+file; if COMMAND is a local file name, run it locally."
   (let* ((cwd-connection (file-remote-p default-directory))
          (command-connection (file-remote-p command))
          (default-directory (if (equal cwd-connection command-connection)
                                 default-directory
-                              command-connection))
+                              (or command-connection (expand-file-name "~"))))
          ;; Never use the remote connection here.  We don't want to
          ;; expand the local name!  Instead, we want it as the user
          ;; typed, so that if COMMAND is "/ssh:host:cat", we just get
          ;; "cat" as the result.
-         (command-localname (file-remote-p command 'localname 'never)))
-    (unless command-connection
-      (error "%s: not a remote command" command))
+         (command-localname (or (file-remote-p command 'localname 'never)
+                                command)))
     (eshell-external-command command-localname args)))
 
-(defun eshell-external-command (command args)
-  "Insert output from an external COMMAND, using ARGS."
+(defun eshell-connection-local-command (command args)
+  "Insert output from an external COMMAND, using ARGS.
+This always runs COMMAND using the connection associated with the
+current working directory."
   (setq args (eshell-stringify-list (flatten-tree args)))
   (let ((interp (eshell-find-interpreter
 		 command
@@ -243,6 +239,19 @@ eshell-external-command
       (eshell-gather-process-output
        (car interp) (append (cdr interp) args)))))
 
+(defun eshell-external-command (command args)
+  "Insert output from an external COMMAND, using ARGS."
+  (cond
+   ((and eshell-explicit-remote-commands
+         (file-remote-p command))
+    (eshell-remote-command command args))
+   ((and eshell-explicit-remote-commands
+         (string-prefix-p eshell--local-prefix command))
+    (eshell-remote-command
+     (substring command (length eshell--local-prefix)) args))
+   (t
+    (eshell-connection-local-command command args))))
+
 (defun eshell/addpath (&rest args)
   "Add a set of paths to PATH."
   (eshell-eval-using-options
diff --git a/test/lisp/eshell/esh-ext-tests.el b/test/lisp/eshell/esh-ext-tests.el
index 8abbd74f737..ce958d788cc 100644
--- a/test/lisp/eshell/esh-ext-tests.el
+++ b/test/lisp/eshell/esh-ext-tests.el
@@ -102,7 +102,7 @@ esh-ext-test/explicitly-local-command
          ;; Check the value of $INSIDE_EMACS using `sh' in order to
          ;; delay variable expansion.
          (eshell-match-command-output
-          (format "/:%s -c 'echo $INSIDE_EMACS'" cmd)
+          (format "/local:%s -c 'echo $INSIDE_EMACS'" cmd)
           "eshell\n"))))))
 
 ;; esh-ext-tests.el ends here
-- 
2.25.1

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jim Porter <jporterbugs@gmail.com> writes:

Hi Jim,

> Here's a patch for the first part of this: it just changes the Eshell
> syntax for "run this command on localhost even if cwd is remote" to
> use "/local:program" instead of "/:program". Now, users can instead
> use "/:" in an Eshell command name to *disable* any remote file
> handlers.

I've scanned the changes, and they look OK to me as far as I unsderstand
them.

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/13/2024 12:39 AM, Michael Albinus via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:
> I've scanned the changes, and they look OK to me as far as I unsderstand
> them.

Thanks for taking a look. I've now merged this part as e260bf1be7b. 
(Leaving this bug open while I work on the rest of it.)

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

Hi,

>> There's just one open question with this: if I'm on a remote system, how
>> do I type the fully-qualified *local* file name? I propose using "/:" as
>> the prefix to mean "always look on the local host", so "/:/etc/foo.log"
>> is a local file name no matter what. For this case, I'm open to other
>> spellings, so long as we have *some* way to fully-qualify a local file name.
>
> I think "/:" quoting should not change the host of the file name.
> That's because the user might need this quoting for file names on the
> remote host.
>
> If the user wants to specify a local file name while default-directory
> is remote, the user can use the normal Tramp "/METHOD:..." notation.

FTR, we *have* already two different kinds of quoting. "/:<something>"
makes <something> local, whatever syntax it has (for example, Tramp file
name syntax).

"/method:user@host:/:<something>" makes <something> "local" on
"/method:user@host:" whatever syntax it has.

--8<---------------cut here---------------start------------->8---
(expand-file-name "/:/ssh::.emacs") => "/:/ssh::.emacs"
(file-truename "/:/ssh::.emacs") => "/:/ssh::.emacs"
(file-remote-p "/:/ssh::.emacs") => nil
(file-local-name "/:/ssh::.emacs") => "/:/ssh::.emacs"

(expand-file-name "/ssh::/:.emacs") => "/ssh:gandalf:/:.emacs"
(file-truename "/ssh::/:.emacs") => "/ssh:gandalf:/:/home/albinus/.emacs"
(file-remote-p "/ssh::/:.emacs") => "/ssh:gandalf:"
(file-local-name "/ssh::/:.emacs") => "/:.emacs"
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Jim Porter <jporterbugs@gmail.com>,  70792@debbugs.gnu.org
> Date: Wed, 08 May 2024 20:17:27 +0200
> 
> FTR, we *have* already two different kinds of quoting. "/:<something>"
> makes <something> local, whatever syntax it has (for example, Tramp file
> name syntax).
> 
> "/method:user@host:/:<something>" makes <something> "local" on
> "/method:user@host:" whatever syntax it has.

There's a fine nuance here: "/:" does NOT make a file name locale, it
prevents interpreting it as remote due to the "/something:" construct
in it.  "/:" does NOT change the semantics of "/foo/bar/baz", it only
changes the semantics of "/method:user@host:/foo/bar".

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> FTR, we *have* already two different kinds of quoting. "/:<something>"
>> makes <something> local, whatever syntax it has (for example, Tramp file
>> name syntax).
>>
>> "/method:user@host:/:<something>" makes <something> "local" on
>> "/method:user@host:" whatever syntax it has.
>
> There's a fine nuance here: "/:" does NOT make a file name locale, it
> prevents interpreting it as remote due to the "/something:" construct
> in it.  "/:" does NOT change the semantics of "/foo/bar/baz", it only
> changes the semantics of "/method:user@host:/foo/bar".

The intenmtion of file name quoting isn't to make a file name local or
remote. This is just a side-effect in case of Tramp.

The intention of file name quoting is to suppress file name handlers of
any kind.

"/:/method:user@host:/foo/bar.gz" makes "/method:user@host:/foo/bar.gz"
a literal file name, not to be transferred to any file name handler.

"/method:user@host:/:/foo/bar.gz" keeps the Tramp file name handler
active, but makes "/:/foo/bar.gz" a literal file name "/foo/bar.gz" on
the remote machine. This suppresses the jka-compr-handler, for example.

Likely, you mean the same, but I wanted to say it explicitly.

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: jporterbugs@gmail.com,  70792@debbugs.gnu.org
> Date: Thu, 09 May 2024 20:22:57 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> Hi Eli,
> 
> >> FTR, we *have* already two different kinds of quoting. "/:<something>"
> >> makes <something> local, whatever syntax it has (for example, Tramp file
> >> name syntax).
> >>
> >> "/method:user@host:/:<something>" makes <something> "local" on
> >> "/method:user@host:" whatever syntax it has.
> >
> > There's a fine nuance here: "/:" does NOT make a file name locale, it
> > prevents interpreting it as remote due to the "/something:" construct
> > in it.  "/:" does NOT change the semantics of "/foo/bar/baz", it only
> > changes the semantics of "/method:user@host:/foo/bar".
> 
> The intenmtion of file name quoting isn't to make a file name local or
> remote. This is just a side-effect in case of Tramp.

I agree (and tried to say the same in different words).

> The intention of file name quoting is to suppress file name handlers of
> any kind.

I agree.

> "/:/method:user@host:/foo/bar.gz" makes "/method:user@host:/foo/bar.gz"
> a literal file name, not to be transferred to any file name handler.
> 
> "/method:user@host:/:/foo/bar.gz" keeps the Tramp file name handler
> active, but makes "/:/foo/bar.gz" a literal file name "/foo/bar.gz" on
> the remote machine. This suppresses the jka-compr-handler, for example.
> 
> Likely, you mean the same, but I wanted to say it explicitly.

Yes, we agree.

Which is why I think using /: for "escaping to local" in Eshell is
confusing and should not be done.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

Hi,

> If Eshel knew that I consider myself on the remote, it could have
> modified the logic accordingly, to DTRT.  Without such an explicit
> knowledge, we are _guessing_, and our guesses are bound to be wrong
> sometimes.
>
> Bottom line: instead of trying to improve our guesswork, I suggest to
> explore the possibility of adding new command(s) that will tell Eshell
> exactly what the user means wrt local/remote operation.

Perhaps we start with a first step to make it visible, what a file name
is intended to be. We could use different faces for local and remote
file names in a command line of Eshell which hasn't been sent yet, as
Eshell would treat them.

Best regards, Michael.

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Sean Whitton via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hello,

On Sun 05 May 2024 at 01:58pm -07, Jim Porter wrote:

> One oddity of Eshell is that even when you're connected to a remote host
> (usually you just "cd" into a remote directory using Tramp syntax), absolute
> file names are still on your *local* host when running any Lisp
> commands. However, running *external* commands (programs on the remote host),
> absolute file names are on that remote host.
>
> When you think about how it's implemented, this makes sense: Lisp commands
> always run in the local Emacs process, but external programs run on the
> remote. So naturally, "absolute" file names are relative to a different host
> in either case. This wouldn't be so bad except that it's not always obvious
> when you're running a Lisp command or not. Eshell provides Lisp
> implementations of some common commands, like "cat", but it also transparently
> falls back to the external program if it doesn't understand some option. This
> results in it being pretty hard to tell what's going to happen when you run a
> command.

Isn't this by design?  It lets you, e.g, transparently copy a file from
the local to the remote host just with 'cp'.

-- 
Sean Whitton

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Eli Zaretskii]

> Cc: 70792@debbugs.gnu.org
> Date: Mon, 06 May 2024 17:56:18 +0100
> From:  Sean Whitton via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Isn't this by design?  It lets you, e.g, transparently copy a file from
> the local to the remote host just with 'cp'.

Which 'cp'? the built-in one or the external one?  And how do you tell
which one will be invoked?

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/6/2024 9:56 AM, Sean Whitton via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
>> When you think about how it's implemented, this makes sense: Lisp commands
>> always run in the local Emacs process, but external programs run on the
>> remote. So naturally, "absolute" file names are relative to a different host
>> in either case. This wouldn't be so bad except that it's not always obvious
>> when you're running a Lisp command or not. Eshell provides Lisp
>> implementations of some common commands, like "cat", but it also transparently
>> falls back to the external program if it doesn't understand some option. This
>> results in it being pretty hard to tell what's going to happen when you run a
>> command.
> 
> Isn't this by design?  It lets you, e.g, transparently copy a file from
> the local to the remote host just with 'cp'.

Yes, but this breaks in non-obvious ways if Eshell's "cp" implementation 
falls back to the external program. For example, today:

   ~ $ cp file /ssh:remote:~/file    # copies "file" to a remote host
   ~ $ cp -b file /ssh:remote:~/file
   /usr/bin/cp: cannot create regular file '/ssh:remote:~/file': No such
   file or directory

Or the second line might work if you get unlucky, or pass --parents, or...

With the new option, Eshell is smart enough to recognize that this is a 
problem even before it calls "/usr/bin/cp":

   ~ $ cp -b file /ssh:remote:~/file
   ‘/ssh:remote:~/file’ is remote, but current directory is local

Similarly, if you're in a remote directory and try to specify an 
absolute file name on that remote to copy, this is what happens today:

   /ssh:remote:~ $ cp /etc/A /etc/B     # copies local A to local B
   /ssh:remote:~ $ cp -b /etc/A /etc/B  # copies remote A to remote B

With the new option, both cases copy remote A to remote B. If you wanted 
to copy local A to local B with the option enabled, you could do this:

   /ssh:remote:~ $ cp /:/etc/A /:/etc/B     # copies local A to local B
   /ssh:remote:~ $ cp -b /:/etc/A /:/etc/B
   ‘/:/etc/A’ is local, but current directory is remote

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter]

On 5/6/2024 11:28 AM, Jim Porter wrote:
> Yes, but this breaks in non-obvious ways if Eshell's "cp" implementation 
> falls back to the external program. For example, today:
> 
>    ~ $ cp file /ssh:remote:~/file    # copies "file" to a remote host
>    ~ $ cp -b file /ssh:remote:~/file
>    /usr/bin/cp: cannot create regular file '/ssh:remote:~/file': No such
>    file or directory
> 
> Or the second line might work if you get unlucky, or pass --parents, or...

By "work" here, I mean, "cp will copy the file, but to a place you 
likely didn't intend."

bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Sean Whitton via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hello,

On Mon 06 May 2024 at 11:28am -07, Jim Porter wrote:

> On 5/6/2024 9:56 AM, Sean Whitton via Bug reports for GNU Emacs, the Swiss
> army knife of text editors wrote:
>>> When you think about how it's implemented, this makes sense: Lisp commands
>>> always run in the local Emacs process, but external programs run on the
>>> remote. So naturally, "absolute" file names are relative to a different host
>>> in either case. This wouldn't be so bad except that it's not always obvious
>>> when you're running a Lisp command or not. Eshell provides Lisp
>>> implementations of some common commands, like "cat", but it also transparently
>>> falls back to the external program if it doesn't understand some option. This
>>> results in it being pretty hard to tell what's going to happen when you run a
>>> command.
>> Isn't this by design?  It lets you, e.g, transparently copy a file from
>> the local to the remote host just with 'cp'.
>
> Yes, but this breaks in non-obvious ways if Eshell's "cp" implementation falls
> back to the external program. For example, today:
>
>   ~ $ cp file /ssh:remote:~/file    # copies "file" to a remote host
>   ~ $ cp -b file /ssh:remote:~/file
>   /usr/bin/cp: cannot create regular file '/ssh:remote:~/file': No such
>   file or directory
>
> Or the second line might work if you get unlucky, or pass --parents, or...
>
> With the new option, Eshell is smart enough to recognize that this is a
> problem even before it calls "/usr/bin/cp":
>
>   ~ $ cp -b file /ssh:remote:~/file
>   ‘/ssh:remote:~/file’ is remote, but current directory is local
>
> Similarly, if you're in a remote directory and try to specify an absolute file
> name on that remote to copy, this is what happens today:
>
>   /ssh:remote:~ $ cp /etc/A /etc/B     # copies local A to local B
>   /ssh:remote:~ $ cp -b /etc/A /etc/B  # copies remote A to remote B
>
> With the new option, both cases copy remote A to remote B. If you wanted to
> copy local A to local B with the option enabled, you could do this:
>
>   /ssh:remote:~ $ cp /:/etc/A /:/etc/B     # copies local A to local B
>   /ssh:remote:~ $ cp -b /:/etc/A /:/etc/B
>   ‘/:/etc/A’ is local, but current directory is remote

Right okay.  I guess you are basically saying that this aspect of
Eshell's design turned out to be a bit too clever, and doing things in a
more predictable way, always based on the cwd unless explicitly
otherwise, will probably turn out to be more useful for most users.

-- 
Sean Whitton