bug#70792: 30.0.50; [PATCH] Add Eshell support for expanding absolute file names within the current remote connection

[Jim Porter <jporterbugs@gmail.com>]

Before I respond to the initial points, I wanted to emphasize one part 
first: my patch is intended to make Eshell behave more like other shells 
and to simplify how users can perform "common" tasks. By "common", I 
mean specifically that, when you've started a remote "session" (by 
cd'ing into a remote host), normal filenames all refer to something on 
that host, just like if you used SSH on a terminal. To refer to 
something on another host, you use remote file name syntax (or /:quoted 
file name syntax to explicitly refer to a local file[1]).

Also, even with this option, absolutely nothing changes about how Eshell 
works when the current directory is local.

On 5/6/2024 11:43 AM, Eli Zaretskii wrote:
> I know this, but we are explicitly talking about _absolute_ file
> names, which normally trivially expand to themselves.  _That_ is the
> problem which I was talking about: you seem to propose a feature where
> an absolute file name is sometimes expanded not to itself, but to a
> remote file name.

Correct. Eshell's transparent remote access forces us to consider a 
question that other shells don't have: how do I refer to a file that's 
absolute *on the current connection*?

Currently, for Lisp-based commands, you have to type the the full remote 
part of the file name again, like "/ssh:user@remote:/some/file". For 
external commands, you have to type just the local part, like 
"/some/file". For commands that could be *either* Lisp-based or external 
(this includes most Eshell built-ins), there's no way to do this. 
(Unless you know exactly how Eshell implements things.)

>>     ##### 2. Change to an absolute directory, stay as root
>> /sudo:root@host:~ # cd /etc; pwd; *pwd
>> /sudo:root@host:/etc
>> /etc
> 
> So you are saying that to chdir to the _local_ /etc I must quote it as
> in "/:/etc", or somesuch?  If not, how do I chdir to a local directory
> by its absolute file name?

If your current working directory is local, "cd /etc" is enough (since 
the current host is the local one). If your current working directory is 
remote, any of the following would change back to the local /etc:

   cd /:/etc
   cd \/etc
   cd "/etc"
   cd '/etc'

>>     ##### 3. Change to the home directory, stay as root
>> /sudo:root@host:~ # cd ~; pwd; *pwd
>> /sudo:root@host:/root
>> /root
> 
> Likewise here: how to chdir to the _local_ home directory? quote it?

If your cwd is local, just "cd ~" is enough. If cwd is remote then you'd 
use "cd /:~".

>>     ##### 4. Write the expanded "~/foo.txt" to the *local* "~/bar.txt".
>>     ##### Using "/:" quoting lets you explicitly name a local file
>> /sudo:root@host:~ # *echo ~/foo.txt > /:~/bar.txt
>> /sudo:root@host:~ # cat bar.txt
>> /bin/cat: bar.txt: No such file or directory
>>
>>     ##### 5. Change to the *local* home directory, stop being root
>> /sudo:root@host:~ # cd /:~; pwd; *pwd
>> /home/jim
>> /home/jim
> 
> That's awful!  Completely un-natural, let alone a lot of typing!

It's only two extra characters compared to the equivalent command that 
all happens on a single host (which I think would be the more-common 
scenario):

   ##### 4b. Write the expanded "~/foo.txt" to the remote "~/bar.txt".
   ##### Using "/:" quoting lets you explicitly name a local file
/sudo:root@host:~ # *echo ~/foo.txt > ~/bar.txt
/sudo:root@host:~ # cat bar.txt
['-c', '/root/foo.txt']

The example I chose is somewhat contrived, of course. I just wanted to 
show off how you can mix local and remote expanded file names in a 
single command for this case.

> Also, am I still able to specify remote file names when my default
> directory is local?  Or do I have to chdir to a remote directory
> first?

Yes. Just type the full remote name, like "/ssh:user@remote:/file".

>>     ##### 6. "bar.txt" ended up here
>> ~ $ cat bar.txt
>> ['-c', '/root/foo.txt']
> 
> Is "/root/foo.txt" a local or remote file name?  (I know you used
> *echo, but the file bar.text has no memory of that.)

At this point, it's just text, so *technically* it's neither. However, 
that text was created from the local portion of a remote file name, so 
the string is local to the remote host where Python was executed. (In 
the example, I used "sudo", so I suppose local/remote are misnomers, but 
the same reasoning applies if you used "ssh".)

>> In the last line above, note that the value we wrote to our file is just
>> the local part.
> 
> And that's considered a feature??

As mentioned above, this is a contrived example to show the lifecycle of 
these names, but yes. The Python command I used just shows off the 
internals. For a more practical example, suppose I want to write the 
word counts of a remote file into a local file. This might actually come 
up in practice: if the remote file is large, I don't want to copy the 
whole thing locally first. With this new option, I could do the following:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > /:~/counts.txt

Today, you could do this like so:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > ~/counts.txt

That looks a bit simpler at a glance (no "/:"), but now there's a 
problem lurking: "~" points to the remote homedir in the first case, and 
the local homedir in the second.

Now suppose a slightly different case. What if I'm in a remote directory 
and want to write this summary file to my remote homedir? With this new 
option, I could type the following:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > ~/counts.txt

Today, you'd need to do this:

   /ssh:user@remote:/somedir $ *wc ~/file.txt > 
/ssh:user@remote:~/counts.txt

Or this:

   /ssh:user@remote:/somedir $ cd /ssh:user@remote:~
   /ssh:user@remote:~ $ *wc file.txt > counts.txt

In both of the "today" cases, you need to type "/ssh:user@remote:~" 
somewhere, which is the thing I'd consider to be unnatural and a lot of 
typing.

>> With this option *disabled* (the default), there are some problems that
>> (in my opinion) make working with remote file names in Eshell even more
>> complex. For example, suppose I'm on a remote host, and want to change
>> to my home directory on that remote. There's not an easy way to do that:
> 
> The simplest solution is to introduce a special command for that, so
> that the user could tell Eshell explicitly whether he/she wants to
> consider him/herself on the local or the remote host.  Similar to what
> we did with rlogin or ssh.

Eshell users already have a command for explicitly moving between local 
and remote hosts: it's just "cd".

I'm not familiar with what we do about rlogin and ssh though. Where 
would I find that info? If someone else has come up with a better way to 
handle a similar scenario, I'd be happy to take a look.

>> Or suppose my cwd is /ssh:user@remote:/somedir. If I run "cat
>> /etc/hosts", Eshell will print my local hosts file. But what if I run
>> "cat -n /etc/hosts"? Eshell's "cat" implementation doesn't understand
>> "-n", so it calls the "cat" on "remote". Now it prints the remote hosts
>> file. You can only predict which hosts files you'll get if you know
>> exactly which flags Eshell's "cat" implementation supports.
> 
> If Eshel knew that I consider myself on the remote, it could have
> modified the logic accordingly, to DTRT.  Without such an explicit
> knowledge, we are _guessing_, and our guesses are bound to be wrong
> sometimes.

Unless I'm misunderstanding what you mean here, Eshell *does* know that 
you consider yourself on the remote. You previously cd'ed into a remote 
directory, expressing your intention to Eshell explicitly. With my 
patch, there's no guesswork here. At least the way I interpret your 
message, this patch does exactly what you suggest: because Eshell knows 
that you consider yourself on the remote (you cd'ed into it) and that 
you chose this new option, it knows that "/foo/bar" should refer to a 
file on the remote system, ensuring that your commands work the same no 
matter whether they're Lisp-based or external programs.

Without using this option, I don't think there's a way to DTRT in 
general. Currently, the string "/foo/bar" is just that, a string. It 
carries no information about the host Emacs should look on. Existing 
commands (whether Lisp-based or external) will just look on the host 
where the process is running. Conceptually, my patch adds annotations to 
these strings so that we can determine authoritatively which host they 
belong to.

[1] Eshell has some code to handle this syntax, but it's built on the 
one of the intended uses for quoted file names. From "Quoted File Names" 
in the Emacs manual: "For example, you can quote a local file name which 
appears remote, to prevent it from being treated as a remote file name."