bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks

[Paul Pogonyshev <pogonyshev@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3001 bytes --]

Emacs version: GNU Emacs 28.1.50 (build 1, x86_64-pc-linux-gnu, GTK+
Version 2.24.33, cairo version 1.16.0) of 2022-05-24

I have an internet connection with a relatively large ping, so TRAMP for me
works quite slowly. I notice this when editing and saving files (also when
Emacs stores backup copies), with Magit and otherwise.

I tried to understand why it is _so_ slow for me. I have found out variable
`tramp-verbose' and used it to figure out which commands TRAMP executes on
the remote machine. For testing, I used a real library's (Logview's) buffer
refreshing command  that boils down to this:

    (with-temp-buffer
      (insert-file-contents the-file nil from-somewhere-in-the-middle nil)
      ; do something about the data
      )

In other words, it inserts into a buffer not the whole file, but a part of
it (the reason is irrelevant here).

Well, this results in _eleven_ commands from TRAMP! Here they are:

1) check if connection is alive (`echo are you awake');
2) test if the file exists;
3) creating a temporary file for the chunk to be inserted; I guess it tries
until it finds an unused filename, e.g. here it seems to be done after
`test -e /tmp/tramp.OD3cCu', which doesn't exist;
4) 'touch' on the temporary file, presumably to create it;
5) 'chmod' on the temporary, presumably so that other users cannot read it;
6) copying the requested chunk from the full file into the temporary (using
`dd');
7) finding the real name of the temporary with `readlink';
8) finding attributes of the temporary with `stat';
9) gzipping the temporary for transmition from the remote to the local
machine;
10) testing if the temporary is a directory (WTF?);
11) removing the temporary.

I guess it should be obvious that this is a bit too much for one
`insert-file-contents' call.

Suggested improvements:

* TRAMP should issue just one `stat' command to find out most of the things
about a file: whether it exists, if it is a directory, its real name when
dereferencing links and whatever stats it is used to find now; from `$ stat
--help' this seems to be possible. In other words, TRAMP shouldn't use
simple commands like `test -e': any ping, even nominal, will negate any
gains from using a tad faster command. Instead, if it needs to find
anything about a file, it should ask the remote about as many things as
possible in one go: it is very likely that the additional information will
be needed soon and even if not, this is basically free compared to ping
anyway.

* TRAMP code should prefer the approach "try do something and handle
resulting errors" where possible. For example, don't check if the file
exists, try to read it right away and handle failures properly. Code like
`(when (file-exists-p ...) do-something)' adds an unnecessary command call
and creates a racing condition anyway. Also, error-free requests should be
more frequent, so they should be the main optimization goal. I'm not sure
if it is applicable to TRAMP itself and doesn't come from a higher level,
though.

[-- Attachment #2: Type: text/html, Size: 3519 bytes --]