From: Michael Albinus <michael.albinus@gmx.de>
To: Paul Pogonyshev <pogonyshev@gmail.com>
Cc: 56342@debbugs.gnu.org
Subject: bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks
Date: Sat, 02 Jul 2022 17:58:25 +0200 [thread overview]
Message-ID: <8735fjh5ge.fsf@gmx.de> (raw)
In-Reply-To: <CAG7BpapEuJqV6HiBQJt=ABKc1uDYH2LpNmwkpSxpdNqcTks7gA@mail.gmail.com> (Paul Pogonyshev's message of "Fri, 1 Jul 2022 19:14:08 +0200")
Paul Pogonyshev <pogonyshev@gmail.com> writes:
Hi Paul,
> 1) check if connection is alive (`echo are you awake');
> 2) test if the file exists;
> 3) creating a temporary file for the chunk to be inserted; I guess it
> tries until it finds an unused filename, e.g. here it seems to be done
> after `test -e /tmp/tramp.OD3cCu', which doesn't exist;
> 4) 'touch' on the temporary file, presumably to create it;
> 5) 'chmod' on the temporary, presumably so that other users cannot
> read it;
> 6) copying the requested chunk from the full file into the temporary
> (using `dd');
> 7) finding the real name of the temporary with `readlink';
> 8) finding attributes of the temporary with `stat';
> 9) gzipping the temporary for transmition from the remote to the local
> machine;
> 10) testing if the temporary is a directory (WTF?);
> 11) removing the temporary.
>
> I guess it should be obvious that this is a bit too much for one
> `insert-file-contents' call.
In general, I agree. However, some of the commands are caused by
primitive file operations, like file-exists-p. Tramp cannot know what
will be the next call, and it doesn't have all the opportunities to
optimize, compared with the overall picture you see in the eleven steps.
> Suggested improvements:
>
> * TRAMP should issue just one `stat' command to find out most of the
> things about a file: whether it exists, if it is a directory, its real
> name when dereferencing links and whatever stats it is used to find
> now; from `$ stat --help' this seems to be possible. In other words,
> TRAMP shouldn't use simple commands like `test -e': any ping, even
> nominal, will negate any gains from using a tad faster command.
> Instead, if it needs to find anything about a file, it should ask the
> remote about as many things as possible in one go: it is very likely
> that the additional information will be needed soon and even if not,
> this is basically free compared to ping anyway.
Not all remote hosts carry a stat command, and not all existing stat's
are GNU compatible. But yes, if possible, Tramp shall gather as much
information in one run, and cache the results for further use.
I will see what could be done. Will come back with a proposal next days
(note that this will be for Emacs 29, ie git master).
> * TRAMP code should prefer the approach "try do something and handle
> resulting errors" where possible. For example, don't check if the file
> exists, try to read it right away and handle failures properly. Code
> like `(when (file-exists-p ...) do-something)' adds an unnecessary
> command call and creates a racing condition anyway. Also, error-free
> requests should be more frequent, so they should be the main
> optimization goal. I'm not sure if it is applicable to TRAMP itself
> and doesn't come from a higher level, though.
Indeed, this is not Tramp's responsibility. Tramp is a stupid
library. If there is a call for file-exists-p, it must return the
answer. It doesn't know what will be the next request. So I'm rather
pesimistic that Tramp can improve here.
Best regards, Michael.
next prev parent reply other threads:[~2022-07-02 15:58 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-01 17:14 bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks Paul Pogonyshev
2022-07-02 15:58 ` Michael Albinus [this message]
2022-07-02 18:14 ` Paul Pogonyshev
2022-07-03 12:16 ` Michael Albinus
2022-07-03 14:00 ` Paul Pogonyshev
2022-07-03 18:47 ` Michael Albinus
2022-07-03 19:52 ` Paul Pogonyshev
2022-07-04 11:19 ` Michael Albinus
2022-07-04 14:42 ` Paul Pogonyshev
2022-07-04 16:30 ` Michael Albinus
2022-07-26 8:00 ` Paul Pogonyshev
2022-07-26 14:18 ` Michael Albinus
2022-07-26 16:17 ` Paul Pogonyshev
2022-07-26 17:51 ` Michael Albinus
2022-08-01 20:20 ` Paul Pogonyshev
2022-08-02 14:23 ` Michael Albinus
2022-07-04 10:33 ` Michael Albinus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8735fjh5ge.fsf@gmx.de \
--to=michael.albinus@gmx.de \
--cc=56342@debbugs.gnu.org \
--cc=pogonyshev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).