bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks

[Paul Pogonyshev <pogonyshev@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3976 bytes --]

> This is a sanity check. It avoids to hang in a blocked connection,
> because this special command is surrounded by a timeout of 10 sec. Other
> commands w/o this protection could hang forever. See also the comment in
> tramp-maybe-open-connection.

Maybe you could then single out commands that are supposed to return result
nearly-instantly anyway, e.g. `stat' or `test' (though the latter I have
suggested to replace with `stat' before). If they fail to produce the
result in 10 seconds, the connection can be assumed dead, just as with
`echo are you awake'. Commands that involve transmitting (potentially)
large amount of data this or that way can work as now. So, you would
usually avoid one extra command (which with a high ping means sth. like 0.2
s, already pretty noticeable in UI) and achieve the same result.

Sanity checks are good. but not if they visibly slow down normal operation.

> If a caller can live w/o a valid result of file-exists-p, it shouldn't
> call it. Everything else is too sophisticated and good for trouble, I
believe.

It might be easier to convince the rest of Emacs developers to use an extra
variable to possibly skip `file-exists-p' on only remote files than to
remove the call altogether. Though in either case it is required that
`file-missing' error is handled properly which is, incidentally, easier to
test if the call to `file-exists-p' is dropped. A target here would be
`insert-file-contents' and whatever it calls, and this is a part of Emacs,
not some external library.

Yet another idea: removing temporary should be done asynchronously (I
haven't checked, maybe it's already the case, but likely not). The caller
doesn't really care about call result and even if it has succeeded. Again,
not sure if this is easy to achieve interface-wise, maybe it's from a
higher level. But I suspect it's something like `with-temp-file' and can be
optimized: one-time optimization are worth it even if that costs
readability.

Paul


On Sun, 3 Jul 2022 at 14:16, Michael Albinus <michael.albinus@gmx.de> wrote:

> Paul Pogonyshev <pogonyshev@gmail.com> writes:
>
> Hi Paul,
>
> > Some more thoughts. Why does it even need `echo are you awake'? It's a
> > network connection, it can still fail even if it worked fine 1 ms
> > before when you checked. So, why not just let the first command fail
> > if the connection is dead and restart the connection if it fails in
> > such a way as to suspect that it is dead (i.e. no output)? Maybe limit
> > this to read commands.
>
> This is a sanity check. It avoids to hang in a blocked connection,
> because this special command is surrounded by a timeout of 10 sec. Other
> commands w/o this protection could hang forever. See also the comment in
> tramp-maybe-open-connection.
>
> > A way to let higher-level code avoid certain `file-exists-p' calls:
> > add a dynamic variable that tells TRAMP to skip certain commands if
> > the result is not available from a cache. Something similar to
> > `process-file-side-effects'. Calling code could then do sth. like
> > this:
> >
> >     (when (let ((tramp-may-skip-if-not-cached `((file-exists-p unknown
> > ,file))))
> >             (file-exists-p file))  ; TRAMP will return t or nil if it
> > knows or 'unknown if not cached; for local files there is no effect
> >       ...)
> >
> > Suggested semantics: list of (FUNCTION INSTANT-RESULT-IF-NOT-CACHED
> > ARGUMENT...). Any element of the list with unknown function name etc.
> > would be simply ignored.
> >
> > Code that doesn't let-bind this variable will behave as before. Code
> > that cares can be optimized.
>
> If a caller can live w/o a valid result of file-exists-p, it shouldn't
> call it. Everything else is too sophisticated and good for trouble, I
> believe.
>
> In general, packages shall not care what's the implementation of a given
> function like file-exists-p. If they care, they could still use
> file-remote-p in order to distinguish.
>
> > Paul
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 4895 bytes --]