bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks

[Paul Pogonyshev <pogonyshev@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3649 bytes --]

> It returns "'/tmp/c' -> '/tmp/b'". However, we need "/tmp/a". So we must
> still use "readlink --canonicalize".

According to a quick search, it is possible to merge output of several
shell commands together. This seems to work even with dumb `sh', it's not a
Bash extension:

    $ sh -c '{ stat xxx && readlink xxx; }'

I guess TRAMP could just sth. similar, as I understand it doesn't have to
be strictly one executable call, just one command given to the remote shell.

> Tramps communication with the remote host is like a REPL engine. It
> sends shell commands to the remote hosts, reads the result, and waits
> for the shell prompt. If it doesn't wait for the final shell prompt, it
> is likely that the result or the shell prompt will be seen when reading
> the output of the next command. This confuses. So no, I don't see a
> chance to implement this kind of "asynchronity".

I see parameter `nooutput' to `tramp-send-command'. Couldn't that be used?

Even if not, I could imagine sth. like this:

    (defvar pending-commands nil)
    (defvar reading-output nil)

    (defun send-command (x output-inessential)
      (if output-inessential
          (setf pending-commands (nconc pending-commands (list x)))
        (while reading-output  ; make sure the connection is free for the
next essential command
          (read-next-output-chunk)
          (when (and (not reading-output) pending-commands)
            (do-send-command (pop pending-commands))))
        (do-send-command x)
        (read-output-now)))

    (defun do-send-command (x)
      (really-do-send-it x)
      (setf reading-output t))

    (defun read-output-now ()
      (while reading-output
        (read-next-output-chunk))
      (extract-received-output-from-process-buffer))

    (defun emacs-idling ()  ; hooked up using `run-with-idle-timer' or
something like that
      (cond (reading-output
             (read-next-output-chunk))
            (pending-commands
             (do-send-command (pop pending-commands)))))

    (defun read-next-output-chunk ()
      (when reading-output
        (do-read-output-chunk)  ; this should be non-blocking
        (when (end-of-command-output)
          (setf reading-output nil))))

Paul

On Mon, 4 Jul 2022 at 13:19, Michael Albinus <michael.albinus@gmx.de> wrote:

> Paul Pogonyshev <pogonyshev@gmail.com> writes:
>
> Hi Paul,
>
> >> Doing it asynchronously would require a second connection to the remote
> >> host. Performance would rather degrade.
> >
> > Maybe not really asynchronously, just let it return early, not waiting
> > for the result? I'm not sure how TRAMP receives responses, but can it
> > just keep executing commands sequentially, as now, but give control
> > back to the caller in case of commands where the result doesn't really
> > matter (cleanup, e.g. deleting a temporary file). Of course, this
> > means that if an "important" command is issued right away, it has to
> > wait for the response to the previous inessential command. But when
> > such an inessential command is the last in a batch, this waiting would
> > be effectively merged with Emacs' idling in the normal UI command
> > loop, making things more responsive for the user.
>
> Tramps communication with the remote host is like a REPL engine. It
> sends shell commands to the remote hosts, reads the result, and waits
> for the shell prompt. If it doesn't wait for the final shell prompt, it
> is likely that the result or the shell prompt will be seen when reading
> the output of the next command. This confuses. So no, I don't see a
> chance to implement this kind of "asynchronity".
>
> > Paul
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 4754 bytes --]