bug#53041: 29.0.50; TRAMP spins the CPU by polling the child processes without a delay

bug#53041: 29.0.50; TRAMP spins the CPU by polling the child processes without a delay

[Dima Kogan]

Hi. I use TRAMP regularly, and I often see it redline my CPU, which
shouldn't be happening.

The cause in all cases I've seen is TRAMP expecting some output from the
child process, and looking for this output in a delay-less loop. For
instance (tramp-process-one-action) looks like this:

  (defun tramp-process-one-action (proc vec actions)
      ....
    (while (not found)
      (while (tramp-accept-process-output proc 0))
      .... )

The (while (tramp-accept-process-output proc 0)) form does

  Read all available data; returns immediately if none is available

So here we spin the CPU until there's some data to look at AND until the
incoming data meets some condition we're looking for. In order to not
spin, at least one of the (tramp-accept-process-output) calls needs to
block. The simplest thing to do to fix this is to replace

  (while (tramp-accept-process-output proc 0))

with

  (tramp-accept-process-output proc nil)

Here we block until we get SOME data back. I think this is probably
good-enough, since the outer loop will get more data, if it's needed. If
we really want to replace the original logic with blocking, we can do
this instead:

  (let (timeout)
    (while 
        (prog1
            (tramp-accept-process-output proc timeout)
          (setq timeout 0))))

Either one of these makes most of these issues disappear. There are more
places in the code where we call (tramp-accept-process-output ... 0),
and I think they're all wrong: we should always block. I can send a
patch, but let's agree on the approach first. My preference is to
replace all the (while (tramp-accept-process-output proc 0)) with
(tramp-accept-process-output proc nil) unless there's a specific reason
not to.

One easy way to reproduce one such behavior:

1. Start up emacs
2. open /ssh:SERVER:FILE
3. Break the network connection (I'm on a laptop. Leaving the wifi area
   is enough)
4. Try to type into the buffer visiting FILE
5. See emacs block the user while spinning the CPU.

Thanks

bug#53041: 29.0.50; TRAMP spins the CPU by polling the child processes without a delay

[Michael Albinus]

Dima Kogan <dima@secretsauce.net> writes:

> Hi.

Hi Dima,

> I use TRAMP regularly, and I often see it redline my CPU, which
> shouldn't be happening.
>
> The cause in all cases I've seen is TRAMP expecting some output from the
> child process, and looking for this output in a delay-less loop. For
> instance (tramp-process-one-action) looks like this:
>
>   (defun tramp-process-one-action (proc vec actions)
>       ....
>     (while (not found)
>       (while (tramp-accept-process-output proc 0))
>       .... )
>
> The (while (tramp-accept-process-output proc 0)) form does
>
>   Read all available data; returns immediately if none is available
>
> So here we spin the CPU until there's some data to look at AND until the
> incoming data meets some condition we're looking for. In order to not
> spin, at least one of the (tramp-accept-process-output) calls needs to
> block. The simplest thing to do to fix this is to replace
>
>   (while (tramp-accept-process-output proc 0))
>
> with
>
>   (tramp-accept-process-output proc nil)
>
> Here we block until we get SOME data back. I think this is probably
> good-enough, since the outer loop will get more data, if it's needed. If
> we really want to replace the original logic with blocking, we can do
> this instead:
>
>   (let (timeout)
>     (while
>         (prog1
>             (tramp-accept-process-output proc timeout)
>           (setq timeout 0))))
>
> Either one of these makes most of these issues disappear. There are more
> places in the code where we call (tramp-accept-process-output ... 0),
> and I think they're all wrong: we should always block. I can send a
> patch, but let's agree on the approach first. My preference is to
> replace all the (while (tramp-accept-process-output proc 0)) with
> (tramp-accept-process-output proc nil) unless there's a specific reason
> not to.
>
> One easy way to reproduce one such behavior:
>
> 1. Start up emacs
> 2. open /ssh:SERVER:FILE
> 3. Break the network connection (I'm on a laptop. Leaving the wifi area
>    is enough)
> 4. Try to type into the buffer visiting FILE
> 5. See emacs block the user while spinning the CPU.

This was discussed several times already. The most recent discussion wrt
Tramp starts at <https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00301.html>.

The pattern (while (accept-process-output p) was proposed by Stefan
Monnier in <https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00338.html>,
so this is used in Tramp. I do not want to reopen this can of worms, really.

To fix your problem of a broken connection, the Tramp manual recommends
to add "ServerAliveInterval 5" in your ~/.ssh/config, see (info "(tramp)
Frequently Asked Questions") . Additionally, you might set "ServerAliveCountMax 2".

> Thanks

Best regards, Michael.

bug#53041: 29.0.50; TRAMP spins the CPU by polling the child processes without a delay

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> The pattern (while (accept-process-output p) was proposed by Stefan
> Monnier in
> <https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00338.html>,
> so this is used in Tramp. I do not want to reopen this can of worms, really.
>
> To fix your problem of a broken connection, the Tramp manual recommends
> to add "ServerAliveInterval 5" in your ~/.ssh/config, see (info "(tramp)
> Frequently Asked Questions") . Additionally, you might set
> "ServerAliveCountMax 2".

If I understand correctly, this means that we won't be doing anything
further in this bug report, and I'm therefore closing it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53041: 29.0.50; TRAMP spins the CPU by polling the child processes without a delay

[Dima Kogan]

Michael Albinus <michael.albinus@gmx.de> writes:

> This was discussed several times already. The most recent discussion wrt
> Tramp starts at <https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00301.html>.
>
> The pattern (while (accept-process-output p) was proposed by Stefan
> Monnier in <https://lists.gnu.org/archive/html/emacs-devel/2019-01/msg00338.html>,
> so this is used in Tramp. I do not want to reopen this can of worms, really.
>
> To fix your problem of a broken connection, the Tramp manual recommends
> to add "ServerAliveInterval 5" in your ~/.ssh/config, see (info "(tramp)
> Frequently Asked Questions") . Additionally, you might set "ServerAliveCountMax 2".

Thanks for the links, Michael. I'll dogfood some patches for a while,
and we can maybe talk about it later if those consistently work well.

Thanks