Re: Excessive redisplay from lots of process output

[Eli Zaretskii <eliz@gnu.org>]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: azeng@janestreet.com
> Date: Fri, 17 Feb 2023 10:16:11 -0500
> 
> Emacs calls redisplay every time it reads process output.  This causes
> Emacs to become painfully slow when these two conditions are met:
> 
> - When redisplay is expensive (e.g. when using ssh X forwarding)
> 
> - And when there is frequent process output, say hundreds or thousands
>   of times a second (e.g. ERC receiving network traffic, shells
>   running commands which log heavily, other things which run in the
>   background)

This cannot be the whole story, because usually redisplay exits almost
immediately after being called, having determined that nothing needs
to be changed on display.

So the important part of your situation that you didn't disclose is:
why does Emacs in your case perform the expensive redisplay each time
redisplay is entered?  Can you figure that out by stepping through
redisplay_internal?  (Let me know if you need guidance.)

Not sure here is the right place for this kind of discussion, btw.  It
might be better to submit a bug report with all the relevant details,
and discuss this on the bug tracker.

> I realize that reading process output can trigger filters which can
> change window and frame configurations, which in turn means redisplay
> needs to happen.  But isn't there some way we could improve this
> situation?  Right now these redisplays are causing serious
> user-visible performance degradation.

There are many variables and flags Emacs uses to decide when something
changed that needs the display to be updated.  Those should generally
prevent Emacs from running the expensive parts of redisplay if they
are unnecessary.  The question is why this doesn't work in your case.

IOW, it is better to consider this as some specific failure in some
particular situation, and find why it happens, instead of considering
this as some general failure of the Emacs design.  Design-wise, things
are fine; it's something in your specific case that could fail the
existing mechanisms, and the question is: what is that and why it
fails?

> One idea: Could we allow Lisp code to mark a process as not wanting
> redisplay after Emacs reads output?  Then packages which receive heavy
> amounts of process output could mark their processes in this way.  If
> they do rarely need to perform user-visible changes, they could
> explicitly call (redisplay) themselves.

You are thinking in the wrong direction.  You assume that if redisplay
is called frequently, it will necessarily slow down Emacs.  That
assumption is false: redisplay works very hard to avoid expensive
processing when it can.  So reducing the frequency of calls to
redisplay is not the right direction to make progress in your and
other similar cases.