Excessive redisplay from lots of process output

[Spencer Baugh <sbaugh@janestreet.com>]

Hi emacs-devel,

Emacs calls redisplay every time it reads process output.  This causes
Emacs to become painfully slow when these two conditions are met:

- When redisplay is expensive (e.g. when using ssh X forwarding)

- And when there is frequent process output, say hundreds or thousands
  of times a second (e.g. ERC receiving network traffic, shells
  running commands which log heavily, other things which run in the
  background)

Does Emacs really need to redisplay on every process output?  I've
installed the following change in my Emacs, which has radically
reduced the amount of redisplay performed and improved performance,
and I haven't noticed any adverse effects:

--- a/src/process.c
+++ b/src/process.c
@@ -5804,8 +5804,6 @@ wait_reading_process_output (intmax_t time_limit,
 		  FD_ZERO (&Available);
 		  last_read_channel = channel;
 
-		  if (do_display)
-		    redisplay_preserve_echo_area (12);
 		}
 	      else if (nread == -1 && would_block (errno))
 		;
-- 


I realize that reading process output can trigger filters which can
change window and frame configurations, which in turn means redisplay
needs to happen.  But isn't there some way we could improve this
situation?  Right now these redisplays are causing serious
user-visible performance degradation.

One idea: Could we allow Lisp code to mark a process as not wanting
redisplay after Emacs reads output?  Then packages which receive heavy
amounts of process output could mark their processes in this way.  If
they do rarely need to perform user-visible changes, they could
explicitly call (redisplay) themselves.

Thoughts on this problem and any potential solutions for it?

Thanks,
Spencer Baugh