Re: Any idea about what makes Emacs slow reading on pipes?

[David.Kastrup@t-online.de (David Kastrup)]

Richard Stallman <rms@gnu.org> writes:

>     We might change it so that, after stashing the stuff in a
>     buffer, Emacs looks again if more data is available from the
>     same process.

More data will not be available from the same process, since the
process has not had any chance to get CPU time again for generating
more data.  We are talking about a uniprocessor machine here.  The
only way in which more data could become available is if Emacs
_voluntarily_ yielded the CPU instead of processing what is there.
So the question is how to yield the CPU, and for how long, and when
to stop.

The best possibility would be to resume when either a number of
characters is available (a "read" of a certain size can be satisfied),
or a timeout has occured.  Unfortunately, I don't see how to achieve
this without actually reading the pipe out all of the time.  So we do
need a temporary buffer which we serve _fast_.  If it is full, we
process it, if not, we may wait.  If waiting does not yield new
characters, fire up the filter.  How long to wait, depending on the
past history and the fill level of our buffer?  Good question.

>     If so, also read it and stash it in the buffer, too.
> 
> That is worth trying--it might not be too hard to write.  But I
> think the problem is in Linux, so the real solution is to change
> Linux.

I am following this way by discussing this on the Linux kernel list
and am pretty certain that a future version of Linux might be able to
deal with this gracefully.  But there are other operating systems of
which I believe a majority to be afflicted with this
uniprocessor-specific problem, and older Linux systems will remain
with us for a long time.  It would be good if we could equip Emacs
with a few heuristics that would not cause noticeable degradations on
operating systems that would know how to deal with this scheduling
anomaly, but improved matters on the _many_ systems that don't.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum