Re: request for review: Doing direct file I/O in Emacs Lisp

[Eli Zaretskii <eliz@gnu.org>]

> From: David Kastrup <dak@gnu.org>
> Date: 16 May 2004 00:13:57 +0200
> 
> The problem is that I/O using "select" is ready the moment a _single_
> byte is available.

Forgive me for a possibly stupid question, but with applications such
as `cat' with their stdout redirected to a pipe, shouldn't we have
`cat' write the pipe in relatively large blocks, as defined by the
default buffering of its stdout?

If such buffering does take place, then, for this specific scenario,
we should already have the OS reschedule only when a large chunk of
data is written to the pipe.

(I realize that, in general, the reschedule-on-every-byte is a
possibility, but I thought it happens mainly with `dd'-type
applications that write binary data.)

> Perhaps one would need some nicer system calls for telling Linux "ok,
> wake me up immediately if any pipe is _full_.

Almost full, you mean.  If the pipe is full, it's too late, since the
pipe could be written again before the pipe reader actually awakes and
runs, in which case you will lose characters at best and get SIGPIPE
at worst.

Also, this has complications when the pipe writer exits without
filling the pipe (which is the normal case).

> And wake me up
> immediately if there is input on some of [list of files].  Other than
> that, only wake me up if there is input and no other process is
> wanting the CPU".  So we'd need to tell the operating system how
> urgent we want what amount of data on what input to be scheduled for
> processing.

Sounds like a brutal intervention into the OS scheduler, which most
systems will not allow.

It's possible that a much simpler solution would be to have a separate
thread or a helper subprocess read the pipe until a certain amount of
data is available or a timeout expires, and only then present the data
to Emacs's process filter/sentinel.

Again, apologies if this is just line noise.