From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.emacs.devel Subject: Re: request for review: Doing direct file I/O in Emacs Lisp Date: 16 May 2004 19:46:43 +0200 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: <87isf0vjtl.fsf@emptyhost.emptydomain.de> NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1084729705 28440 80.91.224.253 (16 May 2004 17:48:25 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 16 May 2004 17:48:25 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sun May 16 19:48:19 2004 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1BPPkJ-0003gL-00 for ; Sun, 16 May 2004 19:48:19 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1BPPkJ-0000t0-00 for ; Sun, 16 May 2004 19:48:19 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1BPPk9-0006lj-Am for emacs-devel@quimby.gnus.org; Sun, 16 May 2004 13:48:09 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.34) id 1BPPk2-0006kO-5s for emacs-devel@gnu.org; Sun, 16 May 2004 13:48:02 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.34) id 1BPPjV-0006bY-EC for emacs-devel@gnu.org; Sun, 16 May 2004 13:48:00 -0400 Original-Received: from [199.232.76.164] (helo=fencepost.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1BPPjV-0006bI-3V for emacs-devel@gnu.org; Sun, 16 May 2004 13:47:29 -0400 Original-Received: from localhost ([127.0.0.1] helo=lola.goethe.zz) by fencepost.gnu.org with esmtp (Exim 4.34) id 1BPPjK-0000YG-3A; Sun, 16 May 2004 13:47:28 -0400 Original-To: Eli Zaretskii In-Reply-To: User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50 Original-Lines: 67 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:23542 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:23542 Eli Zaretskii writes: > > From: David Kastrup > > Date: 16 May 2004 00:13:57 +0200 > > > > The problem is that I/O using "select" is ready the moment a _single_ > > byte is available. > > Forgive me for a possibly stupid question, but with applications such > as `cat' with their stdout redirected to a pipe, shouldn't we have > `cat' write the pipe in relatively large blocks, as defined by the > default buffering of its stdout? cat will probably write in lines when writing to a pseudo tty. Anyway, it is not as much cat I am concerned with mostly: the most common problem are applications with small buffer size running in comint mode. Running TeX inside of an Emacs buffer is slow, easily 30% slower than running it in an XTerm. And it is the console I/O that causes this. > > Perhaps one would need some nicer system calls for telling Linux > > "ok, wake me up immediately if any pipe is _full_. > > Almost full, you mean. If the pipe is full, it's too late, since > the pipe could be written again before the pipe reader actually > awakes and runs, in which case you will lose characters at best and > get SIGPIPE at worst. I recommend that you look up the semantics of Unix pipes again. You seem to labor under the delusion that there is a constant race condition going on. If the pipe is filled, the writing process will simply get blocked. SIGPIPE is never raised unless the reading process has _closed_ its end of the pipe, for example by exiting. > Also, this has complications when the pipe writer exits without > filling the pipe (which is the normal case). Again, I don't know what kind of pipes you are talking of here. A read call on a pipe will block the reader until enough data is available, or until the writer has closed its end of the pipe. > > And wake me up immediately if there is input on some of [list of > > files]. Other than that, only wake me up if there is input and no > > other process is wanting the CPU". So we'd need to tell the > > operating system how urgent we want what amount of data on what > > input to be scheduled for processing. > > Sounds like a brutal intervention into the OS scheduler, which most > systems will not allow. Intervention? Telling the scheduler where I expect interactive response, and where I expect batch processing? That is no "intervention", it is information. > It's possible that a much simpler solution would be to have a > separate thread or a helper subprocess read the pipe until a certain > amount of data is available or a timeout expires, and only then > present the data to Emacs's process filter/sentinel. Maybe. If we had some fast preprocessor that would score only a moderate penalty when the operating system schedules it all the time, this might help quite a bit. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum