From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: request for review: Doing direct file I/O in Emacs Lisp Date: 16 May 2004 08:41:20 +0200 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: <87isf0vjtl.fsf@emptyhost.emptydomain.de> Reply-To: Eli Zaretskii NNTP-Posting-Host: deer.gmane.org X-Trace: sea.gmane.org 1084686245 25076 80.91.224.253 (16 May 2004 05:44:05 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 16 May 2004 05:44:05 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sun May 16 07:44:00 2004 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1BPERM-0007Yq-00 for ; Sun, 16 May 2004 07:44:00 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1BPERM-0008IG-00 for ; Sun, 16 May 2004 07:44:00 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1BPEOF-00040A-PN for emacs-devel@quimby.gnus.org; Sun, 16 May 2004 01:40:47 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.34) id 1BPENI-0003jn-Ai for emacs-devel@gnu.org; Sun, 16 May 2004 01:39:48 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.34) id 1BPEMl-0003cn-Sv for emacs-devel@gnu.org; Sun, 16 May 2004 01:39:47 -0400 Original-Received: from [207.232.27.5] (helo=WST0054) by monty-python.gnu.org with asmtp (Exim 4.34) id 1BPEMl-0003bt-Ed; Sun, 16 May 2004 01:39:15 -0400 Original-To: David Kastrup In-reply-to: (message from David Kastrup on 16 May 2004 00:13:57 +0200) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.4 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:23520 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:23520 > From: David Kastrup > Date: 16 May 2004 00:13:57 +0200 > > The problem is that I/O using "select" is ready the moment a _single_ > byte is available. Forgive me for a possibly stupid question, but with applications such as `cat' with their stdout redirected to a pipe, shouldn't we have `cat' write the pipe in relatively large blocks, as defined by the default buffering of its stdout? If such buffering does take place, then, for this specific scenario, we should already have the OS reschedule only when a large chunk of data is written to the pipe. (I realize that, in general, the reschedule-on-every-byte is a possibility, but I thought it happens mainly with `dd'-type applications that write binary data.) > Perhaps one would need some nicer system calls for telling Linux "ok, > wake me up immediately if any pipe is _full_. Almost full, you mean. If the pipe is full, it's too late, since the pipe could be written again before the pipe reader actually awakes and runs, in which case you will lose characters at best and get SIGPIPE at worst. Also, this has complications when the pipe writer exits without filling the pipe (which is the normal case). > And wake me up > immediately if there is input on some of [list of files]. Other than > that, only wake me up if there is input and no other process is > wanting the CPU". So we'd need to tell the operating system how > urgent we want what amount of data on what input to be scheduled for > processing. Sounds like a brutal intervention into the OS scheduler, which most systems will not allow. It's possible that a much simpler solution would be to have a separate thread or a helper subprocess read the pipe until a certain amount of data is available or a timeout expires, and only then present the data to Emacs's process filter/sentinel. Again, apologies if this is just line noise.