From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: David.Kastrup@t-online.de (David Kastrup) Newsgroups: gmane.emacs.devel Subject: Re: Fix for slow process output processing (please test). Date: 16 Dec 2003 12:51:45 +0100 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1071576941 2817 80.91.224.253 (16 Dec 2003 12:15:41 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Tue, 16 Dec 2003 12:15:41 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Tue Dec 16 13:15:38 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1AWE70-0001TP-00 for ; Tue, 16 Dec 2003 13:15:38 +0100 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1AWE6z-0000Kg-00 for ; Tue, 16 Dec 2003 13:15:38 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AWEqV-0007sd-C4 for emacs-devel@quimby.gnus.org; Tue, 16 Dec 2003 08:02:39 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AWElz-0005uv-1w for emacs-devel@gnu.org; Tue, 16 Dec 2003 07:57:59 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AWEkl-0004u9-Ig for emacs-devel@gnu.org; Tue, 16 Dec 2003 07:57:14 -0500 Original-Received: from [194.25.134.20] (helo=mailout08.aul.t-online.de) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AWEi7-0003r7-GT for emacs-devel@gnu.org; Tue, 16 Dec 2003 07:53:59 -0500 Original-Received: from fwd02.aul.t-online.de by mailout08.aul.t-online.de with smtp id 1AWDkW-0004yy-03; Tue, 16 Dec 2003 12:52:24 +0100 Original-Received: from localhost.localdomain (GzMSo+ZrYeTttvvTN4a4jzObIBeRQYeIYVYkOchYmtvbQVFBAEXlYH@[217.80.160.125]) by fwd02.sul.t-online.com with esmtp id 1AWDk3-15GvpI0; Tue, 16 Dec 2003 12:51:55 +0100 Original-Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by localhost.localdomain (8.12.8/8.12.8) with ESMTP id hBGBpqTT002935 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 16 Dec 2003 12:51:53 +0100 Original-Received: (from dak@localhost) by localhost.localdomain (8.12.8/8.12.8/Submit) id hBGBplwL002931; Tue, 16 Dec 2003 12:51:47 +0100 Original-To: storm@cua.dk (Kim F. Storm) In-Reply-To: Original-Lines: 33 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50 X-Seen: false X-ID: GzMSo+ZrYeTttvvTN4a4jzObIBeRQYeIYVYkOchYmtvbQVFBAEXlYH X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Emacs development discussions. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:18713 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:18713 storm@cua.dk (Kim F. Storm) writes: > David.Kastrup@t-online.de (David Kastrup) writes: > > > > David Kastrup and others have reported very slow processing of process > > > output, particularly with the Linux kernel's scheduler giving emacs > > > too much attention while starving the processing producing the output. > > > > > > Below is a patch which introduces a small delay on reading output from > > > such processes, without using global delays in processing other events > > > (this is done by temporarily removing the process' file descriptor from > > > the call to select and use a short timeout on the select instead. > > > > I have glanced over that patch. There is one problem I see with it: > > if indeed the generating process is able to produce lots of data with > > a single write call, Emacs will be throttling its input without a > > purpose. > > With the patch, emacs only throttles if it reads less than 1024 bytes > at a time from the process -- i.e. if the process is unable to produce > data fast enough to fill emacs's read buffer. > > So if you have a process which manages to write a lot of data in one > write, emacs should happily read that data with no throttling. And if we have a process that writes in small chunks but with very good CPU utilization (something like dd if=/dev/zero obs=1), then we will alternate between reading 8192 (or whatever the pipe size is) and 1 byte. Of course vastly preferable to alternating between reading 1 byte and 1 byte. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum