From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.emacs.devel Subject: Re: Fix for slow process output processing (please test). Date: 03 Jan 2004 16:12:26 +0100 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1073142971 30955 80.91.224.253 (3 Jan 2004 15:16:11 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sat, 3 Jan 2004 15:16:11 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sat Jan 03 16:16:07 2004 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1AcnVX-0006PF-00 for ; Sat, 03 Jan 2004 16:16:07 +0100 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1AcnVX-000276-00 for ; Sat, 03 Jan 2004 16:16:07 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.24) id 1AcoSz-0004Hm-BE for emacs-devel@quimby.gnus.org; Sat, 03 Jan 2004 11:17:33 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.24) id 1AcoSH-0004GL-Oi for emacs-devel@gnu.org; Sat, 03 Jan 2004 11:16:49 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.24) id 1AcoRk-000480-0d for emacs-devel@gnu.org; Sat, 03 Jan 2004 11:16:48 -0500 Original-Received: from [199.232.41.8] (helo=mx20.gnu.org) by monty-python.gnu.org with esmtp (TLSv1:DES-CBC3-SHA:168) (Exim 4.24) id 1AcoQA-0003ob-4m for emacs-devel@gnu.org; Sat, 03 Jan 2004 11:14:38 -0500 Original-Received: from [62.226.11.147] (helo=localhost.localdomain) by mx20.gnu.org with esmtp (Exim 4.24) id 1AcnSA-0003eB-QM for emacs-devel@gnu.org; Sat, 03 Jan 2004 10:12:39 -0500 Original-Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by localhost.localdomain (8.12.8/8.12.8) with ESMTP id i03FCSwC006428 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 3 Jan 2004 16:12:29 +0100 Original-Received: (from dak@localhost) by localhost.localdomain (8.12.8/8.12.8/Submit) id i03FCSvC006424; Sat, 3 Jan 2004 16:12:28 +0100 Original-To: storm@cua.dk (Kim F. Storm) In-Reply-To: Original-Lines: 52 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.2 Precedence: list List-Id: Emacs development discussions. List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:18972 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:18972 storm@cua.dk (Kim F. Storm) writes: > David.Kastrup@t-online.de (David Kastrup) writes: > > > storm@cua.dk (Kim F. Storm) writes: > > > > > So if you have a process which manages to write a lot of data in one > > > write, emacs should happily read that data with no throttling. > > > > And if we have a process that writes in small chunks but with very > > good CPU utilization (something like dd if=/dev/zero obs=1), then we > > will alternate between reading 8192 (or whatever the pipe size is) and > > 1 byte. Of course vastly preferable to alternating between reading 1 > > byte and 1 byte. > > If I run your own "M-x make-test" command with the patched version of > process.c on GNU/Linux, I get the following measurements: > Pretty good improvement IMHO. > > Of course, if you have examples of things that behave badly with the > patched version (i.e. that ran better without the patch), I'd like > to know. While I have been abysmally absent in testing this patch before it finally got applied, I would like to mention that I now benchmarked the behavior with and without process-adaptive-read-buffering set for a typical preview-latex document. With the patch, the real time for the initial LaTeX run (which is somewhat comint-like) was about half with adaptive read buffering set. And it must be mentioned that I had already trimmed output to the necessary minimum before because I had noticed a heavy speed impact for I/O. The subsequent GhostScript usage as a daemon (where lines of command and very brief responses are passed back and forth between Emacs and GhostScript) was not affected in its operation time. One thing that might be worth mentioning in the variable process-adaptive-read-buffering is that it is conceivable that it would offer no or slightly negative impact on multiprocessor machines where the data generating process can make progress independent of the CPU time Emacs spends on consuming the data. It would be nice if somebody that has a multiprocessor box would test this. Thanks a lot: I guess this change will be quite welcome for all users that use Emacs as shell/terminal/process environment in one way or another. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum