From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: David.Kastrup@t-online.de (David Kastrup) Newsgroups: gmane.emacs.devel Subject: Re: Any idea about what makes Emacs slow reading on pipes? Date: 17 May 2003 02:34:17 +0200 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: References: <5x1xyye3y4.fsf@kfs2.cua.dk> Reply-To: dak@gnu.org NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1053131760 27124 80.91.224.249 (17 May 2003 00:36:00 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sat, 17 May 2003 00:36:00 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sat May 17 02:35:58 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19Gpg6-00073M-00 for ; Sat, 17 May 2003 02:35:58 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.12 #1 (Debian)) id 19Gpoh-0002cP-00 for ; Sat, 17 May 2003 02:44:51 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19GpgK-0006Hi-08 for emacs-devel@quimby.gnus.org; Fri, 16 May 2003 20:36:12 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 19Gpfw-0006G6-00 for emacs-devel@gnu.org; Fri, 16 May 2003 20:35:48 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 19Gpfi-0006Bz-00 for emacs-devel@gnu.org; Fri, 16 May 2003 20:35:35 -0400 Original-Received: from mailout10.sul.t-online.com ([194.25.134.21]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19Gpek-0005sl-00 for emacs-devel@gnu.org; Fri, 16 May 2003 20:34:34 -0400 Original-Received: from fwd07.sul.t-online.de by mailout10.sul.t-online.com with smtp id 19Gpej-0007ne-00; Sat, 17 May 2003 02:34:33 +0200 Original-Received: from localhost.localdomain (520018396234-0001@[62.226.11.155]) by fwd07.sul.t-online.com with esmtp id 19GpeV-1nqVLUC; Sat, 17 May 2003 02:34:19 +0200 Original-Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by localhost.localdomain (8.12.8/8.12.8) with ESMTP id h4H0YI2l032714; Sat, 17 May 2003 02:34:18 +0200 Original-Received: (from dak@localhost) by localhost.localdomain (8.12.8/8.12.8/Submit) id h4H0YHoe032710; Sat, 17 May 2003 02:34:17 +0200 Original-To: storm@cua.dk (Kim F. Storm) In-Reply-To: <5x1xyye3y4.fsf@kfs2.cua.dk> Original-Lines: 78 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50 X-Sender: 520018396234-0001@t-dialin.net X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1b5 Precedence: list List-Id: Emacs development discussions. List-Help: List-Post: List-Subscribe: , List-Archive: List-Unsubscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:13948 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:13948 storm@cua.dk (Kim F. Storm) writes: > David.Kastrup@t-online.de (David Kastrup) writes: > So I do see the "1 byte" reads, but not the slowness. Fine for you. > > I don't know _what_ Emacs does do the pipe/pty that causes only single > > characters to gather/be processed, with a sporadic 1024 packet getting > > through from time to time, but I hate it. Some setting must be on the > > pipe that causes the writer to stall before further writes in many > > cases until Emacs has read the next character. And after some time > > the pipe gets filled properly now and then, with single characters in > > between. > > > > I don't get it. > > Well, it seems that emacs is FAST enough the keep up with your dd > which actually only writes 1 byte at a time, so basically if your > program writes 1 char at a time, emacs will read 1 char at a time if > it can keep up... No, that isn't it. I can pipe the stuff into |cat >/dev/null instead of Emacs, and it will be finished in a whiffy. Emacs must be using some sort of system call that _stops_ dd most of the time from being able to so another write to the pipe until the Linux scheduler has run again (which it does at the rate of about 100 per second, almost exactly the rate at which characters arrive). I am not saying that the current Linux kernel might not play its part in the catastrophic performance here, but the fact is that substituting different readers for Emacs on the delivering pipe yields in vast performance gains of the _writing_ process. > What happens if you try the following version, which forces > start-process to use a pipe rather than a pty: Nothing much, I said already. > But if you really want to speed up things, use a block size > of 1k like this: > "-c" "hexdump -v /dev/zero|dd bs=1k count=100") Oh, come on. The dd was explicitly there to force small writes, since that is the most common situation when talking with applications through a pipe. And that is what makes Emacs dog slow when compared with XEmacs when running things like TeX from within AUCTeX. > Based on this, I would conclude that emacs is behaving _correctly_ > for the input you supply it. Well, it certainly does receive and process all bytes, so of _course_ it is behaving `correctly'. But it causes dd and perhaps Linux to crawl. I am not interested in assigning a culprit, but the fact remains that the implementation causes awful performance. Part of the reason is that Emacs does a lot of partly excessively looking overhead for every accept-process-output or related call (like sit-for). But improving that would not pay off even remotely as well as having a way to let the pipe have better utilization. If Emacs could find a way _not_ to let Linux wake it up until the writing process either is regularly preempted, or has the pipe filled, or is waiting for something other, this would greatly benefit performance by causing Emacs to work on more useful batches than just single characters. We are having here an operating system with a single CPU, and we switch processes here for every byte mainly, and the producing process gets no CPU time for making new bytes while the consuming process is still busy processing the byte it got. Anyhow, I _do_ have the additional suspicion that there is pretty much one "tick" of delay for every of byte involved and that the CPU is mostly idling for that reason: a 600MHz PIII should even with the full processing overhead due to catering for every single character on its own be able to handle more than 100 characters per second. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum