From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: "Stefan Monnier" Newsgroups: gmane.emacs.devel Subject: Re: Any idea about what makes Emacs slow reading on pipes? Date: Sat, 17 May 2003 22:09:02 -0400 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: <200305180209.h4I292Mp007348@rum.cs.yale.edu> References: <5x1xyye3y4.fsf@kfs2.cua.dk> <5xznllw0wp.fsf@kfs2.cua.dk> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1053223698 26119 80.91.224.249 (18 May 2003 02:08:18 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 18 May 2003 02:08:18 +0000 (UTC) Cc: "Kim F. Storm" Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sun May 18 04:08:16 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19HDay-0006mu-00 for ; Sun, 18 May 2003 04:08:16 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.12 #1 (Debian)) id 19HDk1-0000A4-00 for ; Sun, 18 May 2003 04:17:41 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19HDca-00063x-05 for emacs-devel@quimby.gnus.org; Sat, 17 May 2003 22:09:56 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 19HDc5-0005Rt-00 for emacs-devel@gnu.org; Sat, 17 May 2003 22:09:25 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 19HDbq-0004TQ-00 for emacs-devel@gnu.org; Sat, 17 May 2003 22:09:15 -0400 Original-Received: from rum.cs.yale.edu ([128.36.229.169]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19HDbk-00045x-00 for emacs-devel@gnu.org; Sat, 17 May 2003 22:09:04 -0400 Original-Received: from rum.cs.yale.edu (localhost [127.0.0.1]) by rum.cs.yale.edu (8.12.8/8.12.8) with ESMTP id h4I293x6007350; Sat, 17 May 2003 22:09:03 -0400 Original-Received: (from monnier@localhost) by rum.cs.yale.edu (8.12.8/8.12.8/Submit) id h4I292Mp007348; Sat, 17 May 2003 22:09:02 -0400 X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 Original-To: David.Kastrup@t-online.de (David Kastrup) Original-cc: emacs-devel@gnu.org X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1b5 Precedence: list List-Id: Emacs development discussions. List-Help: List-Post: List-Subscribe: , List-Archive: List-Unsubscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:13961 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:13961 > Actually, this isn't it. Pipes behave just the same, and I now know > why. You won't see this problem on an SMP machine. The writing Indeed my old 266Mhz PII is bi-processor and I don't see it. > process makes a _small_ write, and Emacs, waiting on the receiving end > of it with select, gets woken up immediately and gets to process the > small write. But while Emacs is processing the small write, the > writing process does not get any processing time at all, so it won't > produce anything new until Emacs either goes back to sleep, or is > preempted because of taking too long (for example garbage collecting). Sounds pretty convincing. > So what can we do about this? > > a) use only SMP systems > b) pester Linux developers to be less eager with context switches on > select. I am trying this course now, but it will of course take > time to register and other OS might have similar problems > c) make a super-efficient path for process output arriving in tiny > chunks, since this is what we will be force-fed most of the time. The problem with (c) is that we can't cut many corners for this special case, since we have no idea whether some other output is about to come or not, so we have to do the full processing, just in case. I think the problem is really a kernel issue, and it even sounds like something very vaguely familiar (as in "some kind of textbook example about why you don't want to systematically switch when an event wakes up another process"). I haven't hung out in kernel world for a long time, but I suspect that the problem you describe depends on the kernel version. I would also consider it as a kernel-scheduler bug, since the `dd' process ends up starved. I'd appreciate if you could CC me on any email you might send to the kernel crowd. Stefan