From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: storm@cua.dk (Kim F. Storm) Newsgroups: gmane.emacs.devel Subject: Re: Any idea about what makes Emacs slow reading on pipes? Date: 17 May 2003 03:50:59 +0200 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: <5x1xyye3y4.fsf@kfs2.cua.dk> References: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1053129198 6715 80.91.224.249 (16 May 2003 23:53:18 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Fri, 16 May 2003 23:53:18 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Sat May 17 01:53:16 2003 Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19Gp0m-0001k7-00 for ; Sat, 17 May 2003 01:53:16 +0200 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.12 #1 (Debian)) id 19Gp9L-0002GR-00 for ; Sat, 17 May 2003 02:02:08 +0200 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19Gp1P-0001rS-00 for emacs-devel@quimby.gnus.org; Fri, 16 May 2003 19:53:55 -0400 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 19Gp0o-00017t-00 for emacs-devel@gnu.org; Fri, 16 May 2003 19:53:18 -0400 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 19Gp0j-0000sg-00 for emacs-devel@gnu.org; Fri, 16 May 2003 19:53:15 -0400 Original-Received: from mail.filanet.dk ([195.215.206.179]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 19Gp0I-0000Nb-00; Fri, 16 May 2003 19:52:47 -0400 Original-Received: from kfs2.cua.dk.cua.dk (unknown [10.1.82.3]) by mail.filanet.dk (Postfix) with SMTP id E15457C012; Sat, 17 May 2003 01:52:37 +0200 (CEST) Original-To: dak@gnu.org In-Reply-To: Original-Lines: 115 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3.50 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1b5 Precedence: list List-Id: Emacs development discussions. List-Help: List-Post: List-Subscribe: , List-Archive: List-Unsubscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:13947 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:13947 David.Kastrup@t-online.de (David Kastrup) writes: > Here is a more elaborate test file > > (defvar test-pattern nil) > (defvar test-start nil) > > (defun test-filter (process string) > (push (cons (length string) (time-since test-start)) test-pattern) > (with-current-buffer (process-buffer process) > (insert-before-markers string))) > > (defun make-test nil (interactive) > (switch-to-buffer (get-buffer-create "*test*")) > (erase-buffer) > (setq test-pattern nil test-start (current-time)) > (set-process-filter (start-process > "test" (current-buffer) "sh" > "-c" "hexdump -v /dev/zero|dd bs=1 count=100k") > #'test-filter)) > With your test program, I get the following result on redhat 6.2 on a 650HMz P3 system: ((87 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 2) (1023 0 2) (1024 0 2) (1024 0 2) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1023 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1024 0 1) (1024 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (304 0 0) (1024 0 0) (1 0 0) (1 0 0) (582 0 0) (1024 0 0) (1024 0 0) (1024 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0) (1 0 0)) So I do see the "1 byte" reads, but not the slowness. > I don't know _what_ Emacs does do the pipe/pty that causes only single > characters to gather/be processed, with a sporadic 1024 packet getting > through from time to time, but I hate it. Some setting must be on the > pipe that causes the writer to stall before further writes in many > cases until Emacs has read the next character. And after some time > the pipe gets filled properly now and then, with single characters in > between. > > I don't get it. Well, it seems that emacs is FAST enough the keep up with your dd which actually only writes 1 byte at a time, so basically if your program writes 1 char at a time, emacs will read 1 char at a time if it can keep up... I guess the occasional 1024 read is because of scheduling of other processes or some such, and because eventually, emacs cannot keep up anymore (when it has to extend the buffer or some such). What happens if you try the following version, which forces start-process to use a pipe rather than a pty: (defun make-test nil (interactive) (switch-to-buffer (get-buffer-create "*test*")) (erase-buffer) (setq test-pattern nil test-start (current-time)) (let ((process-connection-type nil)) (set-process-filter (start-process "test" (current-buffer) "sh" "-c" "hexdump -v /dev/zero|dd bs=1 count=100k") #'test-filter))) It has some effect in my setup, but nothing dramatic. But if you really want to speed up things, use a block size of 1k like this: (defun make-test nil (interactive) (switch-to-buffer (get-buffer-create "*test*")) (erase-buffer) (setq test-pattern nil test-start (current-time)) (let ((process-connection-type nil)) (set-process-filter (start-process "test" (current-buffer) "sh" "-c" "hexdump -v /dev/zero|dd bs=1k count=100") #'test-filter))) On my system, it gives the optimal behaviour: ((35 0 2) (1024 0 2) (1024 0 2) ... (1024 0 2)) Based on this, I would conclude that emacs is behaving _correctly_ for the input you supply it. -- Kim F. Storm http://www.cua.dk