From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.emacs.devel Subject: Re: emacs communication with subprocess is slow Date: Sat, 04 Oct 2014 00:04:40 +0200 Organization: Organization?!? Message-ID: <878ukwhksn.fsf@fencepost.gnu.org> References: <851tqpf3e1.fsf@stephe-leake.org> <83zjddgg1s.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1412373914 5924 80.91.229.3 (3 Oct 2014 22:05:14 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 3 Oct 2014 22:05:14 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Oct 04 00:05:09 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1XaAyJ-00008G-OX for ged-emacs-devel@m.gmane.org; Sat, 04 Oct 2014 00:05:07 +0200 Original-Received: from localhost ([::1]:41490 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XaAyJ-00084j-BH for ged-emacs-devel@m.gmane.org; Fri, 03 Oct 2014 18:05:07 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:43792) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XaAyB-0007zz-Bx for emacs-devel@gnu.org; Fri, 03 Oct 2014 18:05:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XaAy5-0007G6-WF for emacs-devel@gnu.org; Fri, 03 Oct 2014 18:04:59 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:59414) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XaAy5-0007Fw-MT for emacs-devel@gnu.org; Fri, 03 Oct 2014 18:04:53 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1XaAy2-0008RF-R3 for emacs-devel@gnu.org; Sat, 04 Oct 2014 00:04:50 +0200 Original-Received: from x2f43055.dyn.telefonica.de ([2.244.48.85]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 04 Oct 2014 00:04:50 +0200 Original-Received: from dak by x2f43055.dyn.telefonica.de with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 04 Oct 2014 00:04:50 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 72 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: x2f43055.dyn.telefonica.de X-Face: 2FEFf>]>q>2iw=B6, xrUubRI>pR&Ml9=ao@P@i)L:\urd*t9M~y1^:+Y]'C0~{mAl`oQuAl \!3KEIp?*w`|bL5qr,H)LFO6Q=qx~iH4DN; i"; /yuIsqbLLCh/!U#X[S~(5eZ41to5f%E@'ELIi$t^ Vc\LWP@J5p^rst0+('>Er0=^1{]M9!p?&:\z]|;&=NP3AhB!B_bi^]Pfkw User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux) Cancel-Lock: sha1:1Km6BpCNVWcGcbok0zQzAUCc5+Y= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:174959 Archived-At: Eli Zaretskii writes: >> From: Stephen Leake >> Date: Fri, 03 Oct 2014 12:51:18 -0500 >> >> This may be related to http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18420 >> >> I'm using Emacs pretest 24.3.93, with the patch from 18240, on Windows 7. >> >> I've run into another problem with Emacs communication with external >> processes; there seems to be a delay on every task switch. This shows up >> both when sending data to the subprocess, and when reading from it. >> >> I haven't tested this on Debian yet. >> >> The delay is not present in Emacs 24.3 on Windows 7. >> >> When sending the contents of an Emacs buffer (source code to be parsed) >> to the subprocess, if the read buffer in the subprocess is >> large enough to hold the entire contents, no delay is apparent. However, >> if the buffer is smaller, so that several reads are needed, then a delay >> appears. > > Sorry, I don't understand: Emacs 24.3 would simply deaqdlock in this > situation, as we have established in bug #18420. But you say that not > only does it not hang, it is even faster. What am I missing? > > And what exactly does "large enough" mean here? What sizes are we > talking about? > > Also, can you quantify the delays? > >> I can use a large buffer in my subprocess for read as a workaround, but >> I have no control over the read buffer in Emacs (perhaps that could be >> added?). > > You could try playing with 2 parameters that currently are fixed: the > size of the pipe buffer (set by pipe2 in w32.c, where we use 0 which > AFAIK defaults to 4KB); and the delay used by send_process in > process.c when it gets EAGAIN/EWOULDBLOCK from the 'write' call > (currently 20 milliseconds). > >> Should I reopen 18240, or start a new bug? > > A new one, of course. This is an entirely different issue. This sounds like the culprit may be process-adaptive-read-buffering is a variable defined in `C source code'. Its value is t Documentation: If non-nil, improve receive buffering by delaying after short reads. On some systems, when Emacs reads the output from a subprocess, the output data is read in very small blocks, potentially resulting in very poor performance. This behavior can be remedied to some extent by setting this variable to a non-nil value, as it will automatically delay reading from such processes, to allow them to produce more output before Emacs tries to read it. If the value is t, the delay is reset after each write to the process; any other non-nil value means that the delay is not reset on write. The variable takes effect when `start-process' is called. This variable is a crutch intended to reduce the number of expensive context switches on single processor systems with a scheduler that ends up using a pipe as a synchronous context switcher that requires a context switch for every byte. It may be worth trying to disable it by default and hope that schedulers have improved since it was necessary for reasonable performance. -- David Kastrup