From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stephen Leake Newsgroups: gmane.emacs.bugs Subject: bug#18420: 24.3; interaction with external process hangs emacs Date: Mon, 08 Sep 2014 23:18:39 -0500 Message-ID: <8538c1zasg.fsf@stephe-leake.org> References: <85y4tvn9ek.fsf@stephe-leake.org> <83fvg3cueu.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1410238424 2891 80.91.229.3 (9 Sep 2014 04:53:44 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 9 Sep 2014 04:53:44 +0000 (UTC) Cc: 18420@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue Sep 09 06:53:37 2014 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1XRCuh-0001uO-Rr for geb-bug-gnu-emacs@m.gmane.org; Tue, 09 Sep 2014 06:20:20 +0200 Original-Received: from localhost ([::1]:46904 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XRCug-0005HI-Rb for geb-bug-gnu-emacs@m.gmane.org; Tue, 09 Sep 2014 00:20:18 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39765) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XRCuX-0005Ey-Oc for bug-gnu-emacs@gnu.org; Tue, 09 Sep 2014 00:20:15 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XRCuR-0004YB-QR for bug-gnu-emacs@gnu.org; Tue, 09 Sep 2014 00:20:09 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:42741) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XRCuR-0004We-NV for bug-gnu-emacs@gnu.org; Tue, 09 Sep 2014 00:20:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1XRCuQ-0003fT-Sq for bug-gnu-emacs@gnu.org; Tue, 09 Sep 2014 00:20:03 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stephen Leake Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Sep 2014 04:20:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 18420 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 18420-submit@debbugs.gnu.org id=B18420.141023634314022 (code B ref 18420); Tue, 09 Sep 2014 04:20:02 +0000 Original-Received: (at 18420) by debbugs.gnu.org; 9 Sep 2014 04:19:03 +0000 Original-Received: from localhost ([127.0.0.1]:34305 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XRCtR-0003dv-K0 for submit@debbugs.gnu.org; Tue, 09 Sep 2014 00:19:02 -0400 Original-Received: from dnvrco-outbound-snat.email.rr.com ([107.14.73.227]:21415 helo=dnvrco-oedge-vip.email.rr.com) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XRCtO-0003da-92 for 18420@debbugs.gnu.org; Tue, 09 Sep 2014 00:19:00 -0400 Original-Received: from [70.94.38.149] ([70.94.38.149:50153] helo=TAKVER) by dnvrco-oedge02 (envelope-from ) (ecelerity 3.5.0.35861 r(Momo-dev:tip)) with ESMTP id A4/75-08080-BAF7E045; Tue, 09 Sep 2014 04:18:51 +0000 In-Reply-To: <83fvg3cueu.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 07 Sep 2014 18:38:49 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (windows-nt) X-RR-Connecting-IP: 107.14.64.130:25 X-Authority-Analysis: v=2.1 cv=d5x7yHTE c=1 sm=1 tr=0 a=AppmJ/7ZOOFWL/q6u6u93g==:117 a=AppmJ/7ZOOFWL/q6u6u93g==:17 a=ayC55rCoAAAA:8 a=9XSUBuVRJI8A:10 a=Bi9pSSD5RJcA:10 a=08t0Cx2oaSgA:10 a=o_R75loqY_IA:10 a=9i_RQKNPAAAA:8 a=mDV3o1hIAAAA:8 a=SLWBIbPmFhpQvn784sMA:9 a=CzHYZWsTFaBZafQz:21 a=cCKAo4CXF1RXbfet:21 a=ii61gXl28gQA:10 X-Cloudmark-Score: 0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:93172 Archived-At: Eli Zaretskii writes: > Did you try replacing accept-process-output with sit-for? The current code has: (accept-process-output process 0.1) This waits up to 0.1 seconds for output from the child process ada_mode_wisi_parse (or any other process, but there are none in this setup). In practice in this scenario, it returns much sooner than 0.1 seconds; there is some output very quickly after sending any part of the buffer contents. Replaced with: (sit-for 0.1) This performs redisplay, then waits for SECONDS seconds or until "input" is available. The wait is done with 'read-event', which reads from "the input stream". So apparently this means user input, not child process input. This fixed the hang. There is no user input during this process; it's too fast. The test buffer is 7274 bytes long, so there are three cycles in the send-parse loop. Apparently what happens is each cycle waits for 0.1 seconds, giving the child process enough time to process all of that input. Then Emacs wakes up, processes all of the child process output, resumes send-parse, and sends the next chunk. This works, but is essentially a race condition between the sit-for timer and the child process. It will be too slow for general use; I have clients with 250,000 byte buffers, which would take 12.2 seconds (slower than the current elisp parser). Obviously I can reduce the 0.1, but that runs into the race condition and the deadlock. 0.01 would be fast enough for the largest buffer, so I tried: (sit-for 0.01) This hangs, in the same deadlock described above. So we need a fix for the deadlock. I can use (sit-for 0.1) as a workaround while I continue debugging the rest of the problems I have, on reasonably sized buffers. There is another workaround I can use; queue all output in ada_mode_wisi_parse until all input is read and parsing is done, then send it all. That would lose parallel processing time, which is significant with today's processors. But I'll use it if we can't fix this soon. > How about enlarging the 2nd argument to accept-process-output -- did > you try that, and if so, did it have any effect? tried: (accept-process-output 1.0) This hangs. Since (sit-for 0.1) allows enough time for the child process to process all of one chunk, this is expected. >> The loop is replacing an earlier version that used a single >> process-send-string to send the entire buffer string. I was hoping that >> the explicit accept-process-out calls would allow C-g to work, but >> apparently not. > > I cannot easily interpret this, too much info is missing. For > starters, if the parser gets only part of its expected input (whose > size you are apparently passing via command-line arguments), would it > start parsing, or will it wait for the rest? It starts parsing immediately, and generates some output quickly. The byte count indicates when to stop parsing. > If the latter, then the > way you've broken the buffer string into chunks will not change > anything, right? It doesn't change the child process behavior. I was hoping it would change the emacs side, allowing it to at least check for C-g every 0.1 seconds. Hmm, I'm not clear that accept-process-input checks for input events. Perhaps we need a combined sit-for/accept-process-input? >> The external process implements an LALR parser for Ada source; it >> outputs the parse results back to Emacs. There is a lot of output, so >> it can easily fill up the IO queue. Attaching to that process when >> Emacs is hung shows it is blocked writing to stdout, in a normal part >> of the code flow. > > Can you tell how much did it write to the pipe at that point? No, that info is not tracked in the current code. I can add it. The parser has read 2935 bytes. However, there is some variation in precisely when the hang occurs; the wisi-ada-parse-send-parse loop outputs the sent bytes each cycle, and that number changes with each run (among the 3 possible choices for this size buffer). > What is your versions of GCC, Binutils, MinGW runtime, and w32 API > headers, btw? gcc for Emacs: gcc.exe (Rev3, Built by MSYS2 project) 4.9.1 binutils: 2.24.78559.d43808f-1 (msys2 package) MinGW: not sure what this means; the runtime is just a bunch of packages. For one: libgc 7.2.d-1 w32 API headers: /msys64/mingw32/i686-w64-mingw32/include/windows.h has no version number mingw-w64-i686-headers-git 4.0.0.4210.0177153-1 Hope that helps. -- -- Stephe