From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Matthias Dahl Newsgroups: gmane.emacs.devel Subject: Re: wait_reading_process_ouput hangs in certain cases (w/ patches) Date: Tue, 14 Nov 2017 15:58:32 +0100 Message-ID: <56e722a6-95a4-0e42-185c-f26845d4f4bf@binary-island.eu> References: <83lgjz8eiy.fsf@gnu.org> <831slp98ut.fsf@gnu.org> <83tvyj62qg.fsf@gnu.org> <83r2tetf90.fsf@gnu.org> <5150d198-8dd3-9cf4-5914-b7e945294452@binary-island.eu> <83tvy7s6wi.fsf@gnu.org> <83inemrqid.fsf@gnu.org> <398f8d17-b727-d5d6-4a31-772448c7ca0d@binary-island.eu> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1510671627 2107 195.159.176.226 (14 Nov 2017 15:00:27 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 14 Nov 2017 15:00:27 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 Cc: emacs-devel@gnu.org To: Paul Eggert , Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Nov 14 16:00:13 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eEch5-0008Ap-V6 for ged-emacs-devel@m.gmane.org; Tue, 14 Nov 2017 16:00:08 +0100 Original-Received: from localhost ([::1]:60166 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEchD-00048d-DI for ged-emacs-devel@m.gmane.org; Tue, 14 Nov 2017 10:00:15 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47283) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEcfg-00032T-Jy for emacs-devel@gnu.org; Tue, 14 Nov 2017 09:58:41 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEcfb-0003Pi-Kq for emacs-devel@gnu.org; Tue, 14 Nov 2017 09:58:40 -0500 Original-Received: from ud19.udmedia.de ([194.117.254.59]:47402 helo=mail.ud19.udmedia.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eEcfb-0003LF-7z for emacs-devel@gnu.org; Tue, 14 Nov 2017 09:58:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=binary-island.eu; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=k1; bh=Kx mjNkxkO8z/p7B4pIxnhwtts7+byAIlMi2E9r0Ls+o=; b=w8J1GUlBcdFjr4ly5H OuUmMS+XFRRsQ3DClooHXj/fcmUSfsQsv5aSvX2CCuB6sy2TNXsJ4v1JK48W0TLe XWgO2YEJe2c2GGtH577jQPtgYhAZ01WKFLV13vI1knIua+YfUbh7t/6RoDoLEXrE 4r4Fzt4DQd5PGenWUB32vYqLE= Original-Received: (qmail 30550 invoked from network); 14 Nov 2017 15:58:32 +0100 Original-Received: from unknown (HELO ?IPv6:2a02:810b:c540:234:36aa:25b9:ca8f:d05f?) (ud19?126p1@2a02:810b:c540:234:36aa:25b9:ca8f:d05f) by mail.ud19.udmedia.de with ESMTPSA (ECDHE-RSA-AES128-GCM-SHA256 encrypted, authenticated); 14 Nov 2017 15:58:32 +0100 Openpgp: id=1E87ADA02EFE759EFC20B2D1042F47D273AA780C In-Reply-To: Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 194.117.254.59 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:220178 Archived-At: Hello Paul... On 13/11/17 20:44, Paul Eggert wrote: > It cannot be for the entire process lifetime, since it's an unsigned > integer that is designed to wrap around on overflow. So it sounds like > you want a counter modulo 2**N for the entire process lifetime, and > we're discussing what value of N is large enough for the purposes of > wait_reading_process_output. Since EMACS_UINT might be only 32 bits, and > it's eminently reasonable for a process to read more than 2**32 bytes > during its lifetime, this is not a theoretical issue. > > So: why must the counter contain more than a small number of bits (16, > say)? I'm not seeing the scenario. I'm asking not because I think it's > important to save a few bits in the counter: I'm asking because I want > to understand the change and the reasoning behind it, so that I can be > sure it's fixing the bug. I am well aware that EMACS_UINT can be 32 bits wide and that the counter could and will eventually wrap around, given the right process usage. The scenario my patch(es) fix is the following: We have a wait_... call with a waitproc set. Timers and filters run which can by themselves call us recursively again, triggering read back from our own waitproc whilst we would never notice it and stall waiting for it if no new data becomes available. Thus, with the current simple implementation, there is only one corner case that will miss such read back nevertheless: If during an active call to wait_... all recursive calls happen to read exactly 2**32 (or whatever bit depths EMACS_UINT is) bytes back, then we will miss it completely and stall. If we read more than that, our calculation is off and we report less bytes read back through got_some_output which is pretty much irrelevant. Since I did not want to make any random assumptions about an upper limit I decided to be on the safe side with >= 32 bits which, yes, is probably overkill but arbitrarily setting a lower limit which might even wrap around more often without any real gain, is also not the best thing to do in my opinion. Besides that, I also opted for the "bytes read during process lifetime" metric because I thought it could turn out useful in the future while debugging. A more synthetic metric like chunks read or in-flight bytes read, not so much, imho. What do you say? So long, Matthias -- Dipl.-Inf. (FH) Matthias Dahl | Software Engineer | binary-island.eu