From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: "Basil L. Contovounesios" Newsgroups: gmane.emacs.bugs Subject: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output Date: Wed, 17 Oct 2018 21:46:39 +0100 Message-ID: <87zhvc2te8.fsf@tcd.ie> References: <87r2gwbmma.fsf@tcd.ie> <875zy7cqzd.fsf@gmx.de> <878t30h823.fsf@tcd.ie> <87sh177i36.fsf@gmx.de> <877eiipu8g.fsf@tcd.ie> <83k1mirl9n.fsf@gnu.org> <874ldkjwzn.fsf@tcd.ie> <83zhvcqwxc.fsf@gnu.org> <87o9bswirc.fsf@tcd.ie> <83y3awqvtl.fsf@gnu.org> <83woqgqvl2.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1539809113 19317 195.159.176.226 (17 Oct 2018 20:45:13 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 17 Oct 2018 20:45:13 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: michael.albinus@gmx.de, 33018@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Oct 17 22:45:09 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gCsgn-0004wF-4J for geb-bug-gnu-emacs@m.gmane.org; Wed, 17 Oct 2018 22:45:09 +0200 Original-Received: from localhost ([::1]:39020 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gCsit-0002c6-Cl for geb-bug-gnu-emacs@m.gmane.org; Wed, 17 Oct 2018 16:47:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34360) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gCsii-0002aU-F3 for bug-gnu-emacs@gnu.org; Wed, 17 Oct 2018 16:47:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gCsid-000372-Iz for bug-gnu-emacs@gnu.org; Wed, 17 Oct 2018 16:47:08 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:51576) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gCsid-00036w-Ed for bug-gnu-emacs@gnu.org; Wed, 17 Oct 2018 16:47:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gCsic-0007Ck-8J for bug-gnu-emacs@gnu.org; Wed, 17 Oct 2018 16:47:03 -0400 X-Loop: help-debbugs@gnu.org Resent-From: "Basil L. Contovounesios" Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 17 Oct 2018 20:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33018 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 33018-submit@debbugs.gnu.org id=B33018.153980920927673 (code B ref 33018); Wed, 17 Oct 2018 20:47:02 +0000 Original-Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 20:46:49 +0000 Original-Received: from localhost ([127.0.0.1]:55834 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gCsiP-0007CG-9s for submit@debbugs.gnu.org; Wed, 17 Oct 2018 16:46:49 -0400 Original-Received: from mail-ed1-f41.google.com ([209.85.208.41]:39400) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gCsiN-0007C3-N5 for 33018@debbugs.gnu.org; Wed, 17 Oct 2018 16:46:48 -0400 Original-Received: by mail-ed1-f41.google.com with SMTP id d15-v6so26190752edq.6 for <33018@debbugs.gnu.org>; Wed, 17 Oct 2018 13:46:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=0hILc1GM7YBstkA/IhmeqByE0F3D+HnFUecP9zJoSho=; b=xkRdFQc2bBrf83ODmqK9tIcNWPgtcDUFYytYzFZzzS0kd+nsC52wR8lTFzj35CNOkR lotcgWTlJ94779OEW7zEmjRw7k3p3tje28Hka1cNrPCACmIkGtuAUXIAzZVLGkChJZJI IbVGfrJ2j12njjA6KTRpsrL2bfYpW5J2MkjnGI4TW6RHdbo3UubEOH31VogewJd3DD3i aOWJwKPvEqMnB+YUOnvPf2FPxzQkNZLJq3kHRycNYAoANRLOU6IjjTCjFCTvHw1biHIy 3k7sPXFabhKs1ShAF9zRFd7NwgZ98J0qWW8uJM6SJGQ2g7KD0YMU6MU4+bN7AfT40wS9 Jlrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=0hILc1GM7YBstkA/IhmeqByE0F3D+HnFUecP9zJoSho=; b=KUmGP4c1SLWXCqyypEqNjT6ik7PYQQGqXRKJIqx8YmlBQvTVCf0+eybLBK+cvfdrB5 Qg1h/mK2zgCuBXdc6/iU8mFzZ03FFWzImWdLLKwB4/M/5x/US2ZnXxi2s1R7jvoZCqTZ GI+bschHU6W6SW0Gd+qRTGWZ1oe4YDYmA8FVJZ5JwsDwjFyIqujwc7Bu4P98G/KN5Mdv ox6NLv9fDTEkLXW6VrNfLqZPAohFUOFHLKRgdPMSZmnylVTZ7fjz1Mqw5lMtEp7HEFtB lBK3mG2JhGdMmxP7JvQAGPGPM/UG2NmcOiUBwq6qk4ZQDqfX5GzFRgXCOhoNJkNsZ398 zxcg== X-Gm-Message-State: ABuFfohG1JAkmLgYr0B5YA5GPgf1kUwHJDreeUIIB+/CIlh8TpvPsoRp ziPaPXJjlHmFS14L20qonPm85A== X-Google-Smtp-Source: ACcGV63vcd9L06Wv3CajHnLQ5BA7xBUBiO0t2T677CvFr8ZO3PJdDrk+jPq7kNXi7Q6QT4BpDVP1YQ== X-Received: by 2002:a17:906:88f:: with SMTP id n15-v6mr27897968eje.119.1539809201734; Wed, 17 Oct 2018 13:46:41 -0700 (PDT) Original-Received: from localhost (51-171-243-2-dynamic.agg2.clk.blp-srl.eircom.net. [51.171.243.2]) by smtp.gmail.com with ESMTPSA id z2-v6sm8007539edz.31.2018.10.17.13.46.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 17 Oct 2018 13:46:41 -0700 (PDT) In-Reply-To: <83woqgqvl2.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 17 Oct 2018 21:25:29 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:151356 Archived-At: Eli Zaretskii writes: >> Date: Wed, 17 Oct 2018 21:20:22 +0300 >> From: Eli Zaretskii >> Cc: michael.albinus@gmx.de, 33018@debbugs.gnu.org >> >> > From: "Basil L. Contovounesios" >> > Cc: , <33018@debbugs.gnu.org> >> > Date: Wed, 17 Oct 2018 19:05:59 +0100 >> > >> > > Looks like the buffer of the pipe through which Emacs reads the stuff >> > > is full, and wget waits for some space there? >> > >> > Would that imply that different threads/processes are (re)using the same >> > buffer/pipe? >> >> Could be, but it's more likely that Emacs simply doesn't read the >> output from wget. > > I think the relevant code should be instrumented to show which thread > waits for what process(es). Each thread launches a single wget process which it then waits for before dying, and current-thread is always eq to that thread around calls to accept-process-output. Or are you talking about some other type of thread? Either way, I'll report back when I've had a deeper look into what Emacs is doing, unless someone beats me to it. > Btw, are you sure this is not a bug in your program? No, but the fact that it reliably works when Emacs is run with -batch, and reliably hangs when run with -Q is at least somewhat intriguing. > Michael caused your program to work twice by simple changes, AFAIU, > no? Michael avoided the hang by rewriting the create-join-...-create-join sequence as create-create-...-join-join. But if the latter is done twice from within the same master thread, the hang still occurs. As I said: > There's something about going through a complete create-join cycle more > than once within a non-main-thread which is triggering this behaviour. (Actually, I haven't checked whether the hang occurs when two create-join cycles are completed within main-thread; I was just specifically describing my sample program.) In his second rewrite, Michael replaced wget with echo, which does not suffer from any hangs. As I said: > Indeed, there are many subprocess-within-a-thread examples which don't > suffer from a hang, e.g. by using a different URL. I would like to get > to the bottom of why network programs like wget/curl in particular > eventually hang, though. In other words, I don't (yet) see why my recipe shouldn't work, and I'm curious to eventually get to the bottom of this. I'm sorry I've been talking more than doing, but university and house hunting will dominate my free time for the next few weeks. Thanks, -- Basil