From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.devel Subject: Re: master 64f2c96 2/2: Make a process test faster. Date: Sun, 10 Jan 2021 18:17:58 +0100 Message-ID: References: <20210102125533.3832.43853@vcs0.savannah.gnu.org> <20210102125536.C3FED2094D@vcs0.savannah.gnu.org> <83bldwhhkt.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="9256"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Philipp Stephani , Glenn Morris , Emacs developers To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jan 10 18:19:38 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kyeNO-0002K0-0Z for ged-emacs-devel@m.gmane-mx.org; Sun, 10 Jan 2021 18:19:38 +0100 Original-Received: from localhost ([::1]:40132 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kyeNN-0007Nt-2s for ged-emacs-devel@m.gmane-mx.org; Sun, 10 Jan 2021 12:19:37 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:38090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyeM8-00064i-GP for emacs-devel@gnu.org; Sun, 10 Jan 2021 12:18:20 -0500 Original-Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]:46464) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kyeLz-0003tD-Te; Sun, 10 Jan 2021 12:18:20 -0500 Original-Received: by mail-oi1-x236.google.com with SMTP id q205so17641578oig.13; Sun, 10 Jan 2021 09:18:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bKXofcWx//+paKQLvFvuGgCWRcMZ581F5SRnUio/yQY=; b=jZa3MrjKtqZs/i69wQj6BFmX/oIeRuLsZg0mqu1gtts2R1Eg7qfOX6mWdrzgiueMus ZcQ1Pg+2yLLaxBm8Q7rYsH/Nw0j2sxttJzuHjmoo5/xBYE9GLeToq7encH7mONKJV4zk JvS7503BKv9VmyoF7qL0nsYNH/ilqLK/oQ5PmLha8ZkeNH2LfRs/qZWBBSYr0GWyzXhk dcfISbYxq+dCGp4RTb1GSz2Uv5eFQSMuy1de8YdNLsWrlsmmCMcBh5CjVwIGZfiUU/GQ R3/ACNlrE90xGRsHAllaG64JPc8V1RVRZebpEHBbbRsvVTv0aAEQ5Wk9oSXPVUk/sLmG xTvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bKXofcWx//+paKQLvFvuGgCWRcMZ581F5SRnUio/yQY=; b=jkPl7EBt34nusw/ymnnitRfCBu+tohLNtFDGGJ66S0R0beL0EdC5UiR/rW1UWxyFe3 Zy5lhQ8S5pzqxW2GK0Belkjors0eQXp2Oat+41aGMibB+V1YmfidoBacDeZ5qlso7s5g Ht9ek3HOUrrPFQGD7SsY+vtZEZK6S4X9JbjC0kijYcLDHLezF2DzzTKpu4+ac/fQWWYp S9mG/59JGRB4FeyVtR1osrcBXEvFwRRYksMC9hYuiE6fYY9axfl1q5paxaNeh8ThPg4G ympDVzv5Ja4p3jN+5mP8NZklOSvDvINzXX+p84EG1QbgnCWtzG8iIuMYdWHnXOuWPyht A9cw== X-Gm-Message-State: AOAM533jgwz01OwggU5ThhCIPPwt4TF+YiI6q2jkw9qiqnmGGr03LmDB S5xfs07NTE4v1FTAk+so57TtbZJmSLv/rMKArjnxqT24ebE= X-Google-Smtp-Source: ABdhPJwBF4zazrgU5JUZh2Fc4Ic+X0uP1GnCUXc4SHCYvfFFrYegEPCN3PnlJsxlzPjkn/kfANugj/a7v33GbJlEmNw= X-Received: by 2002:aca:3342:: with SMTP id z63mr7845203oiz.65.1610299088924; Sun, 10 Jan 2021 09:18:08 -0800 (PST) In-Reply-To: <83bldwhhkt.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=p.stephani2@gmail.com; helo=mail-oi1-x236.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:262862 Archived-At: Am So., 10. Jan. 2021 um 17:39 Uhr schrieb Eli Zaretskii : > > > From: Philipp Stephani > > Date: Sat, 9 Jan 2021 22:00:45 +0100 > > Cc: Philipp Stephani , Emacs developers > > 2. The other one happens if SIGCHLD is signaled during > > wait_reading_process_output. Then wait_reading_process_output will > > wait forever, since the stdout FD never gets closed and it doesn't see > > the process status update in time. > > Do you mean that wait_reading_process_output has this problem in > general, or just in this particular scenario? If the former, I'm > surprised, as we are using this code for a very long time. If the > latter, can you elaborate on the situation, and what does SIGCHLD have > to do with closing stdout? I believe is a rather general problem. Anecdotally I've heard occasionally about problems with accept-process-output deadlocking, which might be related. See e.g. some of the other tests in process-tests.el. AIUI, the sequence of events is as follows: 1. (accept-process-output PROC) 2. Here the process isn't finished yet. accept-process-output waits for something to become available on stdout. 3. If the process doesn't write anything to stdout, accept-process-output will block. 4. The process exits without having written anything. 5. Stdout is closed. 6. pselect returns, but since the process hasn't written anything, wait_reading_process_output doesn't return. 7. Emacs receives SIGCHLD. 8. Emacs tries to notify accept-process-output, but it's too late - we are already within the pselect call, which now hangs forever. To test that, I added some printf statements, and indeed I saw that wait_reading_process_output was entered, then SIGCHLD was received, but wait_reading_process_output continued to hang. I remedied this on the scratch/sigchild-fd branch by having the SIGCHLD handler signal a pipe that pselect watches. That fixes the deadlock entirely for me on GNU/Linux and macOS. (Going forward we might want to use pidfds if available, they seem simpler and less error-prone than signals.)