From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Newsgroups: gmane.lisp.guile.bugs Subject: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems Date: Sat, 26 Nov 2022 15:39:06 +0100 Message-ID: <871qppaigl.fsf@gnu.org> References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8643"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) Cc: hylophile@posteo.de, 59321@debbugs.gnu.org To: Andrew Whatson Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Sat Nov 26 15:40:31 2022 Return-path: Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oywM2-00025z-OQ for guile-bugs@m.gmane-mx.org; Sat, 26 Nov 2022 15:40:30 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oywLc-0006CE-72; Sat, 26 Nov 2022 09:40:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oywLb-0006Bn-1Z for bug-guile@gnu.org; Sat, 26 Nov 2022 09:40:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oywLa-0005d5-O0 for bug-guile@gnu.org; Sat, 26 Nov 2022 09:40:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oywLa-0005VN-EW for bug-guile@gnu.org; Sat, 26 Nov 2022 09:40:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sat, 26 Nov 2022 14:40:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 59321 X-GNU-PR-Package: guile Original-Received: via spool by 59321-submit@debbugs.gnu.org id=B59321.166947356121077 (code B ref 59321); Sat, 26 Nov 2022 14:40:02 +0000 Original-Received: (at 59321) by debbugs.gnu.org; 26 Nov 2022 14:39:21 +0000 Original-Received: from localhost ([127.0.0.1]:38334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oywKu-0005Tt-Mx for submit@debbugs.gnu.org; Sat, 26 Nov 2022 09:39:21 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:54622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oywKp-0005Tc-R3 for 59321@debbugs.gnu.org; Sat, 26 Nov 2022 09:39:19 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oywKj-0005Lb-W5; Sat, 26 Nov 2022 09:39:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=2peMiC7JxeskFb9ZIWapZ0SaZu2cvaYfa79Jq78FVRU=; b=mmjfWoo1hecu3t5HEdmY FzBifjdN5k93DwV7zVPxlMcFHtJ+gioNZiarSz9JedHdkl/8DtjKm03BmYIHEeIEVMfVN4pV26K8L xuDTaBJi4eAUf10DcA/BeeHuPURhEBZxCdh6goW/O4/iR5EGzwMPp1ISrkgQQq0ucqoef46Fm7uyv mnF8+Knx0NYFPhMeiGZMK5Jwr8EACxQfX3D52jutAvAych6OStrsT8xYlTW97xcdIjoiGtGC2+N64 qTZhU/CmS0O7EzYQkf9ZmvQbpoICZGyR2QKDy31cTsRjc8xVfyasJSTpzRMbFt7ai8JO8iSlFogwo 1cNnNvSyeunm6w==; Original-Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oywKj-0002c5-Go; Sat, 26 Nov 2022 09:39:09 -0500 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 6 Frimaire an 231 de la =?UTF-8?Q?R=C3=A9volution, ?= jour de la =?UTF-8?Q?M=C3=A2che?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu In-Reply-To: (Andrew Whatson's message of "Mon, 21 Nov 2022 14:22:52 +1000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.bugs:10449 Archived-At: Hi Andrew, Andrew Whatson skribis: > Ludovic Court=C3=A8s wrote: [...] >> Libguile opens all its own file descriptors at O_CLOEXEC (one omission >> was recently fixed in 0aa1a9976fc3c6af4d1087e59d728cb8fe7d369a) so it >> may be possible to remove that FD-closing loop. There=E2=80=99s still t= he >> possibility that application bug unwillingly leaks FDs, but we could >> consider it=E2=80=99s none of our business. >> >> Thoughts? > > I agree with this approach in principle, but from what Tomas is saying > it seems like it's not currently possible for applications to do the > right thing in all cases. OK. [...] > We also need equivalent functionality around SOCK_CLOEXEC. It seems > this is implemented for =E2=80=98accept=E2=80=99, but not =E2=80=98socket= =E2=80=99 or =E2=80=98socketpair=E2=80=99. It=E2=80=99s possible to use SOCK_CLOEXEC with =E2=80=98socket=E2=80=99 and= =E2=80=98socketpair=E2=80=99 already, as in: (socket AF_INET (logior SOCK_CLOEXEC SOCK_STREAM) 0) With commit 1d313bf5f0d296d766bd3a0e6d030df37c71711b, =E2=80=98pipe=E2=80= =99 is also covered. So I think we have pretty much everything we need, at least starting with 3.0.9. > Python's PEP 433 contains a good explanation of the issues which can > arise from leaked file descriptors: > https://peps.python.org/pep-0433/#inherited-file-descriptors-issues > > Given the risks, I'm convinced that Guile's conservative approach is > actually quite sensible. It seems like the best path forward would be > to implement platform-specific optimizations where possible. > > I've attached a draft patch which implements a fast-path on systems > where "/proc/self/fd" is available. The patch LGTM; it=E2=80=99s certainly an improvement on systems configured= with a high per-process FD limit. Now, I believe use of =E2=80=98posix_spawn=E2=80=99 as proposed in makes that unnecessary. Let=E2=80=99s = take a closer look at that other patch and so we can see=E2=80=A6 Thanks, Ludo=E2=80=99.