From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Thu, 23 Aug 2018 15:51:44 +0200 Message-ID: <8736v5qjgv.fsf@gnu.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> <876002dm18.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1535034335 3103 195.159.176.226 (23 Aug 2018 14:25:35 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 23 Aug 2018 14:25:35 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: 31878@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Aug 23 16:25:30 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsqYE-0000g9-MO for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 16:25:30 +0200 Original-Received: from localhost ([::1]:36832 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsqaL-0005Y0-3a for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 10:27:41 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51218) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsq1x-000472-H9 for bug-guile@gnu.org; Thu, 23 Aug 2018 09:52:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsq1v-0005z1-FB for bug-guile@gnu.org; Thu, 23 Aug 2018 09:52:09 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53353) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fsq1q-0005we-DF for bug-guile@gnu.org; Thu, 23 Aug 2018 09:52:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fsq1q-0004k5-8z; Thu, 23 Aug 2018 09:52:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Aug 2018 13:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.153503232018221 (code B ref 31878); Thu, 23 Aug 2018 13:52:02 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 23 Aug 2018 13:52:00 +0000 Original-Received: from localhost ([127.0.0.1]:58371 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsq1l-0004jm-QX for submit@debbugs.gnu.org; Thu, 23 Aug 2018 09:52:00 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:58374) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsq1j-0004jZ-H1 for 31878@debbugs.gnu.org; Thu, 23 Aug 2018 09:51:56 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsq1a-0005hr-P1 for 31878@debbugs.gnu.org; Thu, 23 Aug 2018 09:51:50 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33994) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsq1a-0005gm-Dh; Thu, 23 Aug 2018 09:51:46 -0400 Original-Received: from [193.50.110.186] (port=39682 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fsq1a-0000gR-3I; Thu, 23 Aug 2018 09:51:46 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 6 Fructidor an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu In-Reply-To: <876002dm18.fsf@netris.org> (Mark H. Weaver's message of "Wed, 22 Aug 2018 19:22:27 -0400") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9128 Archived-At: Hi Mark, Mark H Weaver skribis: > reopen 31878 > thanks > > Hi Ludovic, > > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> ludo@gnu.org (Ludovic Court=C3=A8s) skribis: >> >>> ludo@gnu.org (Ludovic Court=C3=A8s) skribis: >>> >>>> I believe this comes from the fact that =E2=80=98autoloads-done=E2=80= =99 and related >>>> alists in (ice-9 boot-9) are manipulated in a non-thread-safe fashion. >>> >>> Here=E2=80=99s a proposed fix for =E2=80=98stable-2.2=E2=80=99 as discu= ssed on #guile, Andy: >> >> After further discussion on IRC, I pushed a variant of this patch as >> commit 761cf0fb8c364e885e4c6fced34563f8157c3b84. > > There are problems with this fix, e.g. . > > More generally, nearly arbitrary code can be run in the top-level > expressions of a module. It could launch other threads which try to > load modules, or even send messages to other existing threads asking > them to do work. In some cases, the body of the module might never > terminate. The entire main program might be run from there. I suspect > that's not unusual. Indeed, good catch. :-/ > I can see another problem as well: while the module is in the process of > loading, the partially-loaded module is globally visible and accessible > to other threads. If I'm not mistaken, with this patch, there's nothing > preventing other threads from attempting to use the partially-loaded > module. The module is not reachable until =E2=80=98set-module-name!=E2=80=99 has be= en called on it, but =E2=80=98process-define-module=E2=80=99 does that right away IIRC, = i.e., before the whole body has been evaluated. So I guess you=E2=80=99re right: other threads could stumble upon partially-loaded modules. If the =E2=80=98define-module=E2=80=99 scoped encompassed the whole body li= ke the R6 =E2=80=98library=E2=80=99 form, it would be easy to determine when the whol= e module top-level has been loaded. Right now, I suppose we have to determine the end-of-module-top-level =E2=80=9Cfrom the outside=E2=80=9D, i.e., from =E2=80=98resolve-module=E2=80=99 or similar, no? > I thought about how to fix this thread-safety problem a long time ago, > and came up with a rough outline of a solution. The idea is that the > module should not be added to the global module table until the module > has finished loading. While the module is being loaded, it would be > made visible only to the loading thread, and to any other threads > spawned during the loading process, by adding the module to a local list > of modules-being-loaded referenced by a fluid variable. If any other > threads attempt to access the module, it would not be found in the > global module table, and thus trigger an auto-load, which would wait for > the lock to be released before proceeding. > > What do you think? It sounds like a good idea. Thanks, Ludo=E2=80=99.