From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Wed, 22 Aug 2018 19:22:27 -0400 Message-ID: <878t4xdfag.fsf@netris.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1534989494 6346 195.159.176.226 (23 Aug 2018 01:58:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 23 Aug 2018 01:58:14 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) To: 31878@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Aug 23 03:58:10 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsesy-0001UI-5n for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 03:58:08 +0200 Original-Received: from localhost ([::1]:33996 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsev3-0007f0-0U for guile-bugs@m.gmane.org; Wed, 22 Aug 2018 22:00:17 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58819) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fseun-0007Sk-TY for bug-guile@gnu.org; Wed, 22 Aug 2018 22:00:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsel9-00018U-3P for bug-guile@gnu.org; Wed, 22 Aug 2018 21:50:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53174) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fsel8-00017q-3j for bug-guile@gnu.org; Wed, 22 Aug 2018 21:50:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fsel7-0008L5-Rv; Wed, 22 Aug 2018 21:50:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Aug 2018 01:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.153498898832030 (code B ref 31878); Thu, 23 Aug 2018 01:50:01 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 23 Aug 2018 01:49:48 +0000 Original-Received: from localhost ([127.0.0.1]:58192 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fseku-0008KY-5H for submit@debbugs.gnu.org; Wed, 22 Aug 2018 21:49:48 -0400 Original-Received: from world.peace.net ([64.112.178.59]:39420) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsekr-0008KG-SD for 31878@debbugs.gnu.org; Wed, 22 Aug 2018 21:49:46 -0400 Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1fsekm-00029u-El for 31878@debbugs.gnu.org; Wed, 22 Aug 2018 21:49:40 -0400 In-Reply-To: <87h8m0uw3z.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 18 Jun 2018 14:17:52 +0200") Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1fscTp-0001dl-TG; Wed, 22 Aug 2018 19:24:02 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9123 Archived-At: Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > >> ludo@gnu.org (Ludovic Court=C3=A8s) skribis: >> >>> I believe this comes from the fact that =E2=80=98autoloads-done=E2=80= =99 and related >>> alists in (ice-9 boot-9) are manipulated in a non-thread-safe fashion. >> >> Here=E2=80=99s a proposed fix for =E2=80=98stable-2.2=E2=80=99 as discus= sed on #guile, Andy: > > After further discussion on IRC, I pushed a variant of this patch as > commit 761cf0fb8c364e885e4c6fced34563f8157c3b84. There are problems with this fix, e.g. . More generally, nearly arbitrary code can be run in the top-level expressions of a module. It could launch other threads which try to load modules, or even send messages to other existing threads asking them to do work. In some cases, the body of the module might never terminate. The entire main program might be run from there. I suspect that's not unusual. I can see another problem as well: while the module is in the process of loading, the partially-loaded module is globally visible and accessible to other threads. If I'm not mistaken, with this patch, there's nothing preventing other threads from attempting to use the partially-loaded module. I thought about how to fix this thread-safety problem a long time ago, and came up with a rough outline of a solution. The idea is that the module should not be added to the global module table until the module has finished loading. While the module is being loaded, it would be made visible only to the loading thread, and to any other threads spawned during the loading process, by adding the module to a local list of modules-being-loaded referenced by a fluid variable. If any other threads attempt to access the module, it would not be found in the global module table, and thus trigger an auto-load, which would wait for the lock to be released before proceeding. What do you think? Mark