From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Wed, 22 Aug 2018 22:18:59 -0400 Message-ID: <87woshbzak.fsf@netris.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> <878t4xdfag.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1534991688 8967 195.159.176.226 (23 Aug 2018 02:34:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 23 Aug 2018 02:34:48 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= To: 31878@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Aug 23 04:34:43 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsfSK-0002An-Ei for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 04:34:40 +0200 Original-Received: from localhost ([::1]:34089 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsfUQ-0007yW-LV for guile-bugs@m.gmane.org; Wed, 22 Aug 2018 22:36:50 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36426) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsfU9-0007qT-GJ for bug-guile@gnu.org; Wed, 22 Aug 2018 22:36:34 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsfF8-00046o-Ld for bug-guile@gnu.org; Wed, 22 Aug 2018 22:21:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53186) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fsfF8-00045w-HP for bug-guile@gnu.org; Wed, 22 Aug 2018 22:21:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fsfF8-0000kT-6x; Wed, 22 Aug 2018 22:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Aug 2018 02:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.15349908402819 (code B ref 31878); Thu, 23 Aug 2018 02:21:02 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 23 Aug 2018 02:20:40 +0000 Original-Received: from localhost ([127.0.0.1]:58204 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsfEm-0000jO-I4 for submit@debbugs.gnu.org; Wed, 22 Aug 2018 22:20:40 -0400 Original-Received: from world.peace.net ([64.112.178.59]:39464) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsfEk-0000jA-T8 for 31878@debbugs.gnu.org; Wed, 22 Aug 2018 22:20:39 -0400 Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1fsfEf-0002Fi-9W; Wed, 22 Aug 2018 22:20:33 -0400 In-Reply-To: <878t4xdfag.fsf@netris.org> (Mark H. Weaver's message of "Wed, 22 Aug 2018 19:22:27 -0400") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9125 Archived-At: I wrote: > I thought about how to fix this thread-safety problem a long time ago, > and came up with a rough outline of a solution. The idea is that the > module should not be added to the global module table until the module > has finished loading. While the module is being loaded, it would be > made visible only to the loading thread, and to any other threads > spawned during the loading process, by adding the module to a local list > of modules-being-loaded referenced by a fluid variable. If any other > threads attempt to access the module, it would not be found in the > global module table, and thus trigger an auto-load, which would wait for > the lock to be released before proceeding. I forgot to mention an important aspect of the proposed auto-load locking here. It would not be a global lock, but rather a lock specific to the module being loaded. So, I guess we would need a global table of locks for modules-being-loaded. Since Guile (unfortunately) allows cyclic module dependencies, we would need a mechanism to avoid deadlocks in case modules A and B both import each other, and two threads concurrently attempt to load those modules. The first idea that comes to mind is to also have a global structure storing a partial order on the modules currently being loaded. If, while module A is being loaded, there's an attempt to auto-load module B, then an entry (A < B) would added to the partial order. The partial order would not allow cycles to be introduced, reporting an error in that case. In case a cycle would be introduced when adding (A < B), then the thread would simply be given access to the partially-loaded module B, by adding B to its local list of modules-being-loaded. Comments and suggestions welcome, Mark