From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Fri, 24 Aug 2018 10:45:57 +0200 Message-ID: <87r2iorw3e.fsf@gnu.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> <878t4xdfag.fsf@netris.org> <87woshbzak.fsf@netris.org> <87va81p4sn.fsf@gnu.org> <87in40g9bx.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1535100528 26946 195.159.176.226 (24 Aug 2018 08:48:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 24 Aug 2018 08:48:48 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: 31878@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Fri Aug 24 10:48:44 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ft7lr-0006sw-IF for guile-bugs@m.gmane.org; Fri, 24 Aug 2018 10:48:43 +0200 Original-Received: from localhost ([::1]:40484 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ft7ny-0006GV-2R for guile-bugs@m.gmane.org; Fri, 24 Aug 2018 04:50:54 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:50930) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ft7kH-0001zB-PZ for bug-guile@gnu.org; Fri, 24 Aug 2018 04:47:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ft7kE-0006X7-Ew for bug-guile@gnu.org; Fri, 24 Aug 2018 04:47:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:54308) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ft7kE-0006WA-9p for bug-guile@gnu.org; Fri, 24 Aug 2018 04:47:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ft7kE-0002yU-51; Fri, 24 Aug 2018 04:47:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Fri, 24 Aug 2018 08:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.153510037211375 (code B ref 31878); Fri, 24 Aug 2018 08:47:02 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 24 Aug 2018 08:46:12 +0000 Original-Received: from localhost ([127.0.0.1]:59326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ft7jP-0002xO-J1 for submit@debbugs.gnu.org; Fri, 24 Aug 2018 04:46:12 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:57483) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ft7jN-0002x9-RY for 31878@debbugs.gnu.org; Fri, 24 Aug 2018 04:46:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ft7jD-00051r-EG for 31878@debbugs.gnu.org; Fri, 24 Aug 2018 04:46:04 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:55259) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ft7jD-00051k-9K; Fri, 24 Aug 2018 04:45:59 -0400 Original-Received: from [193.50.110.186] (port=57748 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ft7jC-0003KI-VM; Fri, 24 Aug 2018 04:45:59 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 Fructidor an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu In-Reply-To: <87in40g9bx.fsf@netris.org> (Mark H. Weaver's message of "Thu, 23 Aug 2018 15:40:50 -0400") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9134 Archived-At: Hi Mark, Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Mark H Weaver skribis: >> >>> Since Guile (unfortunately) allows cyclic module dependencies, we would >>> need a mechanism to avoid deadlocks in case modules A and B both import >>> each other, and two threads concurrently attempt to load those modules. >>> >>> The first idea that comes to mind is to also have a global structure >>> storing a partial order on the modules currently being loaded. If, >>> while module A is being loaded, there's an attempt to auto-load module >>> B, then an entry (A < B) would added to the partial order. The partial >>> order would not allow cycles to be introduced, reporting an error in >>> that case. In case a cycle would be introduced when adding (A < B), >>> then the thread would simply be given access to the partially-loaded >>> module B, by adding B to its local list of modules-being-loaded. >> >> Would it enough to (1) use recursive mutexes, and (2) have >> =E2=80=98resolve-module=E2=80=99 lookup modules first in the global name= space, and >> second in the local list of modules being loaded? > > Item (2) above is something that I had already envisioned in my > proposal, although I neglected to mention it. > > However, I don't see how recursive mutexes would help here, or how they > could obviate the need for the other mechanisms I described above. > > Suppose module A and module B are mutually dependent on each other. If > thread 1 is loading module A concurrently with thread 2 loading module > B, then thread 1 will be the only thread with access to module A (via > thread 1's local list) and will hold the lock on it, and similarly for > thread 2 and module B. > > Now, when thread 1 tries to load module B (while it's in the process of > loading module A), it should normally be blocked until module B is > finished loading. If those modules were _not_ mutually dependent on > each other, we should insist on thread 1 waiting for module B to finish > loading before gaining access to it. Only if there is a cyclic > dependency should it be granted access to the partially-loaded module. > > If we simply use recursive mutexes, I think deadlock would occur in this > case. Thread 1 would try to grab the lock on module B, which is already > held by thread 2, and vice versa. Since it's not self-held, I fail to > see the relevance of the recursive mutex. Oh, got it; you=E2=80=99re right. So yes, the solution you outlined above = is probably what=E2=80=99s needed. Thanks for explaining! Ludo=E2=80=99.