From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Thu, 23 Aug 2018 15:40:50 -0400 Message-ID: <87in40g9bx.fsf@netris.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> <878t4xdfag.fsf@netris.org> <87woshbzak.fsf@netris.org> <87va81p4sn.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1535053321 29851 195.159.176.226 (23 Aug 2018 19:42:01 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 23 Aug 2018 19:42:01 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: 31878@debbugs.gnu.org To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Aug 23 21:41:56 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsvUS-0007g7-Hc for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 21:41:56 +0200 Original-Received: from localhost ([::1]:38539 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsvWY-0006kN-Ni for guile-bugs@m.gmane.org; Thu, 23 Aug 2018 15:44:06 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:43308) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fsvWR-0006jv-11 for bug-guile@gnu.org; Thu, 23 Aug 2018 15:44:02 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fsvVW-0002Cv-QU for bug-guile@gnu.org; Thu, 23 Aug 2018 15:43:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:54080) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fsvVW-0002Cm-Mm for bug-guile@gnu.org; Thu, 23 Aug 2018 15:43:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fsvVW-00059p-FA; Thu, 23 Aug 2018 15:43:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 23 Aug 2018 19:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.153505335019785 (code B ref 31878); Thu, 23 Aug 2018 19:43:02 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 23 Aug 2018 19:42:30 +0000 Original-Received: from localhost ([127.0.0.1]:59098 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsvV0-000593-5g for submit@debbugs.gnu.org; Thu, 23 Aug 2018 15:42:30 -0400 Original-Received: from world.peace.net ([64.112.178.59]:40460) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fsvUy-00058k-Lg for 31878@debbugs.gnu.org; Thu, 23 Aug 2018 15:42:28 -0400 Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1fsvUs-0006tV-KR; Thu, 23 Aug 2018 15:42:22 -0400 In-Reply-To: <87va81p4sn.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 23 Aug 2018 15:54:00 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9131 Archived-At: Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> Since Guile (unfortunately) allows cyclic module dependencies, we would >> need a mechanism to avoid deadlocks in case modules A and B both import >> each other, and two threads concurrently attempt to load those modules. >> >> The first idea that comes to mind is to also have a global structure >> storing a partial order on the modules currently being loaded. If, >> while module A is being loaded, there's an attempt to auto-load module >> B, then an entry (A < B) would added to the partial order. The partial >> order would not allow cycles to be introduced, reporting an error in >> that case. In case a cycle would be introduced when adding (A < B), >> then the thread would simply be given access to the partially-loaded >> module B, by adding B to its local list of modules-being-loaded. > > Would it enough to (1) use recursive mutexes, and (2) have > =E2=80=98resolve-module=E2=80=99 lookup modules first in the global name = space, and > second in the local list of modules being loaded? Item (2) above is something that I had already envisioned in my proposal, although I neglected to mention it. However, I don't see how recursive mutexes would help here, or how they could obviate the need for the other mechanisms I described above. Suppose module A and module B are mutually dependent on each other. If thread 1 is loading module A concurrently with thread 2 loading module B, then thread 1 will be the only thread with access to module A (via thread 1's local list) and will hold the lock on it, and similarly for thread 2 and module B. Now, when thread 1 tries to load module B (while it's in the process of loading module A), it should normally be blocked until module B is finished loading. If those modules were _not_ mutually dependent on each other, we should insist on thread 1 waiting for module B to finish loading before gaining access to it. Only if there is a cyclic dependency should it be granted access to the partially-loaded module. If we simply use recursive mutexes, I think deadlock would occur in this case. Thread 1 would try to grab the lock on module B, which is already held by thread 2, and vice versa. Since it's not self-held, I fail to see the relevance of the recursive mutex. What do you think? Mark