From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#31878: Module autoloading is not thread safe Date: Mon, 22 Oct 2018 12:10:15 +0200 Message-ID: <87k1manvg8.fsf@gnu.org> References: <87k1qwwhu2.fsf@gnu.org> <878t7cwdqu.fsf@gnu.org> <87h8m0uw3z.fsf@gnu.org> <878t4xdfag.fsf@netris.org> <87woshbzak.fsf@netris.org> <87va81p4sn.fsf@gnu.org> <87in40g9bx.fsf@netris.org> <871s8j17xq.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1540202942 6870 195.159.176.226 (22 Oct 2018 10:09:02 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 22 Oct 2018 10:09:02 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: 31878@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Mon Oct 22 12:08:58 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gEX8s-0001h9-66 for guile-bugs@m.gmane.org; Mon, 22 Oct 2018 12:08:58 +0200 Original-Received: from localhost ([::1]:34060 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEXAy-0007Gf-Gn for guile-bugs@m.gmane.org; Mon, 22 Oct 2018 06:11:08 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57412) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEXAt-0007GK-6F for bug-guile@gnu.org; Mon, 22 Oct 2018 06:11:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gEXAs-0002DP-6h for bug-guile@gnu.org; Mon, 22 Oct 2018 06:11:03 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:59125) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gEXAs-0002DF-2I for bug-guile@gnu.org; Mon, 22 Oct 2018 06:11:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gEXAr-0000Fx-Rc; Mon, 22 Oct 2018 06:11:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 22 Oct 2018 10:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31878 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 31878-submit@debbugs.gnu.org id=B31878.1540203030945 (code B ref 31878); Mon, 22 Oct 2018 10:11:01 +0000 Original-Received: (at 31878) by debbugs.gnu.org; 22 Oct 2018 10:10:30 +0000 Original-Received: from localhost ([127.0.0.1]:35150 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gEXAM-0000FB-6W for submit@debbugs.gnu.org; Mon, 22 Oct 2018 06:10:30 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:47610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gEXAJ-0000Ex-Ni for 31878@debbugs.gnu.org; Mon, 22 Oct 2018 06:10:28 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gEXAD-0001ij-Jf for 31878@debbugs.gnu.org; Mon, 22 Oct 2018 06:10:22 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:48282) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEXAD-0001iQ-EU; Mon, 22 Oct 2018 06:10:21 -0400 Original-Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=41630 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gEXA9-0007Vd-KH; Mon, 22 Oct 2018 06:10:21 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 1 Brumaire an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu In-Reply-To: <871s8j17xq.fsf@netris.org> (Mark H. Weaver's message of "Sun, 21 Oct 2018 14:16:49 -0400") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9240 Archived-At: Hi Mark, Mark H Weaver skribis: > I've written a preliminary patch to implement the improved thread-safe > module autoloading that I outlined in earlier messages in this bug > report. Great, thanks a lot! On a cursory look, it LGTM. I wonder if we could somehow split =E2=80=98resolve-module=E2=80=99 and =E2=80=98try-module-autoload=E2=80=99 = into smaller chunks because they=E2=80=99ve become quite large and complex. For example: > + (lambda () > + (when mutex > + ((ice-9-threads 'lock-mutex) mutex)) > + (set-autoloaded! dir-hint name didit))) > + > + ;; If the local module was actually created, then we > + ;; now add it to the global module table. > + (let ((module (car lazy-module-cell))) > + (when module > + (module-define-submodule! parent-module > + (car reverse-name) > + module))) > + > + ;; Signal all threads waiting on the condition variable > + ;; for this module to be loaded. > + (when cond-var > + ((ice-9-threads 'broadcast-condition-variable) cond-var)) > + > + ;; Remove the module from '%modules-being-loaded'. > + (set! %modules-being-loaded > + (assoc-remove! %modules-being-loaded > + module-name)) > + > + ;; Remove all '%modules-waiting-for' entries that are > + ;; directly related to the module that we just loaded > + ;; (or attempted to load). > + (set! %modules-waiting-for > + (filter! (lambda (entry) > + (not (or (equal? module-name (car entry)) > + (equal? module-name (cdr entry))= ))) > + %modules-waiting-for)) Perhaps this bit could go in a =E2=80=98record-module-autoload-completion!= =E2=80=99 procedure or something along these lines? It=E2=80=99s great that you came up with tests to reproduce the problems. I confirm that =E2=80=9C./check-guile threads.test=E2=80=9D works for me. Th= e tests pass even if I remove the {boot-9,threads}.scm changes, though. However peg.test fails and it may be related: --8<---------------cut here---------------start------------->8--- $ ./check-guile peg.test Testing /data/src/guile-2.1/meta/guile ... peg.test with GUILE_LOAD_PATH=3D/data/src/guile-2.1/test-suite Running peg.test Backtrace: In ice-9/eval.scm: 293:34 19 (_ #) In ice-9/boot-9.scm: 3032:4 18 (define-module* _ #:filename _ #:pure _ #:version _ #:imports = _ #:exports _ #:replacements _ # _ # _ \u2026) 2071:24 17 (call-with-deferred-observers #) 3045:24 16 (_) 222:29 15 (map1 (((test-suite lib)) ((ice-9 peg)) ((ice-9 pretty-print))= ((srfi srfi-1)))) 222:17 14 (map1 (((ice-9 peg)) ((ice-9 pretty-print)) ((srfi srfi-1)))) 2958:17 13 (resolve-interface (ice-9 peg) #:select _ #:hide _ #:prefix _ = #:renamer _ #:version _) In ice-9/threads.scm: 390:8 12 (_ _) In ice-9/boot-9.scm: 2881:28 11 (_ #) In ice-9/threads.scm: 390:8 10 (_ _) In ice-9/boot-9.scm: 3171:20 9 (_ #) 2312:4 8 (save-module-excursion #) 3191:26 7 (_) In unknown file: 6 (primitive-load-path "ice-9/peg" #) In ice-9/peg.scm: 20:0 5 (_) In ice-9/boot-9.scm: 3032:4 4 (define-module* _ #:filename _ #:pure _ #:version _ #:imports = _ #:exports _ #:replacements _ # _ # _ \u2026) 3045:24 3 (_) 222:17 2 (map1 (((ice-9 peg codegen)) ((ice-9 peg string-peg)) ((ice-9 = peg simplify-tree)) ((ice-9 peg #)) #)) 2961:6 1 (resolve-interface _ #:select _ #:hide _ #:prefix _ #:renamer = _ #:version _) In unknown file: 0 (scm-error misc-error #f "~A ~S" ("no code for module" (ice-9 = peg codegen)) #f) ERROR: In procedure scm-error: no code for module (ice-9 peg codegen) --8<---------------cut here---------------end--------------->8--- Thoughts? Thank you! Ludo=E2=80=99.