From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#31879: Caching in the "module import obarray" is not thread-safe Date: Mon, 18 Jun 2018 15:48:33 +0200 Message-ID: <877emwurwu.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1529329634 29481 195.159.176.226 (18 Jun 2018 13:47:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 18 Jun 2018 13:47:14 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) To: 31879@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Mon Jun 18 15:47:10 2018 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fUuUu-0007XO-46 for guile-bugs@m.gmane.org; Mon, 18 Jun 2018 15:47:08 +0200 Original-Received: from localhost ([::1]:34875 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUuX1-0002NQ-5u for guile-bugs@m.gmane.org; Mon, 18 Jun 2018 09:49:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33036) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUuWn-0002N4-Ln for bug-guile@gnu.org; Mon, 18 Jun 2018 09:49:11 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUuWk-0005CW-Jl for bug-guile@gnu.org; Mon, 18 Jun 2018 09:49:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:46607) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fUuWk-0005CQ-Fi for bug-guile@gnu.org; Mon, 18 Jun 2018 09:49:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fUuWk-0001v6-8l for bug-guile@gnu.org; Mon, 18 Jun 2018 09:49:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 18 Jun 2018 13:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 31879 X-GNU-PR-Package: guile X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-guile@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.15293297307360 (code B ref -1); Mon, 18 Jun 2018 13:49:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 18 Jun 2018 13:48:50 +0000 Original-Received: from localhost ([127.0.0.1]:54504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fUuWY-0001ue-0Y for submit@debbugs.gnu.org; Mon, 18 Jun 2018 09:48:50 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:42011) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fUuWW-0001uS-Ub for submit@debbugs.gnu.org; Mon, 18 Jun 2018 09:48:49 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUuWQ-00054Q-SU for submit@debbugs.gnu.org; Mon, 18 Jun 2018 09:48:43 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:39829) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fUuWQ-00054M-Om for submit@debbugs.gnu.org; Mon, 18 Jun 2018 09:48:42 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:32977) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUuWM-0002K5-JX for bug-guile@gnu.org; Mon, 18 Jun 2018 09:48:42 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fUuWJ-0004v8-Il for bug-guile@gnu.org; Mon, 18 Jun 2018 09:48:38 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38344) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fUuWJ-0004ur-Ez for bug-guile@gnu.org; Mon, 18 Jun 2018 09:48:35 -0400 Original-Received: from [193.50.110.191] (port=59408 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fUuWJ-0001jF-23 for bug-guile@gnu.org; Mon, 18 Jun 2018 09:48:35 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 Prairial an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:9074 Archived-At: Dear thread lovers, The code below spawns a bunch of threads that look up imported bindings in a module. It usually enters an infinite loop and possibly eats all your memory (on Guile 2.2.3): --8<---------------cut here---------------start------------->8--- (use-modules (ice-9 threads) (ice-9 match) (srfi srfi-1)) (define bindings ;; Bindings exported by (guile). '()) (module-for-each (lambda (name var) (set! bindings (cons name bindings))) (resolve-module '(guile))) (define thread-count 8) (define bindings-per-thread (floor (/ (length bindings) thread-count))) (define iface (resolve-module '(ice-9 q))) (define threads (unfold (lambda (x) (>=3D x thread-count)) (lambda (n) (call-with-new-thread (lambda () (let loop ((bindings (take (drop bindings (* n bindings-per-thread)) bindings-per-thread))) (match bindings (() #t) ((head . tail) (module-variable iface head) (loop tail))))))) 1+ 0)) (for-each join-thread threads) (pk 'done thread-count) --8<---------------cut here---------------end--------------->8--- The issue lies in the =E2=80=9Cimport obarray=E2=80=9D, which is used as a = cache and is accessed in a non-thread-safe manner: --8<---------------cut here---------------start------------->8--- static inline SCM module_imported_variable (SCM module, SCM sym) { #define SCM_BOUND_THING_P scm_is_true register SCM var, imports; /* Search cached imported bindings. */ imports =3D SCM_MODULE_IMPORT_OBARRAY (module); var =3D scm_hashq_ref (imports, sym, SCM_UNDEFINED); if (SCM_BOUND_THING_P (var)) return var; [...] if (SCM_BOUND_THING_P (found_var)) { /* Save the lookup result for future reference. */ (void) scm_hashq_set_x (imports, sym, found_var); return found_var; } --8<---------------cut here---------------end--------------->8--- Possible solutions: 1. Add a field in the module record containing a mutex. Downside is that this breaks the ABI, though I=E2=80=99m not sure how much of a pr= oblem it is. 2. Make the import obarray a weak hash table since they are thread-safe in 2.2. This should be semantically equivalent to using a hash table provided interned symbols are not GC=E2=80=99d. Thoughts? Ludo=E2=80=99.