From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.bugs Subject: bug#19235: make-fresh-user-module procedure leaks memory Date: Sun, 7 Dec 2014 14:19:03 +0000 Message-ID: <20141207141903.1347c764@bother.homenet> References: <20141130232834.32cbf5b2@bother.homenet> <87iohn51dc.fsf@yeeloong.lan> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1417962029 24240 80.91.229.3 (7 Dec 2014 14:20:29 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 7 Dec 2014 14:20:29 +0000 (UTC) Cc: 19235@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Sun Dec 07 15:20:22 2014 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Xxch7-0007Xi-HO for guile-bugs@m.gmane.org; Sun, 07 Dec 2014 15:20:17 +0100 Original-Received: from localhost ([::1]:58142 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xxch7-0005xP-2l for guile-bugs@m.gmane.org; Sun, 07 Dec 2014 09:20:17 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53098) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xxcgy-0005vE-7I for bug-guile@gnu.org; Sun, 07 Dec 2014 09:20:13 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xxcgs-0003Pu-Jx for bug-guile@gnu.org; Sun, 07 Dec 2014 09:20:08 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:58867) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xxcgs-0003Pj-Fn for bug-guile@gnu.org; Sun, 07 Dec 2014 09:20:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1Xxcgs-0007A8-59 for bug-guile@gnu.org; Sun, 07 Dec 2014 09:20:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Chris Vine Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sun, 07 Dec 2014 14:20:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 19235 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 19235-submit@debbugs.gnu.org id=B19235.141796194927448 (code B ref 19235); Sun, 07 Dec 2014 14:20:02 +0000 Original-Received: (at 19235) by debbugs.gnu.org; 7 Dec 2014 14:19:09 +0000 Original-Received: from localhost ([127.0.0.1]:56076 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xxcg0-00078c-MV for submit@debbugs.gnu.org; Sun, 07 Dec 2014 09:19:09 -0500 Original-Received: from smtpout5.wanadoo.co.uk ([80.12.242.80]:53392 helo=smtpout.wanadoo.co.uk) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xxcfx-00078P-9e for 19235@debbugs.gnu.org; Sun, 07 Dec 2014 09:19:06 -0500 Original-Received: from laptop.homenet ([95.146.110.225]) by mwinf5d67 with ME id QeK21p0044rpotr03eK2hu; Sun, 07 Dec 2014 15:19:03 +0100 X-ME-Helo: laptop.homenet X-ME-Date: Sun, 07 Dec 2014 15:19:03 +0100 X-ME-IP: 95.146.110.225 Original-Received: from bother.homenet (localhost [127.0.0.1]) by laptop.homenet (Postfix) with ESMTP id CDDC88BCBE; Sun, 7 Dec 2014 14:19:03 +0000 (GMT) In-Reply-To: <87iohn51dc.fsf@yeeloong.lan> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; i686-pc-linux-gnu) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7669 Archived-At: On Sun, 07 Dec 2014 03:07:43 -0500 Mark H Weaver wrote: > Chris Vine writes: > > > The make-fresh-user-module procedure leaks memory in guile-2.0.11 as > > demonstrated by the attached test case. [...] > > > > The question which might be asked is "Would any sane person ever > > want to invoke the make-fresh-user-module procedure more than a few > > times in a practical program?". The answer to this question is > > "Yes", if guile is being used as an extension framework for a C or > > C++ program, and it executes guile extensions as individual tasks, > > and it is necessary that top levels should not to be shared. The > > execution of tasks concurrently is one such case, but there can be > > many cases where isolated top levels are desirable for tasks > > executed serially also. > > Unfortunately, Guile modules cannot be garbage collected. The problem > is that modules are usually referenced by name, not by direct > pointers. Every module must have a name, due to the way our macro > expander works. Modules created by 'make-module' or > 'make-fresh-user-module' are named using gensyms. > > We maintain a global map from module names to module objects, and we > can never safely delete from this map, because we cannot prove that > the module name won't be looked up in the future. > > I agree that your use case is reasonable. I'll think about how we > might allow unnamed modules to be collected, but I'm afraid it might > be quite difficult. > > In the meantime, I wrote a procedure that uses undocumented interfaces > to forcefully delete a module from the name->module map. However, I > must emphasize that this procedure is likely to break in a future > version of Guile. However, it should work in the 2.0.x series. [snip] > I should mention that creating new modules with > 'make-fresh-user-module' is not thread safe, nor is the procedure > above. Both of them mutate the same name->module map. For now, I > recommend protecting calls to both of them with a mutex. Thanks for that. Having make-fresh-user-module (and possibly set-current-module ??) not thread safe should be relatively easy to deal with in my use case, subject to the next paragraph. I would need to call these procedures in C/C++ code before calling scm_eval_string() to execute a scheme task, but presumably I can obtain a C variable reference for make-fresh-user-module by calling scm_c_lookup() - set-current-module already has a C interface provided by libguile. I would also need to use a POSIX mutex, but that is fine as guile uses native threads. However, that would not be completely effective if guile might also call make-fresh-user-module internally, since that would be unprotected. Are there any circumstances in which guile might do this? I know from what you said some time ago that guile module loading is not thread safe. Without asking you to exercise powers of clairvoyance, can you think of any other thread safety problems I should be on the look out for? The documentation says that "multiple threads can call scm_with_guile concurrently" and whereas that might literally be true there seem a number of other things that one might reasonably expect to do with scm_with_guile that are not. On the memory leak, the usage case involves a library so I do not think I can include code which might be liable to breakage at some indeterminate time. It would also be quite difficult to implement by reference to the same mutex as used to protect make-fresh-user-module because that would mean reimplementing your suggested code on the C/C++ side. So I think it best just to document the leakage in the documentation for the library (my one, not guile), and hope that at some time it will be possible to deal with it in guile. Chris