module GC bug

* module GC bug
@ 2005-06-09 23:32 Han-Wen Nienhuys
  2005-06-10  6:32 ` Neil Jerram
                   ` (2 more replies)
  0 siblings, 3 replies; 31+ messages in thread
From: Han-Wen Nienhuys @ 2005-06-09 23:32 UTC (permalink / raw)


[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]


Hi,

I've found a memory leak in GUILE.

The contents of modules are not garbage collected.

This seems to be related with two errors:

  - scm_stand_in_procs is a hashtable. I believe it should be weak_key
hashtable, just like the scm_object_whash table. For, if a closure is
GC'd, so should it properties.

  - in boot-9.scm, set-module-eval-closure! does

      (set-procedure-property! closure 'module module))

So the closure is a key in a weak hash-table, pointing to the module
as a value (using scm_stand_in_procs), the module is always marked
during GC. However, since the module points back to the closure via
the 'eval-closure slot, the key is always marked. Consequently,
neither closure nor module are ever GC'd.

I've fixed this by introducing a new function (eval-closure-module)
which returns the module of a closure via the eval-closure smob.  Find 
the patch (including test) attached.

I don't really know what I am doing, so please comment. FWIW, the test 
suite compiled without complaints. If noone objects, I will merge this 
patch tomorrow night.  Furthermore, I believe the same bug is in GUILE 
1.6, so a 1.6.8 release is warranted IMO.

Greetings,

Han-Wen


PS. I feel like a broken record, but when will GUILE 1.8 be out?


-- 
  Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

[-- Attachment #2: evalclos --]
[-- Type: text/plain, Size: 7736 bytes --]

? b
? core.11847
? core.8858
? evalclos
? gettext.m4
? libltdl
? t.scm
Index: INSTALL
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/INSTALL,v
retrieving revision 1.40
diff -u -r1.40 INSTALL

--- INSTALL	28 Sep 2004 19:33:40 -0000	1.40
+++ INSTALL	9 Jun 2005 23:27:25 -0000
@@ -1,7 +1,7 @@
 Installation Instructions
 *************************
 
-Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004 Free
+Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005 Free
 Software Foundation, Inc.
 
 This file is free documentation; the Free Software Foundation gives
@@ -189,8 +189,13 @@
 
      ./configure CC=/usr/local2/bin/gcc
 
-will cause the specified gcc to be used as the C compiler (unless it is
-overridden in the site shell script).
+causes the specified `gcc' to be used as the C compiler (unless it is
+overridden in the site shell script).  Here is a another example:
+
+     /bin/bash ./configure CONFIG_SHELL=/bin/bash
+
+Here the `CONFIG_SHELL=/bin/bash' operand causes subsequent
+configuration-related scripts to be executed by `/bin/bash'.
 
 `configure' Invocation
 ======================
Index: ice-9/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/ice-9/ChangeLog,v
retrieving revision 1.650
diff -u -r1.650 ChangeLog
--- ice-9/ChangeLog	5 Jun 2005 17:26:07 -0000	1.650
+++ ice-9/ChangeLog	9 Jun 2005 23:27:27 -0000
@@ -1,3 +1,9 @@
+2005-06-10  Han-Wen Nienhuys  <hanwen@xs4all.nl>
+
+	* boot-9.scm (set-module-eval-closure!): remove
+	set-procedure-property! closure 'module. Setting this property
+	causes un-gc-able modules.
+
 2005-06-05  Marius Vollmer  <mvo@zagadka.de>
 
 	* boot-9.scm (substring-fill!): New, for compatability.
Index: ice-9/boot-9.scm
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/ice-9/boot-9.scm,v
retrieving revision 1.349
diff -u -r1.349 boot-9.scm
--- ice-9/boot-9.scm	5 Jun 2005 17:24:52 -0000	1.349
+++ ice-9/boot-9.scm	9 Jun 2005 23:27:28 -0000
@@ -339,7 +339,7 @@
 
 (define (environment-module env)
   (let ((closure (and (pair? env) (car (last-pair env)))))
-    (and closure (procedure-property closure 'module))))
+    (and closure (eval-closure-module closure))))
 
 \f
 
@@ -1266,10 +1266,12 @@
   (let ((setter (record-modifier module-type 'eval-closure)))
     (lambda (module closure)
       (setter module closure)
-      ;; Make it possible to lookup the module from the environment.
-      ;; This implementation is correct since an eval closure can belong
-      ;; to maximally one module.
-      (set-procedure-property! closure 'module module))))
+      
+
+      ;; do not set procedure properties on closures.
+      ;; since procedure properties are weak-hashes, they cannot
+      ;; have cyclical data, otherwise the data cannot be GC-ed.
+      )))
 
 \f
 
Index: libguile/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/ChangeLog,v
retrieving revision 1.2281
diff -u -r1.2281 ChangeLog
--- libguile/ChangeLog	9 Jun 2005 19:33:38 -0000	1.2281
+++ libguile/ChangeLog	9 Jun 2005 23:27:35 -0000
@@ -1,3 +1,12 @@
+2005-06-10  Han-Wen Nienhuys  <hanwen@xs4all.nl>
+
+	* modules.c (s_scm_eval_closure_module): new function. Return the
+	module inside an eval-closure.
+
+	* gc.c (scm_init_storage): make scm_stand_in_procs a weak_key hash
+	table. This means that procedure properties are GC'd if the
+	procedure dies.
+
 2005-06-09  Han-Wen Nienhuys  <hanwen@xs4all.nl>
 
 	* gc.c (tag_table_to_type_alist): convert tag number to "tag %d"
Index: libguile/gc.c
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/gc.c,v
retrieving revision 1.268
diff -u -r1.268 gc.c
--- libguile/gc.c	9 Jun 2005 19:33:38 -0000	1.268
+++ libguile/gc.c	9 Jun 2005 23:27:35 -0000
@@ -935,7 +935,7 @@
 
 #endif
 
-  scm_stand_in_procs = scm_c_make_hash_table (257);
+  scm_stand_in_procs = scm_make_weak_key_hash_table (scm_from_int (257));
   scm_permobjs = SCM_EOL;
   scm_protects = scm_c_make_hash_table (31);
   scm_gc_registered_roots = scm_c_make_hash_table (31);
Index: libguile/modules.c
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/modules.c,v
retrieving revision 1.59
diff -u -r1.59 modules.c
--- libguile/modules.c	23 May 2005 19:57:20 -0000	1.59
+++ libguile/modules.c	9 Jun 2005 23:27:36 -0000
@@ -346,6 +346,19 @@
 }
 #undef FUNC_NAME
 
+
+SCM_DEFINE (scm_eval_closure_module, "eval-closure-module", 1, 0, 0,
+	    (SCM closure),
+	    "Return the module for @var{closure}.")
+#define FUNC_NAME s_scm_eval_closure_module
+{
+  SCM_ASSERT_TYPE(SCM_EVAL_CLOSURE_P (closure), closure, SCM_ARG1, FUNC_NAME, "eval-closure");
+  return SCM_PACK (SCM_CELL_WORD_1(closure));
+}
+#undef FUNC_NAME
+ 
+
+
 SCM_DEFINE (scm_standard_interface_eval_closure,
 	    "standard-interface-eval-closure", 1, 0, 0,
 	    (SCM module),
Index: libguile/modules.h
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/libguile/modules.h,v
retrieving revision 1.26
diff -u -r1.26 modules.h
--- libguile/modules.h	23 May 2005 19:57:20 -0000	1.26
+++ libguile/modules.h	9 Jun 2005 23:27:36 -0000
@@ -98,6 +98,7 @@
 SCM_API SCM scm_current_module_transformer (void);
 SCM_API SCM scm_eval_closure_lookup (SCM eclo, SCM sym, SCM definep);
 SCM_API SCM scm_standard_eval_closure (SCM module);
+SCM_API SCM scm_eval_closure_module (SCM closure);
 SCM_API SCM scm_standard_interface_eval_closure (SCM module);
 SCM_API SCM scm_get_pre_modules_obarray (void);
 SCM_API SCM scm_lookup_closure_module (SCM proc);
Index: test-suite/ChangeLog
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/ChangeLog,v
retrieving revision 1.350
diff -u -r1.350 ChangeLog
--- test-suite/ChangeLog	5 Jun 2005 21:38:22 -0000	1.350
+++ test-suite/ChangeLog	9 Jun 2005 23:27:37 -0000
@@ -1,3 +1,8 @@
+2005-06-10  Han-Wen Nienhuys  <hanwen@xs4all.nl>
+
+	* tests/gc.test ("gc"): add a test to verify that modules are
+	garbage collected.
+
 2005-06-06  Kevin Ryde  <user42@zip.com.au>
 
 	* tests/strings.test (string-split): Try splitting on an 8-bit char.
Index: test-suite/tests/gc.test
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/tests/gc.test,v
retrieving revision 1.6
diff -u -r1.6 gc.test
--- test-suite/tests/gc.test	23 May 2005 19:57:22 -0000	1.6
+++ test-suite/tests/gc.test	9 Jun 2005 23:27:37 -0000
@@ -55,3 +55,17 @@
       (gc)
       (remove-hook! after-gc-hook thunk)
       foo)))
+
+
+(with-test-prefix "gc"
+  (pass-if "Unused modules are removed"
+	   (let*
+	       ((dummy (gc))
+	        (last-count (cdr (assoc
+				  "eval-closure" (gc-live-object-stats)))))
+	       
+	     (for-each (lambda (x) (make-module)) (iota 1000))
+	     (gc)
+	     (gc) ;; twice: have to kill the weak vectors.
+	     (= last-count (cdr (assoc "eval-closure" (gc-live-object-stats)))))
+	   ))
Index: test-suite/tests/hash.test
===================================================================
RCS file: /cvsroot/guile/guile/guile-core/test-suite/tests/hash.test,v
retrieving revision 1.3
diff -u -r1.3 hash.test
--- test-suite/tests/hash.test	23 May 2005 19:57:22 -0000	1.3
+++ test-suite/tests/hash.test	9 Jun 2005 23:27:37 -0000
@@ -65,7 +65,6 @@
 ;;;
 ;;; hashx-remove!
 ;;;
-
 (with-test-prefix "hashx-remove!"
   (pass-if (->bool (object-documentation hashx-remove!)))
 

[-- Attachment #3: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

^ permalink raw reply	[flat|nested] 31+ messages in thread