impressions on gc

impressions on gc

[Andy Wingo]

Hi,

I recently poked some GC-related things in stable-2.0 and master.

One change was to make GC run more often when a process is growing, in
terms of resident memory size.  This seems to be a good idea in general.

The other was to add a function that users can call to note
non-gc-managed allocations -- allocations that may be freed when GC is
run, but which the GC doesn't know about.  That is
scm_gc_register_allocation.  This routine runs GC after allocated bytes
exceed some limit, currently the GC-managed heap size, reset after every
GC.  This is to catch steady-state mallocation.

We cannot deal well with steady-state mallocation, unknown to the GC.
If the ratio between unknown and known allocation is small enough, the
process image size won't grow too badly.  But if it is large, your
process will grow quite large.  You have to let the GC know about
allocation -- at least, about allocation that can be freed by GC.
Otherwise it can't do its job.

Those are my conclusions.  It is midnight.  Goodnight! ;-)

Andy
-- 
http://wingolog.org/

Re: impressions on gc

[Chris K. Jester-Young]

On Fri, Dec 02, 2011 at 12:09:12AM +0100, Andy Wingo wrote:
> One change was to make GC run more often when a process is growing, in
> terms of resident memory size.  This seems to be a good idea in general.
> 
> The other was to add a function that users can call to note
> non-gc-managed allocations -- allocations that may be freed when GC is
> run, but which the GC doesn't know about.  That is
> scm_gc_register_allocation.  This routine runs GC after allocated bytes
> exceed some limit, currently the GC-managed heap size, reset after every
> GC.  This is to catch steady-state mallocation.

I looked at the change for it, fd51e66190bde8cef74fec9725de4da3471901c4.
It looks cool. :-)

Alas, part of that diff also breaks libgc 7.1 (which is the version that
Debian currently has packaged), which lack the unmapping functions.
Here's my diff to make it work again.

Many thanks,
Chris.

			*	*	*

diff --git a/configure.ac b/configure.ac
index d63dd63..0cfe961 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1259,7 +1259,7 @@ save_LIBS="$LIBS"
 LIBS="$BDW_GC_LIBS $LIBS"
 CFLAGS="$BDW_GC_CFLAGS $CFLAGS"
 
-AC_CHECK_FUNCS([GC_do_blocking GC_call_with_gc_active GC_pthread_exit GC_pthread_cancel GC_allow_register_threads GC_pthread_sigmask GC_set_start_callback GC_get_heap_usage_safe GC_get_free_space_divisor])
+AC_CHECK_FUNCS([GC_do_blocking GC_call_with_gc_active GC_pthread_exit GC_pthread_cancel GC_allow_register_threads GC_pthread_sigmask GC_set_start_callback GC_get_heap_usage_safe GC_get_free_space_divisor GC_gcollect_and_unmap GC_get_unmapped_bytes])
 
 # Though the `GC_do_blocking ()' symbol is present in GC 7.1, it is not
 # declared, and has a different type (returning void instead of
diff --git a/libguile/gc-malloc.c b/libguile/gc-malloc.c
index 72142ca..b7781f3 100644
--- a/libguile/gc-malloc.c
+++ b/libguile/gc-malloc.c
@@ -96,7 +96,11 @@ scm_realloc (void *mem, size_t size)
     return ptr;
 
   /* Time is hard: trigger a full, ``stop-the-world'' GC, and try again.  */
+#ifdef HAVE_GC_GCOLLECT_AND_UNMAP
   GC_gcollect_and_unmap ();
+#else
+  GC_gcollect ();
+#endif
 
   SCM_SYSCALL (ptr = realloc (mem, size));
   if (ptr)
diff --git a/libguile/gc.c b/libguile/gc.c
index ff1ebe6..0ab27a5 100644
--- a/libguile/gc.c
+++ b/libguile/gc.c
@@ -205,7 +205,11 @@ GC_get_heap_usage_safe (GC_word *pheap_size, GC_word *pfree_bytes,
 {
   *pheap_size = GC_get_heap_size ();
   *pfree_bytes = GC_get_free_bytes ();
+#ifdef HAVE_GC_GET_UNMAPPED_BYTES
   *punmapped_bytes = GC_get_unmapped_bytes ();
+#else
+  *punmapped_bytes = 0;
+#endif
   *pbytes_since_gc = GC_get_bytes_since_gc ();
   *ptotal_bytes = GC_get_total_bytes ();
 }

Re: impressions on gc

[Andy Wingo]

On Thu 08 Dec 2011 16:46, "Chris K. Jester-Young" <cky944@gmail.com> writes:

> Alas, part of that diff also breaks libgc 7.1 (which is the version that
> Debian currently has packaged), which lack the unmapping functions.
> Here's my diff to make it work again.

Thanks for the patch!  In the future a git-format-patch attachment is
easiest.  Though, if you like, we can set you up with commit access, I
think.  Request to be added to the `guile' group on savannah and we'll
approve.  I'll push this one though.

Regards,

Andy
-- 
http://wingolog.org/

Re: impressions on gc

[Noah Lavine]

Hello,

As long as we're pinging people for 2.0.5, I don't think this patch
ever got pushed. :-)

I can't build master right now. This is partly my fault for doing so
little sysadmin work that I still have libgc 7.1, but I still think
this one should really, really be in 2.0.5 if the GC changes will also
be there.

Noah

On Thu, Dec 8, 2011 at 3:10 PM, Andy Wingo <wingo@pobox.com> wrote:
> On Thu 08 Dec 2011 16:46, "Chris K. Jester-Young" <cky944@gmail.com> writes:
>
>> Alas, part of that diff also breaks libgc 7.1 (which is the version that
>> Debian currently has packaged), which lack the unmapping functions.
>> Here's my diff to make it work again.
>
> Thanks for the patch!  In the future a git-format-patch attachment is
> easiest.  Though, if you like, we can set you up with commit access, I
> think.  Request to be added to the `guile' group on savannah and we'll
> approve.  I'll push this one though.
>
> Regards,
>
> Andy
> --
> http://wingolog.org/
>

Re: impressions on gc

[Ludovic Courtès]

Hi Noah,

Noah Lavine <noah.b.lavine@gmail.com> skribis:

> As long as we're pinging people for 2.0.5, I don't think this patch
> ever got pushed. :-)

Apparently it was applied as 4eb286127c41e67eb90ef1b69f61f613bcd830b2.

Thanks,
Ludo’.