state of Guile GC

state of Guile GC

[Julian Graham]

Hi Guilers,

Disclaimer: I don't know very much about Guile's garbage collector,
technically or historically, except for what I've gleaned by some
cursory inspection of the code looking for (magical, undocumented)
performance tricks.

So I know I'm kind of blundering in here, but I'm having some trouble
and don't really know what to do.  Specifically, I've got some
performance-sensitive (i.e., user-interactive) C code that uses Guile
for making calls to some Scheme libraries I've written, and those
calls are taking a prohibitively long time to complete.  Most of the
time, it takes <10ms (avg. 4ms) to complete, but when there's a GC (as
observed by doing logging in a GC "before" hook), the call can take
upwards of 100ms.

I realize that a lot of this has to do with the allocation profile of
my Scheme code, and I'm working on profiling it; I also know the GC's
supposed to be pretty transparent to the libguile user; nonetheless,
I'm hoping some Guile developers (Ludovic?) or some developers that
use Guile for interactive applications (Han-Wen?) could clear up a few
things for me:

* In 1.8, you guys removed scm_block_gc, and the comments in the
Changelog seem to indicate that this was a requirement because of the
way the redesigned GC interleaves itself with "Guile mode" C code.
Would it really be impossible to re-add some way of (temporarily)
disabling GC?

* Any suggestions from application authors on how to control the
behavior of the GC from C?  dsmith on #guile suggested that there
might be a way to trick Guile into thinking a GC wasn't necessary by
flubbing the GC stats somehow, but I couldn't figure out a way to do
that that looks portable.

* Out of curiosity, what are the technical / theoretical constraints
that make Guile's GC perform the way it does?  There are other
garbage-collected languages with (what I would imagine are) much
heavier allocation footprints (C#, Java) that seem to have much faster
GC.  (P'raps they collect in a separate thread?)  Do other Schemes do
things differently?

* What's the state of porting Guile to use Boehm GC?  I was following
Ludovic's research into it but things kind of trailed off about a year
ago when he posted some rather unhappy performance information.


Thanks a lot,
Julian


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: state of Guile GC

[Ludovic Courtès]

Hi Julian,

"Julian Graham" <joolean@gmail.com> writes:

> I realize that a lot of this has to do with the allocation profile of
> my Scheme code, and I'm working on profiling it;

It's not only your fault: the evaluator conses every time a Scheme
procedure is invoked, including when invoked in tail position.  That
ought to be fixed.

BTW, did you notice a significant change compared to your previous Guile
version?  Were you actually using `scm_block_gc ()' before?

You might want to build a single-threaded Guile 1.8 if possible, which
should perform much better, although that is admittedly not very
convenient.

> * In 1.8, you guys removed scm_block_gc, and the comments in the
> Changelog seem to indicate that this was a requirement because of the
> way the redesigned GC interleaves itself with "Guile mode" C code.
> Would it really be impossible to re-add some way of (temporarily)
> disabling GC?

Sorry, I don't know what the rationale was, maybe others can help.

> * Any suggestions from application authors on how to control the
> behavior of the GC from C?  dsmith on #guile suggested that there
> might be a way to trick Guile into thinking a GC wasn't necessary by
> flubbing the GC stats somehow, but I couldn't figure out a way to do
> that that looks portable.

Maybe the `GUILE_MIN_YIELD_{1,2}' environment variables can help, as I
noted in a recent post.  Did you try playing with them?

But that's not really targeting the "application authors", rather the
end-user...

> * Out of curiosity, what are the technical / theoretical constraints
> that make Guile's GC perform the way it does?  There are other
> garbage-collected languages with (what I would imagine are) much
> heavier allocation footprints (C#, Java) that seem to have much faster
> GC.  (P'raps they collect in a separate thread?)  Do other Schemes do
> things differently?

It's hard to tell whether it's really so inefficient (although looking
at the code gives the impression that there's room for
micro-optimizations).

> * What's the state of porting Guile to use Boehm GC?  I was following
> Ludovic's research into it but things kind of trailed off about a year
> ago when he posted some rather unhappy performance information.

He.  ;-)

Then there were happier pieces of information, namely about how `libgc'
has to be compiled to get better performance [0].  At this stage,
libgc-enabled Guile performs comparably to the current Guile.  The thing
is: good performance measurements are needed, including an analysis of
the execution time/memory usage tradeoff.  This has been on my TODO list
for a while now...

Also, it's been reported that libgc-Guile provides lower latency than
regular Guile in SND [1].

Hope this helps,
Ludovic.

[0] http://article.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/1516/
[1] http://thread.gmane.org/gmane.lisp.guile.devel/6624



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: state of Guile GC

[Julian Graham]

[I'm always forgetting to CC the list...]

Hi Ludovic,


> BTW, did you notice a significant change compared to your previous Guile
> version?  Were you actually using `scm_block_gc ()' before?

No, I wasn't -- I actually found out about scm_block_gc by googling
"Guile performance."  It came up on a mailing list where some Gnucash
developers were fussing over their build.

> Maybe the `GUILE_MIN_YIELD_{1,2}' environment variables can help, as I
> noted in a recent post.  Did you try playing with them?
>
> But that's not really targeting the "application authors", rather the
> end-user...

Actually, I just tried -- boosting those values actually cuts down on
GC thrashing considerably.  Thanks!  (Plus, I don't actually think
it's out of the question to have the application set those variables
before initializing Guile.)

> Hope this helps,
> Ludovic.

It does, a great deal!  Thanks so much for the information.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user