Re: out-of-control GC

[Marko Rauhamaa <marko@pacujo.net>]

Linas Vepstas <linasvepstas@gmail.com>:
> which suggests that the vast majority of time is spent in GC, and not
> in either scheme/guile, or my wrapped c++ code.

That may well be normal.

> 4 gc's/minute means one every 15 seconds, which sounds reasonable,
> except that RAM usage is so huge, that each GC takes many seconds to
> complete.

That may or may not be normal. However, it should not be a function of
how much RAM has been allocated ourside Guile's heap.

> Anyway, this demo gives a scenario where gc seems to run too often.

How often is too often? I would imagine the objective is to minimize the
stop-the-world periods. IOW, you usually want the program to run very
steadily.

> I have other scenarios where it does not run often enough -- I have
> one where the working set is about 200MBytes, but guile happily grows
> to 10GB or 20GB RAM over a few hours.

You mean the size of Guile's heap? That seems pathological. Maybe there
are lots of values in your data that accidentally look like heap
addresses.

> My solution for that other case is to manually GC whenever gc-stats
> gets above 1GB, because I know a priori my working set is smaller than
> that.
>
> The overall message seems to be that gc either runs too often, or not
> often enough. Some sort of better planning is needed.
>
> I'm motivated to explore this, as it actually impacts my work.

Absolutely, although as an application programmer you should be able to
trust that GC just works.


Marko