Re: out-of-control GC

[Marko Rauhamaa <marko@pacujo.net>]

Linas Vepstas <linasvepstas@gmail.com>:

> On Sun, Sep 10, 2017 at 3:36 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
>
>> Linas Vepstas <linasvepstas@gmail.com>:
>> > which suggests that the vast majority of time is spent in GC, and not
>> > in either scheme/guile, or my wrapped c++ code.
>>
>> That may well be normal.
>
> It might be "normal", but its objectionable. If 90% of the cpu time is
> spent in GC, and almost nothing at all was actually collected, then
> its too much.

Hard to say. I don't know how much garbage Guile generates as part of
its normal execution model, but for example Python generates garbage
with each method call. It is very natural for each application of a Lisp
function to create a throw-away data structure through substitution.

> On the Amazon cloud, you get to pay by the minute. On laptops with
> batteries, people complain. Especially when they are apple users, not
> linux users.

GC overhead is like RAM cache overhead or C function call overhead. Its
the price of doing business.

> If the compute job takes a week, and your user has discovered this GC
> thing, then the user will conclude that it should take less than 24
> hours, and will tell you that you are stupid, and that guile sucks. I
> have been told I'm stupid, and that guile sucks too many times.
>
> CPU cycles aren't free.

I have no idea how good Guile's performance is. Python's performance is
probably about 100 times worse than C, yet it's the fast becoming the
most popular programming language in the world.

Not everything in the world is done for performance. The #1 purpose for
a programming language is managing complexity. Relatively few
computation tasks require excellent performance, and there are other
programming languages for that: C, C++, Java, C#, go

Note that Java, C# and go are performant despite GC. GC generally
doesn't have to be all that costly throughput-wise. The biggest
performance drain for high-level programming languages such as Scheme or
Python is the absence of compile-time typing. In fact, Python is adding
optional static type annotation (which I'm afraid might be a big
mistake).

> However, guile uses gc_malloc_atomic for these strings, and so GC
> should never-ever peer into the contents of them. So I don't think its
> "accidental heap addresses". Its probably fragmentation, and the very
> heavy churn of large malloc chunks.

You aren't thrashing by any chance...?

> So let me redirect: is there some central spot within guile, where the
> decision to run GC is made? If so, what file would that be in? If not,
> is there some string I should grep for?

I believe that decision is left for GC_malloc(1L):

   GC_malloc will attempt to reclaim inaccessible space automatically by
   invoking a conservative garbage collector at appropriate points.


Marko