Re: Mark procedures and LilyPond

[Mark H Weaver <mhw@netris.org>]

Andy Wingo <wingo@pobox.com> writes:

> On Thu 05 Nov 2015 11:29, ludo@gnu.org (Ludovic Courtès) writes:
>
>> What we need above all is to address LilyPond’s use case.  I proposed a
>> solution at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19883#23> but
>> never understood whether/why it was considered unfit.
>
> I agree with you that the patch there looks reasonable to me too, though
> AFAIU the original code should work just fine too.
>
> There area few things at play.
>
>  (1) A bug related to SMOB finalization and marking that affects
>      LilyPond
>
>  (2) The utility of mark procedures in general
>
>  (3) The suitability of mark procedures for future uses
>
>  (4) Whether we can get by without mark procedures, and if so, how.
>
> For (1) it seems to me that we just have a bug.  A SMOB mark function
> was called on an object after the finalizer.  ****Note**** that having
> the finalizer called doesn't mean that the GC object was collected -- it
> just means it was collectable, perhaps in a clique of objects.
> Finalization being asynchronous with marking it's possible that a clique
> of objects was only half-finalized when a new mark procedure runs.  The
> mark procedure saw an object on which free() was already called -- this
> is possible.

Yes, exactly.

> We should fix Guile so to "null out" the SMOB typecode when the SMOB
> finalizer is called.  If our mark procedure sees a SMOB that has already
> been finalized, it just returns.

Unfortunately, I doubt this will be sufficient for LilyPond.  The small
example case in <http://bugs.gnu.org/19883>, which is apparently
representative of how things are typically done in LilyPond, has
structures like this:

                   __________                    __________
  Objects in      |          |                  |          |
  GC-managed      |  SMOB 1  |                  |  SMOB 2  |
     heap         |__________|                  |__________|
                     |   ^                         |   ^
.....................|...|.........................|...|..........
                   __v___|___      _________     __v___|___
  Objects in      |          |    |   STL   |   |          |
  normal heap     |C++ object|--->|container|-->|C++ object|
  (not scanned    |__________|    |_________|   |__________|
     by GC)


The SMOB finalizers free the associated C++ objects below them.  Now,
suppose that none of the objects above are reachable, so both SMOBs are
queued for finalization.  Now suppose that SMOB 2 is finalized first,
thus freeing the C++ object below it.  Suppose further that we null out
the SMOB 2 typecode.

Now another GC occurs and the marker is called on SMOB 1.  It's typecode
has not yet been nulled, so the user-specified mark procedure is called.
The mark procedure for SMOB 1 iterates over the STL container, and for
each C++ object within, marks the corresponding SMOB via the upward
pointer, in this case SMOB 2.  Although SMOB 2's typecode has been
zeroed out, the discovery of the fact is too late, because the (freed)
C++ object below it has been referenced.

As far as I can tell, this way of doing things depends on the old 1.8 GC
finalization semantics, where all of the finalizers are called before
the mutator resumes execution.

      Mark