Mark procedures

Mark procedures

[Andy Wingo]

Greetings Mark!  And hello to guile-devel :)  Sorry for being
incommunicado in these last months.  The good news is that we're finally
ready to release 2.1.1, yay!  I'll be easing myself back into the
mailing list soon but I wanted to expand on a topic we discussed over
IRC yesterday: mark procedures.

To recap, since the ages of yore, Guile has allowed SMOB implementations
to specify "mark procedures".  Indeed in the olden days, these
procedures were practically required, as the GC wouldn't otherwise trace
any field of a SMOB.  When we started using the BDW conservative
collector, mark procedures became less necessary.  At least in master,
Guile itself doesn't specify any SMOB mark procedures, and the only
markers set via the lower-level "gc_kind" mechanism from BDW-GC are to
implement weak tables, and to precisely mark VM stacks.

I would ideally like a world in which mark procedures don't exist.  Mark
procedures have many really negative things about them: they are (1)
insufficient for what we want to do with them, (2) extraordinarily
error-prone, (3) (mostly) unnecessary, and (4) slow.  Let's take them
one by one.

Firstly, if we consider things we would like to do with our GC, mark
procedures don't help us get there.  They don't help with a moving GC;
that prototype would rather be "void (*mark) (SCM* scm_loc)", to allow
Guile to relocate a pointer, and even that doesn't quite cut it because
we allow for non-SCM-valued pointers as well.  What you need is a slot
visitor; something like
https://chromium.googlesource.com/v8/v8.git/+/master/src/objects-inl.h#1533.
This is the same facility you need to do heap profiling as well -- you
need both objects, and you get that by working on void** instead of
void*.

Secondly, mark procedures are an ***amazing*** foot-gun.  They run
concurrently with the user's program -- on random threads, if the
collector is doing parallel marking, but even without parallel marking,
a mark procedure can run from any allocation site -- or indeed any site,
if the program has multiple threads.  The mutator can hold almost any
lock at the time the mark procedure is called.  With finalizers we have
been able to avoid this via `scm_set_automatic_finalization_enabled' and
`scm_run_finalizers' but with mark procedures this isn't possible.  It's
also really hard to write a good mark procedure, once you start trying
anything interesting; and if you're not doing anything interesting, you
surely don't need a mark procedure, and can just rely on the builtin
tracer.  Additionally when you mess up somehow with your mark procedure,
the errors you get will appear far from their cause.

Thirdly, mark procedures are usually unnecessary.  By far the simplest
way around a mark procedure is to just let the conservative tracer
handle it, and just make sure your SMOB or whatever object has fields to
the things you need to trace.  However thinking forward to a
re-introduction of precise GC so that we can relocate objects, maybe
this is unsatisfactory; and yet mark procedures are still insufficient
for this case as I noted.  There are a few options we have today and a
few more things we could build in the future:

  * If your SMOB A holds on to field B and there is no path from B to A,
    you can just scm_gc_protect_object on B, and arrange for A's
    finalizer to scm_gc_unprotect_object on B.  Finalizers are safer
    than mark procedures (and that's saying something).

  * Or you could add an association from A to B to a weak-key table.

  * If there is a possibility of a path from B to A, you need an
    ephemeron table, and Guile doesn't do that right now.  But it
    should!

  * If your object A has a fixed shape, in that it will always hold on
    to an object B and that object will be in word N of the SMOB, then
    the SMOB can have an associated descriptor indicating how the object
    is laid out.  Guile's GC could use this descriptor to know what to
    mark, whether the values are tagged or not (if we adopted a
    different tagging system that made this necessary), and so on.  This
    is already possible with GC_make_descriptor.

  * If your object A does not have a fixed shape and you are using mark
    procedures to punch through other C/C++ data structures to grab
    references, it is probable that you have already gotten your mark
    function wrong due to concurrency.  The C/C++ data structures are
    not necessarily in a suitable state for access.  This is the case in
    which mark procedures actually do something for you, but probably it
    means randomly crashing your program, so there's that.

Finally, mark procedures are slow.  They call out from the hot path in
GC and they recurse back into the library via scm_gc_mark.  scm_gc_mark
actually communicates state via thread-local variables, which are slow
as well.  Their existence can prevent us as hackers from making changes
to the GC that would improve performance for users that don't use mark
procedures.  Probably the fastest way to use them is for each object
kind with mark procedures to be allocated out of its own pool, meaning
that objects with mark procedures increase fragmentation of the system
as a whole.

In summary, mark procedures have a lot of points against them,
especially as currently implemented!  There's only one case that I am
aware of in which mark procedures actually bring anything to the
table -- the case in which you punch through complicated data structures
that the GC can't trace -- but (a) I'm not sure that case wouldn't be
served just as well via ephemerons (b) I seriously doubt that any such
mark procedure implementation is actually correct.  Besides the fact
that the formulation of mark procedures as they are doesn't serve our
future purposes :P

Not even the JNI makes the mistake of exporting mark functions, nor do
any of the high-performance JavaScript implementations, nor does Lua,
etc etc.  At the most we should be making our foreign objects define
type descriptors and letting Guile itself handle the tracing
implementation.

At the end of our IRC conversation yesterday you put the burden on me to
argue against mark procedures, which was fair, but at this point I think
we would need good arguments for keeping them, at least in the long run
:)  In the short run as we add new APIs like the foreign object
facility, I think it makes sense to *avoid* adding mark functions to the
new APIs, at least for now.  We don't have the use cases right now and
so we would surely specify the wrong thing.  We can always add something
later.

Regards,

Andy

Re: Mark procedures

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 11098 bytes --]

Good post!

I first like to second the need for our own gc.

Regarding mark procedures. I prefer that we collect the use cases and make
sure that the framework can handle them in
a robust and effective way. I don't like to write mark procedures at all
and your fixes seam to address quite a lot of my own
hack using mark procedures.

So let's see if we can support guile-log's logical programs. I can run
quite advanced programs from the
prolog world like constraint programming in finite domains. To run these
programs you need gc'ing of
logical variables. The problem with an assoc datastructure to represent
prolog variables are that they
keep a reference to all assigned variables and you loose the ability to gc
them. kanren has a few trick
that can help but they are of the kind "if your lucky you will not blow the
stack" so I can't use that. What
I do is to maintain all logical asignement in a global stack and work with
that hands on.

One of the essential features is to detect when a variable in the stack is
ready for garbage collection. How to solve this?

This is my plan
finalizers:
if v is in the stack, then I would mark them as in the stack. In the
finilizer I would look if the variable is in the stack
and then mark it as ready for reclaim, and then keep them as referenced and
also clear the data. It would then be good
if we could get hook to the sweep phase so that we can scan the prolog
stack, and remove the variables not reachable but in the stack.
This means that we could benefit from a secondary reclaim feature of memory
else the variables would stay in memory until
the next gc.

The problem is that these stacks needs to mark some fields and not other
fields. This can be fixed by protecting added stuff and finaliziers
but it will slow down the handling. But this pattern can be coded what you
need is a function that tells if a field should be marked or not
we could for example add a pair of a vector and a bitvector where the
bitvector is telling about gc or not similarly to your struct suggestion

Another interesting gc feature is that when you set a variable multiple
times you can clear all data untill the last one after
a backtracking point until the next one, but the backtracking point can be
garbage collected as well. So I need to scan the stack to see if there is
data
that can be reclaimed at each sweep phase.This means it can take multiple
gc's to reclaim all potential memory and I'm not happy
as it is now - When working on this I would probably introduce a doubly
linked list between the backtracking points and maintain them
so essentially when you mark the backtracking point I need to scan the
stack to the next backtracking point and mark all the last reseted
values. Doing this will be a good solution that does not need multiple gc.

Another issue is that a backtracking point can be located in a state tree.
Here again I need to maintain links correctly so that I can scan
the list and mark reseted values just as explained for the stack.

One way around this issue is maintain a library in guile that implements a
general datastructure that can be used as a prolog stack. Or some
other infrastructure that can be used. The pattern is indeed interesting to
try to generalize.

What I do fear though is that the gc of prolog programs with lot's of state
stored will lead to quite a big rucksack of finilizer logic. The state is
stored
in trees and I need to maintain them on the basis of special gc treatment.
A solution again would be to store a bit tree that tells about the marking
of the
elements in the tree so we would have a third data-structure where we would
like to conditionally mark elements. This means that we could use a
finalizer
on the backtracking object.

Finally a question.
If in a finilizer I mark objects. Would I then be able mark what they
reference in turn and so on?

Thinking about this maybe we could use structs to shield of gc but it can
have an awful overhead. I would love to have a bit in SCM
values that tell if they need special gc or no special gc e.g. if this bit
is marked you do your work in the finilizer where you can force
a mark onto the variable although the bit is set.


Regards
Stefan





























On Wed, Nov 4, 2015 at 11:10 AM, Andy Wingo <wingo@pobox.com> wrote:

> Greetings Mark!  And hello to guile-devel :)  Sorry for being
> incommunicado in these last months.  The good news is that we're finally
> ready to release 2.1.1, yay!  I'll be easing myself back into the
> mailing list soon but I wanted to expand on a topic we discussed over
> IRC yesterday: mark procedures.
>
> To recap, since the ages of yore, Guile has allowed SMOB implementations
> to specify "mark procedures".  Indeed in the olden days, these
> procedures were practically required, as the GC wouldn't otherwise trace
> any field of a SMOB.  When we started using the BDW conservative
> collector, mark procedures became less necessary.  At least in master,
> Guile itself doesn't specify any SMOB mark procedures, and the only
> markers set via the lower-level "gc_kind" mechanism from BDW-GC are to
> implement weak tables, and to precisely mark VM stacks.
>
> I would ideally like a world in which mark procedures don't exist.  Mark
> procedures have many really negative things about them: they are (1)
> insufficient for what we want to do with them, (2) extraordinarily
> error-prone, (3) (mostly) unnecessary, and (4) slow.  Let's take them
> one by one.
>
> Firstly, if we consider things we would like to do with our GC, mark
> procedures don't help us get there.  They don't help with a moving GC;
> that prototype would rather be "void (*mark) (SCM* scm_loc)", to allow
> Guile to relocate a pointer, and even that doesn't quite cut it because
> we allow for non-SCM-valued pointers as well.  What you need is a slot
> visitor; something like
> https://chromium.googlesource.com/v8/v8.git/+/master/src/objects-inl.h#1533
> .
> This is the same facility you need to do heap profiling as well -- you
> need both objects, and you get that by working on void** instead of
> void*.
>
> Secondly, mark procedures are an ***amazing*** foot-gun.  They run
> concurrently with the user's program -- on random threads, if the
> collector is doing parallel marking, but even without parallel marking,
> a mark procedure can run from any allocation site -- or indeed any site,
> if the program has multiple threads.  The mutator can hold almost any
> lock at the time the mark procedure is called.  With finalizers we have
> been able to avoid this via `scm_set_automatic_finalization_enabled' and
> `scm_run_finalizers' but with mark procedures this isn't possible.  It's
> also really hard to write a good mark procedure, once you start trying
> anything interesting; and if you're not doing anything interesting, you
> surely don't need a mark procedure, and can just rely on the builtin
> tracer.  Additionally when you mess up somehow with your mark procedure,
> the errors you get will appear far from their cause.
>
> Thirdly, mark procedures are usually unnecessary.  By far the simplest
> way around a mark procedure is to just let the conservative tracer
> handle it, and just make sure your SMOB or whatever object has fields to
> the things you need to trace.  However thinking forward to a
> re-introduction of precise GC so that we can relocate objects, maybe
> this is unsatisfactory; and yet mark procedures are still insufficient
> for this case as I noted.  There are a few options we have today and a
> few more things we could build in the future:
>
>   * If your SMOB A holds on to field B and there is no path from B to A,
>     you can just scm_gc_protect_object on B, and arrange for A's
>     finalizer to scm_gc_unprotect_object on B.  Finalizers are safer
>     than mark procedures (and that's saying something).
>
>   * Or you could add an association from A to B to a weak-key table.
>
>   * If there is a possibility of a path from B to A, you need an
>     ephemeron table, and Guile doesn't do that right now.  But it
>     should!
>
>   * If your object A has a fixed shape, in that it will always hold on
>     to an object B and that object will be in word N of the SMOB, then
>     the SMOB can have an associated descriptor indicating how the object
>     is laid out.  Guile's GC could use this descriptor to know what to
>     mark, whether the values are tagged or not (if we adopted a
>     different tagging system that made this necessary), and so on.  This
>     is already possible with GC_make_descriptor.
>
>   * If your object A does not have a fixed shape and you are using mark
>     procedures to punch through other C/C++ data structures to grab
>     references, it is probable that you have already gotten your mark
>     function wrong due to concurrency.  The C/C++ data structures are
>     not necessarily in a suitable state for access.  This is the case in
>     which mark procedures actually do something for you, but probably it
>     means randomly crashing your program, so there's that.
>
> Finally, mark procedures are slow.  They call out from the hot path in
> GC and they recurse back into the library via scm_gc_mark.  scm_gc_mark
> actually communicates state via thread-local variables, which are slow
> as well.  Their existence can prevent us as hackers from making changes
> to the GC that would improve performance for users that don't use mark
> procedures.  Probably the fastest way to use them is for each object
> kind with mark procedures to be allocated out of its own pool, meaning
> that objects with mark procedures increase fragmentation of the system
> as a whole.
>
> In summary, mark procedures have a lot of points against them,
> especially as currently implemented!  There's only one case that I am
> aware of in which mark procedures actually bring anything to the
> table -- the case in which you punch through complicated data structures
> that the GC can't trace -- but (a) I'm not sure that case wouldn't be
> served just as well via ephemerons (b) I seriously doubt that any such
> mark procedure implementation is actually correct.  Besides the fact
> that the formulation of mark procedures as they are doesn't serve our
> future purposes :P
>
> Not even the JNI makes the mistake of exporting mark functions, nor do
> any of the high-performance JavaScript implementations, nor does Lua,
> etc etc.  At the most we should be making our foreign objects define
> type descriptors and letting Guile itself handle the tracing
> implementation.
>
> At the end of our IRC conversation yesterday you put the burden on me to
> argue against mark procedures, which was fair, but at this point I think
> we would need good arguments for keeping them, at least in the long run
> :)  In the short run as we add new APIs like the foreign object
> facility, I think it makes sense to *avoid* adding mark functions to the
> new APIs, at least for now.  We don't have the use cases right now and
> so we would surely specify the wrong thing.  We can always add something
> later.
>
> Regards,
>
> Andy
>
>

[-- Attachment #2: Type: text/html, Size: 13158 bytes --]

Re: Mark procedures, LilyPond, and backward compatibility

[Mark H Weaver]

Hi Andy,

Thanks for the detailed analysis of the problems with mark procedures.

Andy Wingo <wingo@pobox.com> writes:
> At the end of our IRC conversation yesterday you put the burden on me to
> argue against mark procedures, which was fair, but at this point I think
> we would need good arguments for keeping them,

The most important reason that we need to preserve mark procedures is
that we've supported them for most of Guile's existence, and there's a
lot of important code out there that depends upon them.

Remember LilyPond?  As much as we might like to forget that LilyPond
exists and move onward with more fun things, the fact remains that as
you prepare a 2.1.x pre-release, LilyPond still doesn't work with Guile
2.0.x, and not for lack of effort.  This problem is not just going to go
away.  We _must_ help to get LilyPond working, and we must *not* take more
steps that would make that job more difficult.

We must demonstrate that Guile is a safe base upon which to build a
large application.  LilyPond is by far the most important project that
has done that, and the ongoing LilyPond+Guile2 disaster is a glaring
example that "mistakes were made" in the 1.8 -> 2.0 transition.  I think
we need to acknowledge those mistakes, learn from them, and pay more
attention to backward compatibility now and in the future.

What mistakes were made?  Among many other things: Guile 2.0 switched to
a new macro expander that doesn't support lazy macro expansion, and to a
GC with very different finalization semantics.

LilyPond has a lot of code that depends on the semantics of our old GC.
For starters, it depends heavily upon our long-standing support for mark
procedures, and will surely require ongoing support for a non-threaded
marking mode as well.

While we're on the subject: <https://bugs.gnu.org/19883> demonstrated
that Boehm GC will invoke marking procedures on objects that have
already been finalized.  It seems likely that fixing LilyPond will
require adding support to Boehm GC for another kind of finalizer with
semantics similar to the finalizers in Guile 1.8, where the finalizers
were strictly called before the mutator resumes execution.

> I think it makes sense to *avoid* adding mark functions to the new
> APIs, at least for now.  We don't have the use cases right now and so
> we would surely specify the wrong thing.  We can always add something
> later.

In summary, if we're going to deprecate and replace an existing API, the
new API needs to be designed such that it's straightforward to update
existing code to use the new API.  Therefore, we must continue to
support mark procedures.  I would go further and assert that the new API
should also include what's needed to support a precise, moving GC.  To
do otherwise would be to strongly encourage the creation of code that
cannot possibly work with a moving GC.

Deprecating and replacing the SMOB API is a significant event in the
history of Guile.  It should not be rushed, nor should it be finalized
until it supports existing use cases and anticipated future
developments.

As an aside, I'm not happy with the fact that this new foreign objects
API was pushed to the stable-2.0 branch less than 25 hours after the
initial proposal was posted.  It feels like establishing "facts on the
ground", and sets up a situation where the burden is now on me to argue
that it's not ready to be included in 2.0.12, whereas the burden should
rather be on you to obtain a consensus that this new major API is ready
for release.

For now, I would prefer to revert the commits that added this new API,
to enable the immediate release of 2.0.12.  Then we can have a proper
discussion about what a SMOB replacement API should look like, and make
sure that both past and future anticipated use cases are covered before
releasing it.

What do you think?

      Mark

Foreign object API

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Deprecating and replacing the SMOB API is a significant event in the
> history of Guile.  It should not be rushed, nor should it be finalized
> until it supports existing use cases and anticipated future
> developments.

Agreed.

> As an aside, I'm not happy with the fact that this new foreign objects
> API was pushed to the stable-2.0 branch less than 25 hours after the
> initial proposal was posted.  It feels like establishing "facts on the
> ground", and sets up a situation where the burden is now on me to argue
> that it's not ready to be included in 2.0.12, whereas the burden should
> rather be on you to obtain a consensus that this new major API is ready
> for release.

Agreed too.

But we had already agreed on both long ago.  :-)

> For now, I would prefer to revert the commits that added this new API,
> to enable the immediate release of 2.0.12.  Then we can have a proper
> discussion about what a SMOB replacement API should look like, and make
> sure that both past and future anticipated use cases are covered before
> releasing it.

I sympathize with your concerns.  (I wish we had had this discussion
long ago, when Andy pushed the foreign objects API, or when I called for
us to “do something” about it so we can release 2.0.12.)

Could you summarize your practical concerns with the foreign object API
so we can decide whether reverting is the best thing to do to move
forward?

I expressed mine in
<https://lists.gnu.org/archive/html/guile-devel/2015-05/msg00005.html>
to which Andy replied on IRC the other day that (roughly; correct me if
I’m wrong):

  1. GOOPS is faster in 2.1.

  2. In 2.4 we may be able to avoid loading GOOPS altogether, but not
     before that.

  3. It’s OK if foreign objects introduce some overhead compared to
     SMOBs in 2.0, because the addition of this API in 2.0 is for people
     who want to write forward-compatible code where overhead is less of
     an issue.

  4. Even though some primitives are turned into primitive-generics,
     there are no observable semantic differences when GOOPS is loaded.

  5. Using structs as I suggested above appears to be undoable for
     obscure reasons.

I am not fully convinced, but I prefer to live with that than with a
frozen project.

Besides, it is clear to me that the SMOB API is here to stay, as is, in
the 2.2 series and probably the 2.4 series as well.  I believe Andy
agrees with that.

Don’t escape this thread people, you’re trapped!  :-)

Thanks,
Ludo’.

Foreign objects removed from ‘stable-2.0’

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

Hello!

After discussing it this week-end, Andy and I agreed that it may be best
to revert foreign objects on ‘stable-2.0’ to break the longstanding
deadlock.

So I just did that with commit ff98cbb.  I couldn’t use directly ‘git
revert’ because there had been several commits, so I instead removed the
source and test files and selectively reverted the documentation bits.
I think I didn’t miss anything, but you’re welcome to double-check and
report any issues.

Hopefully this will allow us to release 2.0.12.  We can resume
discussion on what to do with this API anytime, and without the
“pressure” that it’s “already there.”

Thank you gentlefolks!

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Mark procedures, LilyPond, and backward compatibility

[Andy Wingo]

Hi :)

I appreciate what you say.  I do think that we are mixing topics a bit.
Regarding the future applicability of the foreign object API to precise
and moving GC -- this can be added.  I have worked with precise and
moving GC systems.  During the time that I hacked on SpiderMonkey, it
moved from a conservative to a precise moving GC.  It's possible to do,
though I think it is naive to consider doing so without any action
required on the part of a libguile API user.  Specifically I would have
the foreign object type be annotated with a descriptor of its fields.
I do not want to add it now, however; it is too premature IMO.

I hope that clarifies some points.  I am happy to keep the discussion
going :)

I am top-posting though because I think there's a general message in
your mail, that we made wrong decisions in the past and that we should
feel bad about them for some reason in the present.  I strenuously
disagree.  We did the best that we could with the knowledge we had.  I
don't even admit to the mistakes, I don't think -- I mean, if I were
there and did it again, I would most certainly adopt the BDW-GC and
psyntax.  No question.  I would have done different things around
finalizers and mark procedures, but we were ignorant; postulating
retroactive omniscience doesn't help us make better decisions in the
present.

I also disagree very strongly that we are being irresponsible with
respect to backward compatibility.  We're doing what we can, and doing
much more than most other projects.  To me it is especially galling to
hear this, after having been practically the only developer working on
2.2.

I also think you underestimate the abilities to change -- both of Guile
and of our users.  The important thing is that existing interfaces keep
their semantics, and that we help users identify when those interfaces
need to be removed for whatever reason, and what to do.  In the lilypond
case we had trouble maintaining the semantics.  Maybe now that we know
about it we can fix it.  We do what we can.

Dunno where I'm going with this, so I'll just stop here :)  Happy
hacking :)  Let's pick up the foreign object discussion in another
thread.

Andy

On Wed 04 Nov 2015 18:01, Mark H Weaver <mhw@netris.org> writes:

> Andy Wingo <wingo@pobox.com> writes:
>> At the end of our IRC conversation yesterday you put the burden on me to
>> argue against mark procedures, which was fair, but at this point I think
>> we would need good arguments for keeping them,
>
> The most important reason that we need to preserve mark procedures is
> that we've supported them for most of Guile's existence, and there's a
> lot of important code out there that depends upon them.
>
> Remember LilyPond?  As much as we might like to forget that LilyPond
> exists and move onward with more fun things, the fact remains that as
> you prepare a 2.1.x pre-release, LilyPond still doesn't work with Guile
> 2.0.x, and not for lack of effort.  This problem is not just going to go
> away.  We _must_ help to get LilyPond working, and we must *not* take more
> steps that would make that job more difficult.
>
> We must demonstrate that Guile is a safe base upon which to build a
> large application.  LilyPond is by far the most important project that
> has done that, and the ongoing LilyPond+Guile2 disaster is a glaring
> example that "mistakes were made" in the 1.8 -> 2.0 transition.  I think
> we need to acknowledge those mistakes, learn from them, and pay more
> attention to backward compatibility now and in the future.
>
> What mistakes were made?  Among many other things: Guile 2.0 switched to
> a new macro expander that doesn't support lazy macro expansion, and to a
> GC with very different finalization semantics.
>
> LilyPond has a lot of code that depends on the semantics of our old GC.
> For starters, it depends heavily upon our long-standing support for mark
> procedures, and will surely require ongoing support for a non-threaded
> marking mode as well.
>
> While we're on the subject: <https://bugs.gnu.org/19883> demonstrated
> that Boehm GC will invoke marking procedures on objects that have
> already been finalized.  It seems likely that fixing LilyPond will
> require adding support to Boehm GC for another kind of finalizer with
> semantics similar to the finalizers in Guile 1.8, where the finalizers
> were strictly called before the mutator resumes execution.
>
>> I think it makes sense to *avoid* adding mark functions to the new
>> APIs, at least for now.  We don't have the use cases right now and so
>> we would surely specify the wrong thing.  We can always add something
>> later.
>
> In summary, if we're going to deprecate and replace an existing API, the
> new API needs to be designed such that it's straightforward to update
> existing code to use the new API.  Therefore, we must continue to
> support mark procedures.  I would go further and assert that the new API
> should also include what's needed to support a precise, moving GC.  To
> do otherwise would be to strongly encourage the creation of code that
> cannot possibly work with a moving GC.
>
> Deprecating and replacing the SMOB API is a significant event in the
> history of Guile.  It should not be rushed, nor should it be finalized
> until it supports existing use cases and anticipated future
> developments.
>
> As an aside, I'm not happy with the fact that this new foreign objects
> API was pushed to the stable-2.0 branch less than 25 hours after the
> initial proposal was posted.  It feels like establishing "facts on the
> ground", and sets up a situation where the burden is now on me to argue
> that it's not ready to be included in 2.0.12, whereas the burden should
> rather be on you to obtain a consensus that this new major API is ready
> for release.
>
> For now, I would prefer to revert the commits that added this new API,
> to enable the immediate release of 2.0.12.  Then we can have a proper
> discussion about what a SMOB replacement API should look like, and make
> sure that both past and future anticipated use cases are covered before
> releasing it.
>
> What do you think?
>
>       Mark

-- 
http://wingolog.org/

Re: Mark procedures

[Ludovic Courtès]

Hello!

I think we all agree that mark procedures suck in many ways, so that’s
not the problem.

When I ported the old Guile to BDW-GC, I kept them mostly so existing
code that uses SMOB can still work as expected.  Of course, 90% of the
time you could just remove them and things would work, but if you wanted
to support both 1.8 and 2.0, you didn’t *have* to change the code.

The GnuTLS Guile bindings support both 1.8 and 2.0; mark procedures are
used unconditionally, and there haven’t been any issues with that on 2.0.

AIUI LilyPond has more stringent expectations, and that strategy didn’t
work out there because the new GC behavior is observable.

I agree with you that we must keep recommending against using them, and
that remaining uses should probably be questioned; I think we can’t
“just” remove them though.

What we need above all is to address LilyPond’s use case.  I proposed a
solution at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19883#23> but
never understood whether/why it was considered unfit.

Ludo’.

Re: Mark procedures

[Andy Wingo]

On Thu 05 Nov 2015 11:29, ludo@gnu.org (Ludovic Courtès) writes:

> I think we all agree that mark procedures suck in many ways, so that’s
> not the problem.

:)

Some of the points I made have not been made before, AFAIK.  The point
about marking occuring concurrently with the mutator even in the absence
of threads, for example.

> I agree with you that we must keep recommending against using [mark
> procedures], and that remaining uses should probably be questioned; I
> think we can’t “just” remove them though.

I am not sure, to be honest.  I am not proposing "just" removing them.
However if we can remove them, then we *certainly* should do so -- we
should plan our API in that way.

> What we need above all is to address LilyPond’s use case.  I proposed a
> solution at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19883#23> but
> never understood whether/why it was considered unfit.

I agree with you that the patch there looks reasonable to me too, though
AFAIU the original code should work just fine too.

There area few things at play.

 (1) A bug related to SMOB finalization and marking that affects
     LilyPond

 (2) The utility of mark procedures in general

 (3) The suitability of mark procedures for future uses

 (4) Whether we can get by without mark procedures, and if so, how.

For (1) it seems to me that we just have a bug.  A SMOB mark function
was called on an object after the finalizer.  ****Note**** that having
the finalizer called doesn't mean that the GC object was collected -- it
just means it was collectable, perhaps in a clique of objects.
Finalization being asynchronous with marking it's possible that a clique
of objects was only half-finalized when a new mark procedure runs.  The
mark procedure saw an object on which free() was already called -- this
is possible.

We should fix Guile so to "null out" the SMOB typecode when the SMOB
finalizer is called.  If our mark procedure sees a SMOB that has already
been finalized, it just returns.

Though finalizers in general can resuscitate objects, that was never a
contract of SMOB finalizers, and I think in fact it's an anti-contract.
Unfortunately for maximum robustness we probably need to grab the alloc
lock when swapping out the typecode; too bad.  This is not the same as
marking dead objects on free lists.  An object is first finalized, then
BDW-GC removes its finalizer, then if it is collected in the next GC it
goes on a freelist.

For (2).  To me the existence of other systems with proper GC but no
mark procedures, especially JS VMs, means that mark procedures are not
_necessary_.

For (3) I hope I have successfully argued that what we need for precise
and moving collection is not the same as mark procedures.  I think Mark
would like to deal with these topics at the same time but I strongly
disagree.

Point (4) indicates to me that if we are making new abstractions that we
would like to encourage people to use in the future, we should not
encourage mark functions.  We can add some other mechanism (type
descriptors, for example).  SMOBs and their horrible mark procedures
aren't going away any time soon, of course.

Andy
-- 
http://wingolog.org/

Re: Mark procedures

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> Some of the points I made have not been made before, AFAIK.  The point
> about marking occuring concurrently with the mutator even in the absence
> of threads, for example.

Yes, right.

>> I agree with you that we must keep recommending against using [mark
>> procedures], and that remaining uses should probably be questioned; I
>> think we can’t “just” remove them though.
>
> I am not sure, to be honest.  I am not proposing "just" removing them.
> However if we can remove them, then we *certainly* should do so -- we
> should plan our API in that way.

Right.  I think we cannot remove them yet, but we should do what it
takes so users rely less and less on them.

>> What we need above all is to address LilyPond’s use case.  I proposed a
>> solution at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19883#23> but
>> never understood whether/why it was considered unfit.
>
> I agree with you that the patch there looks reasonable to me too, though
> AFAIU the original code should work just fine too.
>
> There area few things at play.
>
>  (1) A bug related to SMOB finalization and marking that affects
>      LilyPond
>
>  (2) The utility of mark procedures in general
>
>  (3) The suitability of mark procedures for future uses
>
>  (4) Whether we can get by without mark procedures, and if so, how.
>
> For (1) it seems to me that we just have a bug.  A SMOB mark function
> was called on an object after the finalizer.  ****Note**** that having
> the finalizer called doesn't mean that the GC object was collected -- it
> just means it was collectable, perhaps in a clique of objects.
> Finalization being asynchronous with marking it's possible that a clique
> of objects was only half-finalized when a new mark procedure runs.  The
> mark procedure saw an object on which free() was already called -- this
> is possible.
>
> We should fix Guile so to "null out" the SMOB typecode when the SMOB
> finalizer is called.  If our mark procedure sees a SMOB that has already
> been finalized, it just returns.
>
> Though finalizers in general can resuscitate objects, that was never a
> contract of SMOB finalizers, and I think in fact it's an anti-contract.
> Unfortunately for maximum robustness we probably need to grab the alloc
> lock when swapping out the typecode; too bad.  This is not the same as
> marking dead objects on free lists.  An object is first finalized, then
> BDW-GC removes its finalizer, then if it is collected in the next GC it
> goes on a freelist.

Since (1) is difficult to fix, and since mark procedures have other
problems as you explain, I was inclined to address the Lily problem via (4).

> For (2).  To me the existence of other systems with proper GC but no
> mark procedures, especially JS VMs, means that mark procedures are not
> _necessary_.

Point taken.  BDW-GC makes them useless in most cases anyway.

> For (3) I hope I have successfully argued that what we need for precise
> and moving collection is not the same as mark procedures.  I think Mark
> would like to deal with these topics at the same time but I strongly
> disagree.

I think so.  I admit I’m more concerned with the immediate problems of
LilyPond though.  ;-)

> Point (4) indicates to me that if we are making new abstractions that we
> would like to encourage people to use in the future, we should not
> encourage mark functions.  We can add some other mechanism (type
> descriptors, for example).  SMOBs and their horrible mark procedures
> aren't going away any time soon, of course.

Agreed!  Commit d9a4a1c uses very clear wording to recommend against
mark procedures, which is the right thing IMO; commit 56273de from 2009
already hinted in that direction, though not as prominently.

Since I missed the original discussion between you and Mark, I might
well be completely off, in which case I apologize.

Thanks,
Ludo’.

Re: Mark procedures and LilyPond

[Mark H Weaver]

Andy Wingo <wingo@pobox.com> writes:

> On Thu 05 Nov 2015 11:29, ludo@gnu.org (Ludovic Courtès) writes:
>
>> What we need above all is to address LilyPond’s use case.  I proposed a
>> solution at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19883#23> but
>> never understood whether/why it was considered unfit.
>
> I agree with you that the patch there looks reasonable to me too, though
> AFAIU the original code should work just fine too.
>
> There area few things at play.
>
>  (1) A bug related to SMOB finalization and marking that affects
>      LilyPond
>
>  (2) The utility of mark procedures in general
>
>  (3) The suitability of mark procedures for future uses
>
>  (4) Whether we can get by without mark procedures, and if so, how.
>
> For (1) it seems to me that we just have a bug.  A SMOB mark function
> was called on an object after the finalizer.  ****Note**** that having
> the finalizer called doesn't mean that the GC object was collected -- it
> just means it was collectable, perhaps in a clique of objects.
> Finalization being asynchronous with marking it's possible that a clique
> of objects was only half-finalized when a new mark procedure runs.  The
> mark procedure saw an object on which free() was already called -- this
> is possible.

Yes, exactly.

> We should fix Guile so to "null out" the SMOB typecode when the SMOB
> finalizer is called.  If our mark procedure sees a SMOB that has already
> been finalized, it just returns.

Unfortunately, I doubt this will be sufficient for LilyPond.  The small
example case in <http://bugs.gnu.org/19883>, which is apparently
representative of how things are typically done in LilyPond, has
structures like this:

                   __________                    __________
  Objects in      |          |                  |          |
  GC-managed      |  SMOB 1  |                  |  SMOB 2  |
     heap         |__________|                  |__________|
                     |   ^                         |   ^
.....................|...|.........................|...|..........
                   __v___|___      _________     __v___|___
  Objects in      |          |    |   STL   |   |          |
  normal heap     |C++ object|--->|container|-->|C++ object|
  (not scanned    |__________|    |_________|   |__________|
     by GC)


The SMOB finalizers free the associated C++ objects below them.  Now,
suppose that none of the objects above are reachable, so both SMOBs are
queued for finalization.  Now suppose that SMOB 2 is finalized first,
thus freeing the C++ object below it.  Suppose further that we null out
the SMOB 2 typecode.

Now another GC occurs and the marker is called on SMOB 1.  It's typecode
has not yet been nulled, so the user-specified mark procedure is called.
The mark procedure for SMOB 1 iterates over the STL container, and for
each C++ object within, marks the corresponding SMOB via the upward
pointer, in this case SMOB 2.  Although SMOB 2's typecode has been
zeroed out, the discovery of the fact is too late, because the (freed)
C++ object below it has been referenced.

As far as I can tell, this way of doing things depends on the old 1.8 GC
finalization semantics, where all of the finalizers are called before
the mutator resumes execution.

      Mark

Re: Mark procedures and LilyPond

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Unfortunately, I doubt this will be sufficient for LilyPond.  The small
> example case in <http://bugs.gnu.org/19883>, which is apparently
> representative of how things are typically done in LilyPond, has
> structures like this:
>
>                    __________                    __________
>   Objects in      |          |                  |          |
>   GC-managed      |  SMOB 1  |                  |  SMOB 2  |
>      heap         |__________|                  |__________|
>                      |   ^                         |   ^
> .....................|...|.........................|...|..........
>                    __v___|___      _________     __v___|___
>   Objects in      |          |    |   STL   |   |          |
>   normal heap     |C++ object|--->|container|-->|C++ object|
>   (not scanned    |__________|    |_________|   |__________|
>      by GC)

Thanks for the picture, that’s very helpful!

I still think we should be able to get rid of GC mark procedures in this
case, sidestepping the bug you describe.

This pattern is very common when writing bindings.  I can’t imagine that
there's something insurmountable here.  Am I missing something?

Ludo’.

Re: Mark procedures and LilyPond

[Stefan Monnier]

>                    __________                    __________
>   Objects in      |          |                  |          |
>   GC-managed      |  SMOB 1  |                  |  SMOB 2  |
>      heap         |__________|                  |__________|
>                      |   ^                         |   ^
> .....................|...|.........................|...|..........
>                    __v___|___      _________     __v___|___
>   Objects in      |          |    |   STL   |   |          |
>   normal heap     |C++ object|--->|container|-->|C++ object|
>   (not scanned    |__________|    |_________|   |__________|
>      by GC)


> The SMOB finalizers free the associated C++ objects below them.  Now,
> suppose that none of the objects above are reachable, so both SMOBs are
> queued for finalization.  Now suppose that SMOB 2 is finalized first,
> thus freeing the C++ object below it.

It's clearly wrong for SMOB2's finalizer to free its C++ object here
since that object is still reachable from C++ objects.


        Stefan

Re: Mark procedures and LilyPond

[Hans Åberg]

> On 6 Nov 2015, at 13:32, Mark H Weaver <mhw@netris.org> wrote:

>> We should fix Guile so to "null out" the SMOB typecode when the SMOB
>> finalizer is called.  If our mark procedure sees a SMOB that has already
>> been finalized, it just returns.
> 
> Unfortunately, I doubt this will be sufficient for LilyPond.  The small
> example case in <http://bugs.gnu.org/19883>, which is apparently
> representative of how things are typically done in LilyPond, has
> structures like this:
> 
>                   __________                    __________
>  Objects in      |          |                  |          |
>  GC-managed      |  SMOB 1  |                  |  SMOB 2  |
>     heap         |__________|                  |__________|
>                     |   ^                         |   ^
> .....................|...|.........................|...|..........
>                   __v___|___      _________     __v___|___
>  Objects in      |          |    |   STL   |   |          |
>  normal heap     |C++ object|--->|container|-->|C++ object|
>  (not scanned    |__________|    |_________|   |__________|
>     by GC)
> 
> 
> The SMOB finalizers free the associated C++ objects below them.  Now,
> suppose that none of the objects above are reachable, so both SMOBs are
> queued for finalization.  Now suppose that SMOB 2 is finalized first,
> thus freeing the C++ object below it.  Suppose further that we null out
> the SMOB 2 typecode.
> 
> Now another GC occurs and the marker is called on SMOB 1.  It's typecode
> has not yet been nulled, so the user-specified mark procedure is called.
> The mark procedure for SMOB 1 iterates over the STL container, and for
> each C++ object within, marks the corresponding SMOB via the upward
> pointer, in this case SMOB 2.  Although SMOB 2's typecode has been
> zeroed out, the discovery of the fact is too late, because the (freed)
> C++ object below it has been referenced.
> 
> As far as I can tell, this way of doing things depends on the old 1.8 GC
> finalization semantics, where all of the finalizers are called before
> the mutator resumes execution.

Is Guile 2 doing its own finalization, not synced withe the Boehm GC?

Re: Mark procedures

[Andy Wingo]

Picking up an old thread....

On Thu 05 Nov 2015 14:11, Andy Wingo <wingo@pobox.com> writes:

>  (1) A bug related to SMOB finalization and marking that affects
>      LilyPond
>
> For (1) it seems to me that we just have a bug.  A SMOB mark function
> was called on an object after the finalizer.  ****Note**** that having
> the finalizer called doesn't mean that the GC object was collected -- it
> just means it was collectable, perhaps in a clique of objects.
> Finalization being asynchronous with marking it's possible that a clique
> of objects was only half-finalized when a new mark procedure runs.  The
> mark procedure saw an object on which free() was already called -- this
> is possible.
>
> We should fix Guile so to "null out" the SMOB typecode when the SMOB
> finalizer is called.  If our mark procedure sees a SMOB that has already
> been finalized, it just returns.

AFAIU the Lilypond / Guile 2.0 bug is still there.  Has anyone tried
this strategy to fix it?

Andy

Re: Mark procedures

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> Picking up an old thread....
>
> On Thu 05 Nov 2015 14:11, Andy Wingo <wingo@pobox.com> writes:
>
>>  (1) A bug related to SMOB finalization and marking that affects
>>      LilyPond
>>
>> For (1) it seems to me that we just have a bug.  A SMOB mark function
>> was called on an object after the finalizer.  ****Note**** that having
>> the finalizer called doesn't mean that the GC object was collected -- it
>> just means it was collectable, perhaps in a clique of objects.
>> Finalization being asynchronous with marking it's possible that a clique
>> of objects was only half-finalized when a new mark procedure runs.  The
>> mark procedure saw an object on which free() was already called -- this
>> is possible.
>>
>> We should fix Guile so to "null out" the SMOB typecode when the SMOB
>> finalizer is called.  If our mark procedure sees a SMOB that has already
>> been finalized, it just returns.
>
> AFAIU the Lilypond / Guile 2.0 bug is still there.  Has anyone tried
> this strategy to fix it?

I haven’t.  :-/

Ludo’.

Ephemerons, self-referentality in weak hashtables

[Christopher Lemmer Webber]

Hello,

I'm finally taking some time to port Goblins to Guile, in-between other
tasks anyway.  In Goblins there is a weak hashtable that maps current
actor references to their current behavior.  I found that for
self-referential actors, I needed ephemerons for GC stuff to work right.

In this old thread I found Wingo mentioning them:

Andy Wingo writes:

>   * If there is a possibility of a path from B to A, you need an
>     ephemeron table, and Guile doesn't do that right now.  But it
>     should!

Is there something ephemeron-like I should be doing?

Re: Ephemerons, self-referentality in weak hashtables

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

Christopher Lemmer Webber schreef op di 18-05-2021 om 11:46 [-0400]:
> Hello,
> 
> I'm finally taking some time to port Goblins to Guile, in-between other
> tasks anyway.  In Goblins there is a weak hashtable that maps current
> actor references to their current behavior.  I found that for
> self-referential actors, I needed ephemerons for GC stuff to work right.

Ephemeron SRFI: <https://srfi.schemers.org/srfi-124/srfi-124.html>

> In this old thread I found Wingo mentioning them:
> 
> Andy Wingo writes:
> 
> >   * If there is a possibility of a path from B to A, you need an
> >     ephemeron table, and Guile doesn't do that right now.  But it
> >     should!
> 
> Is there something ephemeron-like I should be doing?

Guile doesn't seem to have ephemerons, though it would be nice to
have them!

I've been looking at gc.h for how these could be implemented
(Guile uses BDW-GC for garbage collection).
The function GC_general_register_disappearing_link (void **link, const void *obj)
(https://github.com/ivmai/bdwgc/blob/master/include/gc.h#L1218) looks promising.
The idea is to have a C structure representing ephemerons

struct ephemeron {
  SCM key;
  SCM value; /* called ‘datum’in SRFI-124 */
}

do some magic to make ‘value’ and ‘keys’ ‘disguised pointers’ (i.e., tell BDW-GC that
‘key’ and ‘value’ doesn't really point to anything), and during construction
register ‘disappearing links’, such that ‘key’ and ‘value’ will be cleared
when ‘key’ becomes unreachable.

static void
initialise_ephemeron (struct ephemeron *eph, SCM key, SCM value)
{
  eph->key = key:
  eph->value = value;
  /* TODO: this can return an error */
  GC_general_register_disappearing_link (&eph->value, key);
  GC_general_register_disappearing_link (&eph->key, key);
}

Note that, according to the GC_general_register_disappearing_link docs,
reading eph->key and eph->value required holding th allocation lock,
so:

SCM
scm_ephemeron_key (SCM eph_scm)
{
  struct ephemeron *e = [type checks ...]
  SCM ret;
  [hold the lock]
  ret = e->key;
  [release the lock]
  return ret;
}

Likewise for ‘value’. Or maybe GC_general_register_disappearing_link
doesn't work that way ... requires testing!

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Ephemerons, self-referentality in weak hashtables

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Maxime Devos schreef op zo 20-06-2021 om 17:01 [+0200]:
> Christopher Lemmer Webber schreef op di 18-05-2021 om 11:46 [-0400]:
> > Hello,
> > 
> > I'm finally taking some time to port Goblins to Guile, in-between other
> > tasks anyway.  In Goblins there is a weak hashtable that maps current
> > actor references to their current behavior.  I found that for
> > self-referential actors, I needed ephemerons for GC stuff to work right.
> 
> [bla bla on how this could be implemented in Guile]

This doesn't work because there is nothing preventing the
'value' from being freed. Trying to fix that now, using
the example implementation of "weak maps" in libgc.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Ephemerons, self-referentality in weak hashtables

[Christine Lemmer-Webber]

Maxime Devos <maximedevos@telenet.be> writes:

> [[PGP Signed Part:Undecided]]
> Maxime Devos schreef op zo 20-06-2021 om 17:01 [+0200]:
>> Christopher Lemmer Webber schreef op di 18-05-2021 om 11:46 [-0400]:
>> > Hello,
>> > 
>> > I'm finally taking some time to port Goblins to Guile, in-between other
>> > tasks anyway.  In Goblins there is a weak hashtable that maps current
>> > actor references to their current behavior.  I found that for
>> > self-referential actors, I needed ephemerons for GC stuff to work right.
>> 
>> [bla bla on how this could be implemented in Guile]
>
> This doesn't work because there is nothing preventing the
> 'value' from being freed. Trying to fix that now, using
> the example implementation of "weak maps" in libgc.
>
> Greetings,
> Maxime.

I fell off the radar on replying to this, but did path turn out to work?

Thank you for researching and developing a solution to this!

Re: Ephemerons, self-referentality in weak hashtables

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1712 bytes --]

Christine Lemmer-Webber schreef op wo 08-09-2021 om 12:18 [-0400]:
> Maxime Devos <maximedevos@telenet.be> writes:
> 
> > [[PGP Signed Part:Undecided]]
> > Maxime Devos schreef op zo 20-06-2021 om 17:01 [+0200]:
> > > Christopher Lemmer Webber schreef op di 18-05-2021 om 11:46 [-0400]:
> > > > Hello,
> > > > 
> > > > I'm finally taking some time to port Goblins to Guile, in-between other
> > > > tasks anyway.  In Goblins there is a weak hashtable that maps current
> > > > actor references to their current behavior.  I found that for
> > > > self-referential actors, I needed ephemerons for GC stuff to work right.
> > > 
> > > [bla bla on how this could be implemented in Guile]
> > 
> > This doesn't work because there is nothing preventing the
> > 'value' from being freed. Trying to fix that now, using
> > the example implementation of "weak maps" in libgc.
> > 
> > Greetings,
> > Maxime.
> 
> I fell off the radar on replying to this, but did path turn out to work?

The example implementation was a bit complicated and not well-documented.
The ‘disclaimer’ support is disabled by default, a configuration flag
needs to be set while compiling libgc to use disclaimers.  Guix (the distro I
use) doesn't set it.  I needed to be careful in libguile/weak-table.c to not
protect too much (otherwise it wouldn't be an ephemeral weak hash table) or
too little (otherwise ‘freed’ objects would be re-used).

I think it can be made to work, but I didn't succeed, and moved on to other
things.  If someone would like to implement this, I would recommend starting
with something like ‘ephemeral pairs’ before moving to ephemeral hash tables.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Ephemerons, self-referentality in weak hashtables

[Christine Lemmer-Webber]

Maxime Devos <maximedevos@telenet.be> writes:

> [[PGP Signed Part:Undecided]]
> Christine Lemmer-Webber schreef op wo 08-09-2021 om 12:18 [-0400]:
>> Maxime Devos <maximedevos@telenet.be> writes:
>> 
>> > [[PGP Signed Part:Undecided]]
>> > Maxime Devos schreef op zo 20-06-2021 om 17:01 [+0200]:
>> > > Christopher Lemmer Webber schreef op di 18-05-2021 om 11:46 [-0400]:
>> > > > Hello,
>> > > > 
>> > > > I'm finally taking some time to port Goblins to Guile, in-between other
>> > > > tasks anyway.  In Goblins there is a weak hashtable that maps current
>> > > > actor references to their current behavior.  I found that for
>> > > > self-referential actors, I needed ephemerons for GC stuff to work right.
>> > > 
>> > > [bla bla on how this could be implemented in Guile]
>> > 
>> > This doesn't work because there is nothing preventing the
>> > 'value' from being freed. Trying to fix that now, using
>> > the example implementation of "weak maps" in libgc.
>> > 
>> > Greetings,
>> > Maxime.
>> 
>> I fell off the radar on replying to this, but did path turn out to work?
>
> The example implementation was a bit complicated and not well-documented.
> The ‘disclaimer’ support is disabled by default, a configuration flag
> needs to be set while compiling libgc to use disclaimers.  Guix (the distro I
> use) doesn't set it.  I needed to be careful in libguile/weak-table.c to not
> protect too much (otherwise it wouldn't be an ephemeral weak hash table) or
> too little (otherwise ‘freed’ objects would be re-used).
>
> I think it can be made to work, but I didn't succeed, and moved on to other
> things.  If someone would like to implement this, I would recommend starting
> with something like ‘ephemeral pairs’ before moving to ephemeral hash tables.
>
> Greetings,
> Maxime.

Okay... thank you for the work you put into it! :)