smob mark functions in 2.0

smob mark functions in 2.0

[Andy Wingo]

Hello all,

I don't know how you feel, but for me the BDW garbage collector used in
Guile 2.0 has made my life much easier than it was.  You typically don't
have to write SMOB marking functions any more.

However, I just fixed a bug in g-wrap that was a bit surprising to me.
If you do implement a SMOB marking function, and you touch Scheme
objects in that marking function, you need to be very careful.

Specifically, there is a warning in gc/gc_mark.h:

    /* WARNING: Such a mark procedure may be invoked on an unused object    */
    /* residing on a free list.  Such objects are cleared, except for a     */
    /* free list link field in the first word.  Thus mark procedures may    */
    /* not count on the presence of a type descriptor, and must handle this */
    /* case correctly somehow.                                              */

So, your mark function might see freed objects.  This is terrible, but
it is the way that it is.  The key is that, if you touch a Scheme object
in your mark function, to first do a check on that object, to see that
it is valid.  You can check the TC bits of the first word, or otherwise
check that other words are non-NULL.

Caveat mercator!

Andy
-- 
http://wingolog.org/

Re: smob mark functions in 2.0

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@pobox.com> skribis:

> Specifically, there is a warning in gc/gc_mark.h:
>
>     /* WARNING: Such a mark procedure may be invoked on an unused object    */
>     /* residing on a free list.  Such objects are cleared, except for a     */
>     /* free list link field in the first word.  Thus mark procedures may    */
>     /* not count on the presence of a type descriptor, and must handle this */
>     /* case correctly somehow.                                              */

Arrgh!

> So, your mark function might see freed objects.  This is terrible, but
> it is the way that it is.  The key is that, if you touch a Scheme object
> in your mark function, to first do a check on that object, to see that
> it is valid.  You can check the TC bits of the first word, or otherwise
> check that other words are non-NULL.

What about making that check in libguile before invoking the user’s mark
function?

Thanks,
Ludo’.

Re: smob mark functions in 2.0

[Andy Wingo]

On Thu 24 Nov 2011 00:12, ludo@gnu.org (Ludovic Courtès) writes:

>> So, your mark function might see freed objects.  This is terrible, but
>> it is the way that it is.  The key is that, if you touch a Scheme object
>> in your mark function, to first do a check on that object, to see that
>> it is valid.  You can check the TC bits of the first word, or otherwise
>> check that other words are non-NULL.
>
> What about making that check in libguile before invoking the user’s mark
> function?

Yes, we do that.  I think you wrote that code!  The problem was in a
mark function, accessing *other* Scheme objects.

Andy
-- 
http://wingolog.org/

Re: smob mark functions in 2.0

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> skribis:

> On Thu 24 Nov 2011 00:12, ludo@gnu.org (Ludovic Courtès) writes:
>
>>> So, your mark function might see freed objects.  This is terrible, but
>>> it is the way that it is.  The key is that, if you touch a Scheme object
>>> in your mark function, to first do a check on that object, to see that
>>> it is valid.  You can check the TC bits of the first word, or otherwise
>>> check that other words are non-NULL.
>>
>> What about making that check in libguile before invoking the user’s mark
>> function?
>
> Yes, we do that.  I think you wrote that code!

Yes, when I was young.  ;-)

> The problem was in a mark function, accessing *other* Scheme objects.

Oh, right.

Anyway, users are encouraged to #ifdef out mark functions when using
Guile 2.0.

Thanks,
Ludo’.

Re: smob mark functions in 2.0

[Mark H Weaver]

Andy Wingo <wingo@pobox.com> writes:
> Specifically, there is a warning in gc/gc_mark.h:
>
>     /* WARNING: Such a mark procedure may be invoked on an unused object    */
>     /* residing on a free list.  Such objects are cleared, except for a     */
>     /* free list link field in the first word.  Thus mark procedures may    */
>     /* not count on the presence of a type descriptor, and must handle this */
>     /* case correctly somehow.                                              */
>
> So, your mark function might see freed objects.

How can this happen?  If you are marking an object, then presumably it
is still reachable, and therefore the objects it references are also
still reachable.  If any of those reachable objects has been freed,
isn't that already a bug of a different kind?  What am I missing?

   Thanks,
     Mark

Re: smob mark functions in 2.0

[Mark H Weaver]

Hi Andy,

Reviving a 3-year-old thread...

Andy Wingo <wingo@pobox.com> wrote in November 2011:

> If you do implement a SMOB marking function, and you touch Scheme
> objects in that marking function, you need to be very careful.
> 
> Specifically, there is a warning in gc/gc_mark.h:
> 
>     /* WARNING: Such a mark procedure may be invoked on an unused object    */
>     /* residing on a free list.  Such objects are cleared, except for a     */
>     /* free list link field in the first word.  Thus mark procedures may    */
>     /* not count on the presence of a type descriptor, and must handle this */
>     /* case correctly somehow.                                              */
> 
> So, your mark function might see freed objects.  This is terrible, but
> it is the way that it is.  The key is that, if you touch a Scheme object
> in your mark function, to first do a check on that object, to see that
> it is valid.  You can check the TC bits of the first word, or otherwise
> check that other words are non-NULL.

Andy Wingo <wingo@pobox.com> later replied to Ludovic:

> On Thu 24 Nov 2011 00:12, ludo@gnu.org (Ludovic Courtès) writes:
>
>>> So, your mark function might see freed objects.  This is terrible, but
>>> it is the way that it is.  The key is that, if you touch a Scheme object
>>> in your mark function, to first do a check on that object, to see that
>>> it is valid.  You can check the TC bits of the first word, or otherwise
>>> check that other words are non-NULL.
>>
>> What about making that check in libguile before invoking the user’s mark
>> function?
>
> Yes, we do that.  I think you wrote that code!  The problem was in a
> mark function, accessing *other* Scheme objects.

Mark H Weaver <mhw@netris.org> writes:

> Andy Wingo <wingo@pobox.com> writes:
>> Specifically, there is a warning in gc/gc_mark.h:
>>
>>     /* WARNING: Such a mark procedure may be invoked on an unused object    */
>>     /* residing on a free list.  Such objects are cleared, except for a     */
>>     /* free list link field in the first word.  Thus mark procedures may    */
>>     /* not count on the presence of a type descriptor, and must handle this */
>>     /* case correctly somehow.                                              */
>>
>> So, your mark function might see freed objects.
>
> How can this happen?  If you are marking an object, then presumably it
> is still reachable, and therefore the objects it references are also
> still reachable.  If any of those reachable objects has been freed,
> isn't that already a bug of a different kind?  What am I missing?

I never received an answer to this question.  At the time it was merely
a curiosity, but now I have a more pressing need to understand what's
going on here.

    Regards,
      Mark

Re: smob mark functions in 2.0

[Andy Wingo]

On Fri 26 Sep 2014 06:50, Mark H Weaver <mhw@netris.org> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Andy Wingo <wingo@pobox.com> writes:
>>> Specifically, there is a warning in gc/gc_mark.h:
>>>
>>>     /* WARNING: Such a mark procedure may be invoked on an unused object    */
>>>     /* residing on a free list.  Such objects are cleared, except for a     */
>>>     /* free list link field in the first word.  Thus mark procedures may    */
>>>     /* not count on the presence of a type descriptor, and must handle this */
>>>     /* case correctly somehow.                                              */
>>>
>>> So, your mark function might see freed objects.
>>
>> How can this happen?

This can happen if the object is on a free list.  Free lists are just
normal heap-managed data structures -- traced by the GC, counting as
live.  See e.g. gc-inline.h in master; a variation of this exact code is
used internally in libgc by the allocation API.

Andy
-- 
http://wingolog.org/