Making every goops object applicable

Making every goops object applicable

[Krister Svanlund]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

In our work to look into how Python 3 could be implemented for Guile we
have figured out that the only way to make a goops object applicable is to
have it inherit <applicable-struct>. This does not always work the way it
could be expected, for example when inheriting from several classes.
Apparently this works by some flag being set by <applicable-strukt> in
libguile for the object and that flag is checked during application,
calling the 'procedure slot if it's set with some optimization assuming
that 'procedure is the first slot.
Is this correct and if so, is there any particular reasoning behind this
rather than just having all classes that has the slot 'procedure be
applicable?

Yours,
Krister Svanlund

[-- Attachment #2: Type: text/html, Size: 762 bytes --]

Re: Making every goops object applicable

[Mark H Weaver]

Krister Svanlund <krister.svanlund@gmail.com> writes:
> In our work to look into how Python 3 could be implemented for Guile
> we have figured out that the only way to make a goops object
> applicable is to have it inherit <applicable-struct>. This does not
> always work the way it could be expected, for example when inheriting
> from several classes.

Can you elaborate on what doesn't work as expected?

> Apparently this works by some flag being set by
> <applicable-strukt> in libguile for the object and that flag is
> checked during application, calling the 'procedure slot if it's set
> with some optimization assuming that 'procedure is the first slot.
> Is this correct and if so, is there any particular reasoning behind
> this rather than just having all classes that has the slot 'procedure
> be applicable?

I see at least two problems with your suggestion: (1) it would be very
slow to search the list of slots for a slot named 'procedure' on every
application.  (2) a user might create a class with a slot named
'procedure' without intending that the slot should be used in this way.

     Mark

Fwd: Making every goops object applicable

[Krister Svanlund]

[-- Attachment #1: Type: text/plain, Size: 2909 bytes --]

---------- Forwarded message ----------
From: Krister Svanlund <krister.svanlund@gmail.com>
Date: Tue, May 15, 2012 at 2:45 AM
Subject: Re: Making every goops object applicable
To: Mark H Weaver <mhw@netris.org>


On Tue, May 15, 2012 at 12:16 AM, Mark H Weaver <mhw@netris.org> wrote:

> Krister Svanlund <krister.svanlund@gmail.com> writes:
> > In our work to look into how Python 3 could be implemented for Guile
> > we have figured out that the only way to make a goops object
> > applicable is to have it inherit <applicable-struct>. This does not
> > always work the way it could be expected, for example when inheriting
> > from several classes.
>
> Can you elaborate on what doesn't work as expected?
>

For example an instance of a class inheriting a class that inherits
<applicable-struct> that defines 'procedure is not applicable.


>
> > Apparently this works by some flag being set by
> > <applicable-strukt> in libguile for the object and that flag is
> > checked during application, calling the 'procedure slot if it's set
> > with some optimization assuming that 'procedure is the first slot.
> > Is this correct and if so, is there any particular reasoning behind
> > this rather than just having all classes that has the slot 'procedure
> > be applicable?
>
> I see at least two problems with your suggestion: (1) it would be very
> slow to search the list of slots for a slot named 'procedure' on every
> application.  (2) a user might create a class with a slot named
> 'procedure' without intending that the slot should be used in this way.
>

(1) Given that slots aren't dynamic at runtime (as they are in Python for
example) and there is already some magic being done by <applicable-struct>
I have a hard time seeing how looking up a slot for application can't be
done in almost constant time (one look-up and then storing a pointer to the
position of the slot for future applications) and only a tiny bit slower
than the current implementation. And after all, Guile is, judging by it's
documentation, focused on being an extension language where the heavy
lifting is supposed to be done in C code.
(2) This I'm not entirely sure how to fix but I have the impression that
this could be done analogous to the initialization method of the
meta-object protocol, but I assume this could require some severe internal
changes and in fact slow down applications in general in Guile. My thought
is that you simply have a metaclass that has a method ('apply or similar)
that specifies what will happen if an instance of any of it's classes is
being used as a function.

I'm mostly interested in getting all possible input about how good/bad of
an idea this is so please prove (in the looses sense of the word) that I'm
wrong. When I have some free time this summer I will probably make an
attempt at doing this anyhow, I just want to know what I'm getting myself
into.

Yours,
Krister Svanlund

[-- Attachment #2: Type: text/html, Size: 3942 bytes --]

Re: Making every goops object applicable

[Ludovic Courtès]

Hi,

Krister Svanlund <krister.svanlund@gmail.com> skribis:

> Apparently this works by some flag being set by <applicable-strukt> in
> libguile for the object and that flag is checked during application,
> calling the 'procedure slot if it's set with some optimization assuming
> that 'procedure is the first slot.

There’s also a vtable flag that determines whether a struct is
applicable:

  #define SCM_STRUCT_APPLICABLE_P(X) 	(SCM_STRUCT_VTABLE_FLAG_IS_SET ((X), SCM_VTABLE_FLAG_APPLICABLE))

And indeed, the struct’s procedure is the first slot:

  #define scm_applicable_struct_index_procedure 0 /* The procedure of an applicable
                                                     struct. Only valid if the
                                                     struct's vtable has the
                                                     applicable flag set. */

For instance, every struct whose vtable is <applicable-struct-vtable>
(defined in struct.c) is applicable.

Then you can investigate by looking at the indices defined in struct.h:

  scheme@(guile-user)> (struct-vtable? <class>)
  $2 = #t
  scheme@(guile-user)> (struct-ref <class> 1) ; scm_vtable_index_flags
  $3 = 12291
  scheme@(guile-user)> (logand $3 4)          ; SCM_VTABLE_FLAG_APPLICABLE_VTABLE
  $4 = 0                                      ; → not applicable

  scheme@(guile-user)> (logand (struct-ref <applicable-struct-vtable> 1) 4)
  $7 = 4                                      ; → applicable

So you could fiddle with the flags of a class to make its instances
applicable:

  scheme@(guile-user)> (define (applicable-struct? s)
                         (logand 4 (struct-ref (struct-vtable (struct-vtable s)) 1)))
  scheme@(guile-user)> (applicable-struct? current-input-port)
  $27 = 4
  scheme@(guile-user)> (define (applicable-struct-procedure s) (struct-ref s 0))

  scheme@(guile-user)> (define-class <appclass> (<class>) (foo))
  scheme@(guile-user)> (struct-set! <appclass> 1 (logior (struct-ref <appclass> 1) 4))
  $32 = 12295
  scheme@(guile-user)> (define-class <foo> ()
                         (bar #:init-value (lambda args (pk 'apply args)))
                         #:metaclass <appclass>)
  $33 = #<<appclass> <foo> 16e0d20>
  scheme@(guile-user)> (define f (make <foo>))
  scheme@(guile-user)> (applicable-struct? f)
  $34 = 4
  scheme@(guile-user)> (applicable-struct-procedure f)
  $35 = #<procedure 1863060 at <current input>:51:0 args>
  scheme@(guile-user)> (f 1 2 3)

  ;;; (apply (1 2 3))
  $36 = (1 2 3)

But there should certainly be a higher-level facility.  :-)

Thanks,
Ludo’.

Re: Making every goops object applicable

[Mark H Weaver]

Krister Svanlund <krister.svanlund@gmail.com> writes:
> For example an instance of a class inheriting a class that inherits
> <applicable-struct> that defines 'procedure is not applicable.

Looking at the code, it is clear that in order for a GOOPS instance to
be applicable, it is not enough for <applicable-struct> to be a
superclass.  It is also necessary for the _metaclass_ of its class to
have the SCM_VTABLE_FLAG_APPLICABLE_VTABLE bit set.

One such metaclass is already built-in: <applicable-struct-class>, which
is the metaclass for <applicable-struct>, but if you'd like to define
your own metaclass for applicable objects, you can set the bit manually
on the metaclass as described by Ludovic.

However, keep in mind that the presence of that bit in the metaclass
indicates that the first slot of all instances _must_ be the procedure
slot.

     Mark

Re: Making every goops object applicable

[Andy Wingo]

Hi Krister,

On Mon 14 May 2012 19:24, Krister Svanlund <krister.svanlund@gmail.com> writes:

> In our work to look into how Python 3 could be implemented for Guile
> we have figured out that the only way to make a goops object
> applicable is to have it inherit <applicable-struct>. This does not
> always work the way it could be expected, for example when inheriting
> from several classes.

It should work.  It doesn't, as you have found.  The reason is that the
meta-object protocol for generic functions was never completely
implemented.  When/if someone implements it, the result will be objects
that do have that magical applicable flag set.

Regards,

Andy
-- 
http://wingolog.org/

Re: Making every goops object applicable

[Neil Jerram]

Mark H Weaver <mhw@netris.org> writes:

> Krister Svanlund <krister.svanlund@gmail.com> writes:
>> For example an instance of a class inheriting a class that inherits
>> <applicable-struct> that defines 'procedure is not applicable.
>
> Looking at the code, it is clear that in order for a GOOPS instance to
> be applicable, it is not enough for <applicable-struct> to be a
> superclass.  It is also necessary for the _metaclass_ of its class to
> have the SCM_VTABLE_FLAG_APPLICABLE_VTABLE bit set.
>
> One such metaclass is already built-in: <applicable-struct-class>, which
> is the metaclass for <applicable-struct>,

So the solution to Krister's problem is to make sure that every class
he's interested in is defined as

  (define-class <whatnot> (supers...)
    ...
    #:metaclass <applicable-struct-class>)

Right?

> but if you'd like to define
> your own metaclass for applicable objects, you can set the bit manually
> on the metaclass as described by Ludovic.

Would this be needed, if the above works?

> However, keep in mind that the presence of that bit in the metaclass
> indicates that the first slot of all instances _must_ be the procedure
> slot.

Might the (define-class ... #:metaclass <applicable-struct-class>)
ensure that, by some kind of of compute-slots customisation...?  I took
a look but unfortunately couldn't see this in the code.

       Neil

Re: Making every goops object applicable

[Mark H Weaver]

Hi Neil!

Neil Jerram <neil@ossau.homelinux.net> writes:
> Mark H Weaver <mhw@netris.org> writes:
>> Krister Svanlund <krister.svanlund@gmail.com> writes:
>>> For example an instance of a class inheriting a class that inherits
>>> <applicable-struct> that defines 'procedure is not applicable.
>>
>> Looking at the code, it is clear that in order for a GOOPS instance to
>> be applicable, it is not enough for <applicable-struct> to be a
>> superclass.  It is also necessary for the _metaclass_ of its class to
>> have the SCM_VTABLE_FLAG_APPLICABLE_VTABLE bit set.
>>
>> One such metaclass is already built-in: <applicable-struct-class>, which
>> is the metaclass for <applicable-struct>,
>
> So the solution to Krister's problem is to make sure that every class
> he's interested in is defined as
>
>   (define-class <whatnot> (supers...)
>     ...
>     #:metaclass <applicable-struct-class>)
>
> Right?

That was indeed my suggestion, though your comments below raise an
additional complication that I hadn't considered: some additional magic
will be needed to ensure that the 'procedure' slot comes first.

>> but if you'd like to define
>> your own metaclass for applicable objects, you can set the bit manually
>> on the metaclass as described by Ludovic.
>
> Would this be needed, if the above works?

No, but since this is part of an effort to implement Python 3 in Guile,
it's possible that they may need to create their own metaclass for other
reasons.

>> However, keep in mind that the presence of that bit in the metaclass
>> indicates that the first slot of all instances _must_ be the procedure
>> slot.
>
> Might the (define-class ... #:metaclass <applicable-struct-class>)
> ensure that, by some kind of of compute-slots customisation...?

I guess so, but I'm not convinced that this is the right approach.  It
now seems reasonably clear that GOOPS has poor support for applicable
objects, so we should think about a more proper solution that's easy to
use.

I don't see why applicable objects should need a custom metaclass.  In
general, it seems to me that features that require custom metaclasses do
not compose well.  Subclasses of <applicable-struct> should "just work"
without users resorting to these hacks.

As I've suggested before, the most elegant solution I can think of is to
simply make a two-argument generic function called something like
'generic-apply'.  I suspect that there's some clever way to make this
very fast using the MOP, but I have yet to work out the details.
Unfortunately my copy of AMOP is 50 miles away at the moment.

Suggestions solicited, and if anyone wants to take the lead on this,
I'd be grateful.  If not, I'll try to come up with something.

     Thanks,
       Mark