goops and memoization

goops and memoization

[Dirk Herrmann]

Hi folks,

my attempt to get goops working again after my changes to the evaluator
failed.  First, I find the goops implementation incomprehensible.  Second,
from what I understood the implementation of goops uses a lot of internal
knowledge about how the evaluator works internally.  It is not a trivial
task to keep goops in sync when some evaluator internals change.

Is there any goops expert around willing to improve the situation with
goops?

Best regards, 
Dirk Herrmann



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> Is there any goops expert around willing to improve the
    Dirk> situation with goops?

No, but I'm willing to try to become one.

If you can describe some of the things that you want
investigated/understood as a few specific questions, I'll take a look.

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Dirk Herrmann]

On 17 Nov 2002, Neil Jerram wrote:

> >>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
>     Dirk> Is there any goops expert around willing to improve the
>     Dirk> situation with goops?
> 
> No, but I'm willing to try to become one.
> 
> If you can describe some of the things that you want
> investigated/understood as a few specific questions, I'll take a look.

First, some explanation about the way my memoizer works:  My memoizer code
memoizes local variable accesses, that is, replaces the symbols by the
corresponding ilocs.  Symbols that correspond to accesses to global
variables are (at least for now) just left as they are.  The executor will
memoize them whenever the code is executed.  Thus, symbols that are left
within the code are known to belong to global variables.  Thus, I have
changed the executor such that if it finds a symbol in the code it only
looks up the module for a binding, not the local environment frames.

This conflicts with goops:  goops unmemoizes the function code, using
'procedure-source' (look into oop/goops/compile.scm).  This will re-create
the original code, including all the symbols that refer to local
variables.  This un-memoized code is then optimized in some way, and
re-written into the closure object.  Then, if the closure is evaluated, it
is not run through the memoizer again (since it is already a closure).
Now, if the executor runs over a symbol corresponding to a local variable,
it will treat is as a symbol belonging to a global variable and try to
find it within the module.

It would be great if you could take a look at goops and find a solution
for this problem.

Best regards,
Dirk



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

First a disclaimer:

I added GOOPS to Guile because I needed an object system in my neural
simulator.  I did not have infinite time to code, but I tried to focus
on at least getting the user-level definition reasonable so that we
would not build ourselves into a corner.  I looked at many different
kinds of object systems when arriving at the current GOOPS.

That said, quite a lot of the code was developed during a few days a
certain Christmas, and parts of compile.scm was written on Arlanda
airport while waiting for boarding an aircraft.  I urgently needed
someting working and planned to rewrite it later.

Well, later I judged it would serve people better to add the code to
the Guile source tree than letting people wait infinitely until my
rewrite would get high enough priority.  I then tried to get
volunteers to rewrite the code in C.

While there are "dark corners" in GOOPS I still dare to claim that it
works on sound principles, works reliably and is probably one of the
most efficient Scheme object systems.  Benchmarks show that GOOPS
method dispatch is negligible and a GOOPS generic function nearly as
efficient as an ordinary closure.

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> This conflicts with goops:  goops unmemoizes the function code, using
> 'procedure-source' (look into oop/goops/compile.scm).  This will re-create
> the original code, including all the symbols that refer to local
> variables.  This un-memoized code is then optimized in some way, and
> re-written into the closure object.  Then, if the closure is evaluated, it
> is not run through the memoizer again (since it is already a closure).
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Just a note: This is something which holds *after* your change, not previously.

> Now, if the executor runs over a symbol corresponding to a local variable,
> it will treat is as a symbol belonging to a global variable and try to
> find it within the module.

I'm sure Neil would find his way in the code, but since the answer
seems obvious to me because of my previous knowledge, I'll give some
hints:

A few notes on GOOPS method dispatch:

As in most CLOS-like systems, GOOPS method dispatch is quite advanced.
For example, a generic function application involves sorting the
applicable methods after specificity.  We might in fact like to make
it even more advanced (adding support for singletons and type coercion
for example).

To get GOOPS fast, we cache the results of this involved process in a
"method cache" in the generic function.  Next time a GF is applied to
the same set of argument types, the evaluator just makes a quick
lookup in the cache and can start to evaluate the method body without
delay.  The format of the method cache is described in
oop/goops/dispatch.scm.

The procedure compile-method in oop/goops/compile.scm takes the sorted
list of applicable methods and a list of the argument types in the GF
application and returns a "cmethod", which is the tail of a method
cache entry:

  CMETHOD ::= (CLOSURE-ENVIRONMENT FORMALS BODY-FORM1 ...)

Currently, compile-method only makes sure the method has a local
next-method binding (specific to this method cache entry) if that is
needed.  However, there are plans to let it make powerful
optimizations of the code, with great potential of reducing overhead.

It seems to me that what you need to do is to run the tail of the
cmethod (BODY-FORM1 ...) through your memoizer.  That should fix it.
In fact, as you see above, a cmethod contains the same information as
a closure.

Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

> Currently, compile-method only makes sure the method has a local
> next-method binding (specific to this method cache entry) if that is
> needed.  However, there are plans to let it make powerful
> optimizations of the code, with great potential of reducing overhead.
>
> It seems to me that what you need to do is to run the tail of the
> cmethod (BODY-FORM1 ...) through your memoizer.  That should fix it.

(An alternative solution is to implement compile-method in C,
memoizing the source code while walking through it.  In fact, that
could mean that we don't need to call procedure-source => no need to
unmemoize code.)


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Mikael" == Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

    Mikael> (An alternative solution is to implement compile-method in
    Mikael> C, memoizing the source code while walking through it.  In
    Mikael> fact, that could mean that we don't need to call
    Mikael> procedure-source => no need to unmemoize code.)

A further possibility occurs to me - transforming the method
definition at define-method time something like this:

(define-macro (make-method gf specializers formals . body)
  `(letrec ((m (lambda ,formals
                 (define (next-method)
                   (call-next-method ,gf
                                     ,specializers
                                     m
                                     ,@formals))
                 ,@body)))
     m))

Given the gf and specializers, call-next-method works out a list of
applicable methods (probably cached) in the same way as for a normal
gf application, then it looks through this list for m and applies the
method after m to the supplied arguments (formals).

Would this kind of approach work?

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Neil Jerram <neil@ossau.uklinux.net> writes:

>>>>>> "Mikael" == Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:
>
>     Mikael> (An alternative solution is to implement compile-method in
>     Mikael> C, memoizing the source code while walking through it.  In
>     Mikael> fact, that could mean that we don't need to call
>     Mikael> procedure-source => no need to unmemoize code.)
>
> A further possibility occurs to me - transforming the method
> definition at define-method time something like this:
>
> (define-macro (make-method gf specializers formals . body)
>   `(letrec ((m (lambda ,formals
>                  (define (next-method)
>                    (call-next-method ,gf
>                                      ,specializers
>                                      m
>                                      ,@formals))
>                  ,@body)))
>      m))
>
> Given the gf and specializers, call-next-method works out a list of
> applicable methods (probably cached) in the same way as for a normal
> gf application, then it looks through this list for m and applies the
> method after m to the supplied arguments (formals).
>
> Would this kind of approach work?

It would "work", but gee...  then you are talking about a totally
different speed regime than what holds now.

Are you saying that you'd want to compute the list of applicable
methods at every call to (next-method)?  That way overhead per call
would become O(NM) where N is the number of applicable methods and M
the maximum length of argument lists.

Well, then I'd start avoiding to use next-method in certain code, and
I'd really hate that.

Call me a speed maniac, but I want to be able to write my programs in
a form that is similar to how I think of the problem without worrying
that it would be too slow.  Basic language mechanisms *must* be
efficient.  You may say (well, *you*, Neil, wouldn't) that "well, then
you shouldn't use scheme", but my point is that I'd like to use these
nice ways of expressing programs in as great extent as possible.

[For those who still think I'm being silly: My programs currently run
for *days* just analyzing one data set...]

Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

>> Given the gf and specializers, call-next-method works out a list of
>> applicable methods (probably cached) in the same way as for a normal
>> gf application, then it looks through this list for m and applies the
>> method after m to the supplied arguments (formals).

[...]

> Are you saying that you'd want to compute the list of applicable
> methods at every call to (next-method)?  That way overhead per call
> would become O(NM) where N is the number of applicable methods and M
> the maximum length of argument lists.

Oops, didn't see that you wrote "probably cached".  But if you mean in
the same way as in gf application your suggested approach wouldn't
work because the list of applicable methods is unique per application
arglist signature and gf application also caches per application, not
per method definition.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Mikael" == Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

    Mikael> [...] because the list of applicable methods is unique per
    Mikael> application arglist signature and gf application also
    Mikael> caches per application, not per method definition.

I see this now.  A case that makes it clear is this:

(define-method (foo (arg <super1>))
  ...
  (next-method))
(define-class <derived> (<super1> <super2>))
(define-method (foo (arg <super2>))
  ...)
(foo (make <derived> ...))

The first define-method cannot possibly know at definition time that,
in the eventual application, next-method should be pointing to the
method for super2.

There is also this, which doesn't rely on multiple inheritance ...

(define-class <base>)
(define-class <derived1> (<base>))
(define-class <derived2> (<derived1>))
(define-method (foo (arg <base>))
  ...)
(define-method (foo (arg <derived2>))
  ...
  (next-method))
(foo (make <derived2> ...))
(define-method (foo (arg <derived1>))
  ...
  (next-method))
(foo (make <derived2> ...))

But this one could be handled by fixing up the next-method chain when
the method for <derived1> is defined, so isn't such a compelling
example.

I'm not proposing to pursue the method-definition-time approach.  Out
of interest, however, are there any hard cases that don't rely on
multiple inheritance?

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Neil Jerram <neil@ossau.uklinux.net> writes:

> I'm not proposing to pursue the method-definition-time approach.  Out
> of interest, however, are there any hard cases that don't rely on
> multiple inheritance?

Yes.  Consider:

(define-method (foo (x <top>) (y <top>)) 'top)
(define-method (foo (x <real>) (y <integer>)) 'middle)
(define-method (foo (x <integer>) (y <integer>)) (next-method))
(foo 1 1) --> middle
(foo 1.0 1.0) --> top

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Mikael" == Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

    Mikael> I added GOOPS to Guile because I needed an object system
    Mikael> in my neural simulator. ... later I judged it would serve
    Mikael> people better to add the code to the Guile source tree
    Mikael> than letting people wait infinitely until my rewrite would
    Mikael> get high enough priority.

There's no question in my mind that you did the right thing here.

    Mikael> While there are "dark corners" in GOOPS I still dare to
    Mikael> claim that it works on sound principles, works reliably
    Mikael> and is probably one of the most efficient Scheme object
    Mikael> systems.  Benchmarks show that GOOPS method dispatch is
    Mikael> negligible and a GOOPS generic function nearly as
    Mikael> efficient as an ordinary closure.

Are those benchmarks available?  If I do change the code, it would be
good to check that the changes don't hurt performance.

    Mikael> To get GOOPS fast, we cache the results of this involved
    Mikael> process in a "method cache" in the generic function.  Next
    Mikael> time a GF is applied to the same set of argument types,

... and if no new methods have been added in the meantime ...

    Mikael> the evaluator just makes a quick lookup in the cache and
    Mikael> can start to evaluate the method body without delay.  The
    Mikael> format of the method cache is described in
    Mikael> oop/goops/dispatch.scm.

    Mikael> Currently, compile-method only makes sure the method has a
    Mikael> local next-method binding (specific to this method cache
    Mikael> entry) if that is needed.  However, there are plans to let
    Mikael> it make powerful optimizations of the code, with great
    Mikael> potential of reducing overhead.

Do you mean IOR ?  If so, couldn't most IOR optimizations be made when
the method is first defined?

Thanks for the explanations.

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Neil Jerram <neil@ossau.uklinux.net> writes:

>     Mikael> Benchmarks show that GOOPS method dispatch is
>     Mikael> negligible and a GOOPS generic function nearly as
>     Mikael> efficient as an ordinary closure.
>
> Are those benchmarks available?

Checkout the guile/guile-modules/benchmarks CVS module at
subversions.gnu.org.  There should be a benchmark goops.scm.  You run
it with (type-dispatch-run).

> If I do change the code, it would be good to check that the changes
> don't hurt performance.

Yes.  I would like to be able to continue using GF:s in the inner
loops of my applications.

>     Mikael> Currently, compile-method only makes sure the method has a
>     Mikael> local next-method binding (specific to this method cache
>     Mikael> entry) if that is needed.  However, there are plans to let
>     Mikael> it make powerful optimizations of the code, with great
>     Mikael> potential of reducing overhead.
>
> Do you mean IOR ?

Some of the optimizations mentioned in the IOR text can be done in
current GOOPS.

> If so, couldn't most IOR optimizations be made when the method is
> first defined?

No.  They are dependent on the actual argument types in the
application, not the possible ones.


BTW, maybe I should add something I didn't mention about GOOPS type
dispatch: The method cache uses hashing if the number of methods goes
above a small threshold.  This hashing scheme uses an optimization
which allows for direct hits of the correct entry in the majority of
lookups.

Method cache hash optimization scheme
-------------------------------------

The basic idea is to compute a hash code from the actual arguments of
an application and reduce this code to a hash index in the cache.

The type of each argument is represented by a class object.  Each
class object has 8 hash values.  (Hash values # 1 of all classes in
the system constitutes "hash set 1" etc.)  When building the method
cache, all hash sets are tried, and the one which gives most direct
hits is chosen.  Information about which hash set is used is stored in
the GF object.

Apart from optimizing dispatch this also results in compact
representations of the method caches.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Mikael" == Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

    Mikael> Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    >> This conflicts with goops:  goops unmemoizes the function code, using
    >> 'procedure-source' (look into oop/goops/compile.scm).  This will re-create
    >> the original code, including all the symbols that refer to local
    >> variables.  This un-memoized code is then optimized in some way, and
    >> re-written into the closure object.  Then, if the closure is evaluated, it
    >> is not run through the memoizer again (since it is already a closure).
    Mikael>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    Mikael> Just a note: This is something which holds *after* your
    Mikael> change, not previously. [...]  

    Mikael> It seems to me that what you need to do is to run the tail of the
    Mikael> cmethod (BODY-FORM1 ...) through your memoizer.  That should fix it.

Indeed.  If I understand correctly, what happens is that
scm_memoize_method in eval.c returns an unevaluated and unmemoized
lambda form, which the current evaluator handles by calling
scm_m_lambda, which performs the memoization stage.

The problem I presume is that your optimized evaluator doesn't handle
symbols in the CAR, because it shouldn't need to.

If this is correct, the fix is to pass `x' through your memoizer just
before `goto nontoplevel_begin;', like this:

	    apply_cmethod: /* inputs: z, arg1 */
	      {
		SCM formals = SCM_CMETHOD_FORMALS (z);
		env = EXTEND_ENV (formals, arg1, SCM_CMETHOD_ENV (z));
		x = SCM_CMETHOD_BODY (z);
                x = memoize (x);             /* ADDED */
		goto nontoplevel_begin;
	      }

A few notes:

- You could just call eval instead of memoizing and goto
  nontoplevel_begin, but that wouldn't be tail-recursive.  I wonder if
  tail recursion is important here?

- Is the memoization stage sufficient here?  What if the method
  definition came from a module with a syntax transformer in effect?
  If `procedure-source' returns post-transformation code, all is OK.
  If it doesn't, there are two possible problems:

  - The code returned by scm_memoize_method should be retransformed as
    well as rememoized.

  - `compile-method's manipulation of the source code might not work
    in the pre-transformation language.

- Quite a few places in the GOOPS code use local-eval.  Does
  local-eval still include memoization (and syntax transformation?) in
  your codebase?

I hope this helps; let me know if you would like me to dig or
experiment further.

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Neil" == Neil Jerram <neil@ossau.uklinux.net> writes:

    Neil> If this is correct, the fix is to pass `x' through your memoizer just
    Neil> before `goto nontoplevel_begin;', like this:

    Neil> 	    apply_cmethod: /* inputs: z, arg1 */
    Neil> 	      {
    Neil> 		SCM formals = SCM_CMETHOD_FORMALS (z);
    Neil> 		env = EXTEND_ENV (formals, arg1, SCM_CMETHOD_ENV (z));
    Neil> 		x = SCM_CMETHOD_BODY (z);
    Neil>                 x = memoize (x);             /* ADDED */
    Neil> 		goto nontoplevel_begin;
    Neil> 	      }

And to follow up immediately to myself ... actually this is suboptimal
because it means that even a cached method is memoized every time it
is called.  (Also I was wrong about the lambda; the expression
returned by scm_memoize_method doesn't begin with a lambda.)

What we should really do is memoize before the method is installed
into the method cache.  I think the place for this is in
compute-entry-with-cmethod in oop/goops/compile.scm: just before
`(cons entry place-holder)':

(define (compute-entry-with-cmethod methods types)
  (or (code-table-lookup (slot-ref (car methods) 'code-table) types)
      (let* ((method (car methods))
	     (place-holder (list #f))
	     (entry (append types place-holder)))
	;; In order to handle recursion nicely, put the entry
	;; into the code-table before compiling the method 
	(slot-set! (car methods) 'code-table
		   (cons entry (slot-ref (car methods) 'code-table)))
	(let ((cmethod (compile-method methods types)))
	  (set-car! place-holder (car cmethod))
	  (set-cdr! place-holder (cdr cmethod)))
        (set-cdr! (cdr place-holder)                  ; ADDED
                  (memoize (cdr place-holder)))       ; ADDED
	(cons entry place-holder))))

   Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> It would be great if you could take a look at goops and find
    Dirk> a solution for this problem.

I will (try to) do so.  Do you have any particular time that you need
this by?

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Dirk Herrmann]

On 21 Nov 2002, Neil Jerram wrote:

> >>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:
> 
>     Dirk> It would be great if you could take a look at goops and find
>     Dirk> a solution for this problem.
> 
> I will (try to) do so.  Do you have any particular time that you need
> this by?

I can't tell you date and time, but I think it will not be possible to
integrate separate memoization and execution before this issue is solved.

Best regards
Dirk Herrmann



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Neil Jerram]

>>>>> "Dirk" == Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

    Dirk> The best solution would be if the optimization that goops
    Dirk> performs would be done directly on the memoized code,
    Dirk> without having to unmemoize-transform-rememoize the code.
    Dirk> It would be great if you could look into places in goops
    Dirk> (or, maybe even guile in general) where such interactions
    Dirk> occur.  If I had a list of these places, I could then try to
    Dirk> remove the optimizing code from the scheme level and re-code
    Dirk> it in C, where it would be possible to perform the
    Dirk> optimizations directly on the memoized code.

Fair enough; I'll keep thinking and looking into this.  Can you share
your code with me somehow, so that I can try possibilities out?

        Neil



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

>>     Mikael> It seems to me that what you need to do is to run the
>>     Mikael> tail of the cmethod (BODY-FORM1 ...) through your
>>     Mikael> memoizer.  That should fix it.

[...]

> Simple re-memoization is also not possible, except you use special
> tricks, similar to using local-eval.

I don't understand why this is so complicated.  I haven't seen your
memoizer, but doesn't it consist of recursively called function(s)
which pass an argument representing the local environment, just as
eval and the unmemoizer?  As you know, a cmethod has the local
environment as a component.  Can't you use that?  You could then
implement a primitive which can memoize a cmethod, and you could call
it at the end of compile-method just as Neil suggests.

>> - You could just call eval instead of memoizing and goto
>>   nontoplevel_begin, but that wouldn't be tail-recursive.  I wonder if
>>   tail recursion is important here?
>
> I can't tell.

It is important.  GOOPS methods need to handle tail-recursion nicely.

> procedure-source returns post-transformation code.  But it is still a
> hack.  Further, it is not a good idea to rely on the fact that is returns
> post-transformation code:  From a debugging perspective, it might be
> better to return the original code.  It is IMO bad style to rely on the
> behaviour of procedure-source.  It was a short term hack, as the comments
> in goops indicate.

I don't agree that this in particular is a hack.  compile-method needs
to work on some representation of the logic of a method.  I think the
source is an excellent representation for the logic of the method, and
we have a primitive in the Guile API which provides the source:
procedure-source.  I actually think working on the memoized
representation is more of a hack, but that might be motivated anyway
for reasons of efficiency.

Then we can argue about the inner workings and the exact result of
procedure-source, but that is a different issue.  Maybe we should
finally store the original source somewhere, but currently there is
not much difference between unmemoized source and the original.  The
only difference is that the (irrelevant) distinction between let and
let* for a single binding is lost.

Also, note that for GOOPS, it doesn't matter much how well the source
resembles the original source as long as the semantics is intact.

>> - Quite a few places in the GOOPS code use local-eval.  Does
>>   local-eval still include memoization (and syntax transformation?) in
>>   your codebase?
>
> Yes, but IMO local eval is also quite hackish.  It is placed in debug.c
> which indicates to me that it was planned as a debugging aid.  It should
> probably be avoided where possible.  Maybe it is just a bad idea anyway.

It is placed in debug.c for obscure historic reasons.  But you're
right that we may want to remove local-eval in the future.  (BTW, note
that some scheme interpreters, like MIT scheme does support evaluation
in local environments.)

However, as long as the interpreter is organized the way it is now, I
see no reason why we can't use it internally if that gives
performance.  It's pretty easy to change when we want to do that.

> I think the whole concept of unmemoization and re-memoization is broken.
> IMO the optimizations that are done by goops should either be performed on
> scheme code or some close-to-scheme intermediate representation before
> memoization, or if they are based on already memoized code, they should
> work on the memoized code itself.  Mixing different stages as it is
> currently done introduces a lot of dependencies between different parts of
> guile.  This makes maintaining guile quite hard - our current discussion
> is already an example of the problem.

As I say above, I think one can look upon it differently: We have
procedure-source in our API.  That call will always be supported, so
we can rely on it.  Then it's completely OK to do optimizations and
then memoize, just as we'd memoize novel source.  But then, again, if
we want method compilation to work fast, we might want to work in the
memoized source, with the disadvantage that we'd be bound to do all
manipulations on the Scheme level and couldn't include method
compilation in the MOP.

> It would be great if you could look into places in goops (or, maybe
> even guile in general) where such interactions occur.  If I had a
> list of these places, I could then try to remove the optimizing code
> from the scheme level and re-code it in C, where it would be
> possible to perform the optimizations directly on the memoized code.

I'm not sure what you mean by "interactions", but as far as I remember
right now, the *only* reason why GOOPS doesn't work with your changes
is that the output of compile-method should now be memoized.

Again, can't you just provide a primitive which can take a cmethod
(which has environment and body as components) and memoize it?  That
primitive can then by applied at just the location in compile-cmethod
where Neil suggested.

On the other hand: Sure, it would be very nice if you re-implemented
compile-method in C.  Please tell me if you have any questions.

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

> Mixing different stages as it is currently done introduces a lot of
> dependencies between different parts of guile.  This makes
> maintaining guile quite hard - our current discussion is already an
> example of the problem.

Just to clarify my points:

Working on the output of procedure-source does not introduce
dependencies.  On the contrary: Since the output of procedure-source
is required to be Scheme code, we have a completely clean separation
of different parts of Guile.  Regardless of how the evaluator changes,
this representation will remain valid and method compilation will
continue working.

However, working on the memoized representation *would* introduce a
dependency between Goops code and the current version of the
evaluator.  (I have the view, though, that that is acceptable.)

The current problem arises not because of how compile-method retrieves
its input, but because of how the result of the work of compile-method
is "returned" to the rest of the system.  Previously, it was OK to
regard the compile-method output as an executable representation, now
it is not.

The core of the problem is that while there exists a well-defined
interface for retrieving the representation of the method,
procedure-source, there doesn't exist a well-defined interface for
giving a compiled method back to the evaluator.

A standardized way would be to pass Scheme code in the form of a
lambda expression to "eval", but it is simply not possible to use that
standard interface, because methods need to preserve their local
environment.

So, again, if you want to preserve separation of parts of Guile, you
should provide the interface for giving compiled method source (output
of compile-method) back to Guile.

If you are prepared to sacrifice separation for efficiency, you should
implement compile-method in C and let it operate on the memoized
representation.

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Lynn Winebarger]

On Monday 02 December 2002 03:45, Mikael Djurfeldt wrote:
> procedure-source.  I actually think working on the memoized
> representation is more of a hack, but that might be motivated anyway
> for reasons of efficiency.

       The problem with "unmemoizing" is that you don't have any
guarantee that 'lambda is bound to the constant system lambda
macro when you unmemoize.  The real justification for memoizing
is that you're evaluating the variables 'lambda,'if,etc to system 
constants.  

Lynn


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Lynn Winebarger <owinebar@free-expression.org> writes:

> On Monday 02 December 2002 03:45, Mikael Djurfeldt wrote:
>> procedure-source.  I actually think working on the memoized
>> representation is more of a hack, but that might be motivated anyway
>> for reasons of efficiency.
>
>        The problem with "unmemoizing" is that you don't have any
> guarantee that 'lambda is bound to the constant system lambda
> macro when you unmemoize.  The real justification for memoizing
> is that you're evaluating the variables 'lambda,'if,etc to system 
> constants.  

Well, actually we have a guarantee that when we re-memoize, 'lambda,
'if etc are bound to exactly the same thing as when the procedure was
originally memoized.  This is because the lexical environment of the
method is used when re-memoizing.

Note that for method optimization to work, unmemoization *must* be
faithful to the semantics of the procedure.  The optimizer must be
able to lookup the true binding of every identifier.

Thus, we have the requirement that memoization is semantically 100%
reversible.  Then one might of course argue that that is a too strict
requirement.

The original reason for the choice to work on Scheme code instead of
on the memoized representation was that it was simpler and could be
handled on the Scheme level, and could be made to work quickly.

Then it happens that it is cleaner and more modular since the
procedure-source interface separates goops code from the particular
details of the current evaluator.  (True with regard to this question.
However, the code unfortunately contains dependencies on the
evaluator's representation of the environment.  *That* is a hack.)

But, as said earlier, being a efficiency freak, I'd like a solution
which works on the memoized code.  In order to maintain a reasonably
clean separation of goops code and the details of memoizing, it might
then be a good idea to provide some kind of code traverser which goops
can plug into and operate with minimal knowledge of memoized
representation and environment.

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Tom Lord]

       > But, as said earlier, being a efficiency freak, I'd like a
       > solution which works on the memoized code.  In order to
       > maintain a reasonably clean separation of goops code and the
       > details of memoizing, it might then be a good idea to provide
       > some kind of code traverser which goops can plug into and
       > operate with minimal knowledge of memoized representation and
       > environment.


Abstractly, you're computing something like:

	(M (O (U (M <source>))))

where M is the memoizer, O the optimizer, and U the unmemoizer.

Your ideal is to have O be a clean, perhaps even portable,
source->source transform that knows nothing about memoization, yet for
the whole composition to be fast.

Have you considered the approach of writing a custom optimizer (not O,
but another optimizer) that can do a good job of "compiling" (scheme
to scheme):


	(lambda (ms) (M (O (U ms))))

?

If O is very clean/high-level, you can do all kinds of tricks by
transforming it (e.g. automatic conversion to demand-driven
execution).  Writing O is just writing (with nothing extraneous) the
transforms that describe the possible optimizations, and then tools
that compile O and compositions of O with M and U can work out when to
apply those transformations and short-cut the compositions.


-t


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Tom Lord <lord@regexps.com> writes:

>        > But, as said earlier, being a efficiency freak, I'd like a
>        > solution which works on the memoized code.  In order to
>        > maintain a reasonably clean separation of goops code and the
>        > details of memoizing, it might then be a good idea to provide
>        > some kind of code traverser which goops can plug into and
>        > operate with minimal knowledge of memoized representation and
>        > environment.
>
> Abstractly, you're computing something like:
>
> 	(M (O (U (M <source>))))
>
> where M is the memoizer, O the optimizer, and U the unmemoizer.
>
> Your ideal is to have O be a clean, perhaps even portable,
> source->source transform that knows nothing about memoization, yet for
> the whole composition to be fast.

Yes.

> Have you considered the approach of writing a custom optimizer (not O,
> but another optimizer) that can do a good job of "compiling" (scheme
> to scheme):
>
>
> 	(lambda (ms) (M (O (U ms))))
>
> ?
>
> If O is very clean/high-level, you can do all kinds of tricks by
> transforming it (e.g. automatic conversion to demand-driven
> execution).  Writing O is just writing (with nothing extraneous) the
> transforms that describe the possible optimizations, and then tools
> that compile O and compositions of O with M and U can work out when to
> apply those transformations and short-cut the compositions.

Hmm... could you please clarify this suggestion.  What is the core
idea?  To specify O in a custom language and compile it?  To optimize
the composition of M O and U?  Something else that I have missed?

Best regards,
Mikael Djurfeldt


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Tom Lord]

       > Have you considered the approach of writing a custom optimizer (not O,
       > but another optimizer) that can do a good job of "compiling" (scheme
       > to scheme):
       >
       >
       > 	(lambda (ms) (M (O (U ms))))
       >
       > ?


       Hmm... could you please clarify this suggestion.  What is the
       core idea?  To specify O in a custom language and compile it?
       To optimize the composition of M O and U?  Something else that
       I have missed?


O, I am presuming, optimizes method selection, presumbably by ordering
tests of argument types in favorable orders and by generating
special-case tests where, otherwise, a generic search would be
required.   Have I missed something?

So, O is a fairly straightforward scheme->scheme transform.  It can
almost certainly be expressed in very regular form (a "custom
language" (such as a pattern-matching macro plus quasiquote) -- or
simply a narrow Scheme subset).

M and U are similar.

You can write them as three separate functions, and write a customized
optimizer that folds them together.  It might even be able to take
advantage of non-general optimizations that apply only to these
functions.

You could look at the custom optimizer as a constant-folding exercise,
an abstract-evaluation exercize, or a partial-evaluation exercize
(where you are partially evaluating "(apply (compose M O U)
free-ms)").

The reason I think this is a plausible approach is just that the three
transforms involved (M O and U) don't need to be arbitrary code -- you
can do very well here just by thinking of them as pattern-matching
rewrite systems -- and such systems compose cleanly and should be easy
to optimize.

Clearer?  or did I just make it worse :-)


(I used to suspect that Aubrey secretly generated `eval' using
techniques along these lines but eventually concluded: nah, he just
writes very consistent code. :-)

-t



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Lynn Winebarger]

On Tuesday 03 December 2002 02:59, Mikael Djurfeldt wrote:
> Well, actually we have a guarantee that when we re-memoize, 'lambda,
> 'if etc are bound to exactly the same thing as when the procedure was
> originally memoized.  This is because the lexical environment of the
> method is used when re-memoizing.

   Either  define-syntax or define can change the meaning of
lambda in the global environment.  It will use the same binding,
true, but that's not the same thing.

> Note that for method optimization to work, unmemoization *must* be
> faithful to the semantics of the procedure.  The optimizer must be
> able to lookup the true binding of every identifier.

      The memoizer is not an optimizer.  It merely expands macros
into a constant core representation.  I fail to understand the problem.
The internal constants are precisely what you want - a representation
of core scheme (plus some things like "and" and "or" that are
syntactic sugar but implemented for speed).   They're not pointers
into a run-time symbol table, but so what?  It's easy to translate
them to symbols for printing purposes.

> Thus, we have the requirement that memoization is semantically 100%
> reversible.  Then one might of course argue that that is a too strict
> requirement.

      Requiring reversibility of arbitrary macro expansions is pretty
close to nuts.

> The original reason for the choice to work on Scheme code instead of
> on the memoized representation was that it was simpler and could be
> handled on the Scheme level, and could be made to work quickly.

       If you do it at the scheme level, sure.  But if you're doing it at the
C level you have to use some system-specific representation anyway.
So what's wrong with constants?

Lynn



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Lynn Winebarger <owinebar@free-expression.org> writes:

> On Tuesday 03 December 2002 02:59, Mikael Djurfeldt wrote:
>> Well, actually we have a guarantee that when we re-memoize, 'lambda,
>> 'if etc are bound to exactly the same thing as when the procedure was
>> originally memoized.  This is because the lexical environment of the
>> method is used when re-memoizing.
>
>    Either  define-syntax or define can change the meaning of
> lambda in the global environment.  It will use the same binding,
> true, but that's not the same thing.

Since procedure-source operates on a closure, I already know for sure
what kind of "lambda" created it.  As for other mentions of lambda,
procedure-environment will tell me what they mean.

>       The memoizer is not an optimizer.  It merely expands macros
> into a constant core representation.  I fail to understand the
> problem.

Problem 1: The optimizer gets dependent upon the memoized
           representation.
           
  [The goops dependencies that Dirk mention are really minor (except
   for the current lack of an abstraction barrier towards the
   environment) and can be adjusted quickly if things changes.]

Problem 2: Working on the memoized representation prevents writing
           optimizations in Scheme.

>> Thus, we have the requirement that memoization is semantically 100%
>> reversible.  Then one might of course argue that that is a too strict
>> requirement.
>
>       Requiring reversibility of arbitrary macro expansions is pretty
> close to nuts.

Reversibility of macro expansions is not required.  The optimizer in
fact works better if all macros already are expanded.

>> The original reason for the choice to work on Scheme code instead of
>> on the memoized representation was that it was simpler and could be
>> handled on the Scheme level, and could be made to work quickly.
>
>        If you do it at the scheme level, sure.  But if you're doing it at the
> C level you have to use some system-specific representation anyway.
> So what's wrong with constants?

Nothing except for problems 1 and 2 above.

(Clarification regarding problem 1: The "system-specific
representation" you talk about is not likely to change and trivial to
modify if a change should occur while large parts of the optimizer
might need to be rewritten if the memoized representation changes.
Consider, for example, that the tree structure of IM_DO is different
from that of "do" and it's quite easy to imagine quite a lot stranger
representations.)

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> Certainly this could be done as a temporary solution, but it is still not
> a clean approach, since it still has dependencies on a lot of evaluator
> internals.  Even if my current implementation of the memoizer is closely
> related to the old way the evaluator works, it may make sense to change
> this in the future.  That is, the representation of environment frames
> during memoization and during execution may differ for good reasons.

Certainly.  As I've "confessed" in another letter, the assumptions on
the environment representation is a dirty hack.  One way of handling
this is to use an abstraction such as "lookup":
eval_environment_lookup (SYMBOL, ENV).

[Excellent suggestions for using different env representation during
memoization deleted.]

> I disagree that the result of procedure-source is an excellent
> representation of the logic of the method.  procedure-source does not
> provide the context of the lambda expression:
>   guile> (define foo (let ((bar 1)) (lambda () bar)))
>   guile> (procedure-source foo)
>   --> (lambda () bar)
> That is, the source of the procedure foo does not provide all that is
> needed to understand the logic of the method.  This is a principal problem
> of using procedure-source.  In order to compile the output of
> procedure-source correctly, you need more than the result of
> procedure-source, namely something like procedure-environment-info.

Well, I thought that was implicit in my argument.  compile-method
indeed uses the pair procedure-source, procedure-environment.  Of
course I never meant that the source without its environment is a
valid representation of the logic.  I hope the actual behavior of
goops methods proves that.

But I maintain that the pair of source and environment provides 100%
of the semantics.

> I have suggested this as a temporary fix, but not for performance
> reasons: Goops does already interfere at some places with the
> evaluator and the representation of the memoized code: First, there
> are three memoizer codes that are specifically added for goops.
> Second, the method dispatch code is copied verbatim into the
> evaluator.  Third, during execution of the code dispatch goops
> modifies the memoized code for storing hash tables for faster
> function dispatch.

(...the reason being performance (measured in benchmarks, BTW),
 but I think you make it sound worse than it is.  Two of the codes are
 very simple (object slot access).  The dispatch form does contain
 dependencies on the evaluator for the evaluation of operand forms,
 but the rest is a section of self-contained code operating on a data
 structure which is entirely "owned" and handled by goops.  It's not
 that a lot would need adaptation if the evaluator changes.)

> A cleaner approach to optimization would be to allow compiler
> optimizations in an earlier stage.  Personally I'd prefer a solution with
> five phases:  1) reading 2) macro expansion (scheme -> intermediate
> format) 3) optimization (intermediate format -> intermediate format) 4)
> memoization (intermediate format -> memoized code) 5) execution.  For the
> intermediate format I think of some very close to scheme format using
> keywords like #:if and such (see new-model.text) as Marius has suggested.
> This ought to be very stable.  The memoized code, in contrast could be
> somewhat less stable.  Goops would add its optimizations functions to step
> 3.

Well, this all sounds good.  But note that the use of
procedure-source/environment in no way put serious constraints on the
steps above.  For example, steps 1, 2 and 4 could be performed when
constructing the closure.  Then an unmemoization step (U) followed by
steps 3 and 4 would lead to 5.  Basically, the inserted extra 4 and U
are the only differences.

There are a couple of constraints which I should mention here:

1. The MOP currently specifies that a method is created from a list of
   specializers and a closure by a call to `make'.

   The main reason for using a closure is that it is a nice way of
   capturing the lexical environment without the need to use a
   non-R5RS form and introducing an explicit representation of the
   environment.  Another reason is that this pattern was used in
   Kiczales tiny-clos MOP---reflexive code which describes itself
   beautifully.  Even though goops has dark corners "under the hood",
   the goops MOP provides a conceptually simple model for its
   semantics.

   The MOP also allows us to specialize `make' to a subclass of
   <method>.  If we try to get around the problem of capturing the
   environment by limiting ourselves to the special form `method', we
   loose this aspect of the MOP.

   So, the only alternatives I see are either a closure or passing the
   source expression together with the result of a call to
   (the-environment).

2. Goops method optimizations should not be done in advance.  They are
   supposed to be performed at the exact instance when a generic
   function is called with a particular combination of argument types
   for the first time.  That is of course possible to reconcile with
   the steps above, although steps 3-4 will be folded into the
   execution.

> One could think of the possibility to add hooks for adding arbitrary
> optimization functions, but this is just an idea.

That is an exciting idea.  It would be wonderful to be able to specify
code re-writing rules such as:

  (map foo (map bar ls)) --> (map (lambda (x) (foo (bar x))))

> The reason why goops does not work with my changes is that goops
> relies not only on the working of procedure-source

Which should be of no problem to you unless you want to remove it.  If
you want that, there are other reasons than we discuss here for
keeping it.

> but also on the way environment frames are stored

Yes.

> and on the way the evaluator works.

Not sure what you mean here.

> As long as you don't have a perfect system, you have to build upon
> the things that are available to you.

Yes, unfortunately so.

> With "interactions" I meant holes in the abstraction barrier, where
> code relies on the behaviour of unmemoization

It does in no way depend on the behaviour of unmemoization, unless the
unmemoized source is semantically something different.  The unmemoized
source + the lexical environment is sufficient for full semantics.
Sorry for repeating this statement, but it is important that you prove
me wrong if I am wrong.

> the internal representation of environments

Yes, this is a hole.


> It becomes more and more clear to my why so many previous approaches to
> work on the evaluator have failed:  The dependencies between the evaluator
> and other parts of guile make it necessary to understand and work on quite
> a lot of different parts of guile.

You're absolutely right.  It is very important to work towards good
modularization, especially in this kind of collaborative project.
Don't think I'm not trying...  Although for many years there seemed to
be higher priorities than taking on the monolithic SCM evaluator.

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Tom Lord]

       >> One could think of the possibility to add hooks for adding arbitrary
       >> optimization functions, but this is just an idea.

       > That is an exciting idea.  It would be wonderful to be able to specify
       > code re-writing rules such as:

       > (map foo (map bar ls)) --> (map (lambda (x) (foo (bar x))))

That is not, in general, a valid optimization.

Additionally, only a newbie would write it the "wrong" way unless they
specifically wanted the effects that the bogus optimization would
break or knew that speed was not an issue for this expression.  If you
mean that nested maps arise out of some higher-level code generation
for which the optimization is known to be valid -- then you'd want to
express the optimization in terms of that higher-level notation.

-t


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Rob Browning]

Mikael Djurfeldt <mdj@kvast.blakulla.net> writes:

> 2. Goops method optimizations should not be done in advance.  They are
>    supposed to be performed at the exact instance when a generic
>    function is called with a particular combination of argument types
>    for the first time.  That is of course possible to reconcile with
>    the steps above, although steps 3-4 will be folded into the
>    execution.

Hmm.  So what's your impression of how GOOPS might (or might not) fit
in with offline compilation where you might want to produce a .o file
that will be executed later?

(Note that although I've been talking about offline compilation, I'm
 not dead set on it, but I would like to understand the issues
 involved more clearly.)

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG starting 2002-11-03 = 14DD 432F AE39 534D B592  F9A0 25C8 D377 8C7E 73A4


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: goops and memoization

[Mikael Djurfeldt]

Dirk Herrmann <dirk@sallust.ida.ing.tu-bs.de> writes:

> As stated in my previous mail the output of procedure-source is not the
> full scheme code that is necessary to re-memoize the code:
>   guile> (define foo (let ((bar 1)) (lambda () bar)))
>   guile> (procedure-source foo)
>   --> (lambda () bar)
> That is, trying to re-translate this output is not as clean a separation
> as you claim.  You depend on the fact that during execution there is
> sufficient information available to reliably re-compile any code such that
> it fits into the rest of the compiled code around it.  That is a strong
> assumption and it may prove to be expensive to guarantee it.

Could you explain this to me?  Why is it such a strong assumption?

My obvious counter example is:

(define foo2 (local-eval (procedure-source foo) (procedure-environment foo)))
(foo2) --> 1

[There is a need here to treat the first `lambda' specially, but
 that is a trivial problem.]

Which do you think puts the heavier constraint, procedure-source or
procedure-environment?  Note that there is no assumption needed that
procedure-source returns the verbatim source expression from which the
closure was created.  And an environment is simply a mapping of
variables to bindings.

Note that there is nothing "around" the method into which it should
fit.  The method has it's own closed environment.
  
> I had not actually planned to work on goops itself, and I will try
> to avoid doing so if possible - my resources are limited.  If we can
> agree on one way to go, I would prefer if I would not have to modify
> goops myself.

That is fine.  Can you share your current source?

> To get around the current problems (which are only relevant in my private
> copy of guile - none of my changes are yet submitted) I have suggested to
> replace one kind of dependency between goops and the evaluator
> (compile-method relying on the possibility to re-compile the output of
> procedure-source) by another one (compile-method working directly on the
> memoized code).  You are suggesting a different way, namely to add an
> interface to allow memoization of re-compiled scheme code and re-insert
> this code into the rest of the memoized code, which is just another form
> of dependency.  My preferred long-term solution, to have optimization
> functions work on a post-syntax-expansion-pre-memoization intermediate
> format also gets rid of some dependencies and adds a dependency on the
> intermediate format.  Dependencies are acceptable if they are between
> parts of the code which are stable, which I hope the future and yet to be
> defined intermediate format to be.

Well, I may be dense, but

* You have still failed to show to me why the second alternative
  (working on Scheme source) introduces any kind of dependencies (given
  that we fix the "hole").  I'd really want you to try to convince me
  about that.

* I like your suggestion for long-term solution.  And,

* It's perfectly fine with me to re-implement compile-method in C,
  working on the memoized representation.

Best regards,
Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel