From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Mikael Djurfeldt Newsgroups: gmane.lisp.guile.devel Subject: Re: goops and memoization Date: Wed, 04 Dec 2002 02:53:32 +0100 Sender: guile-devel-admin@gnu.org Message-ID: References: Reply-To: djurfeldt@nada.kth.se NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1038966952 32522 80.91.224.249 (4 Dec 2002 01:55:52 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Wed, 4 Dec 2002 01:55:52 +0000 (UTC) Cc: guile-devel@gnu.org, Neil Jerram Return-path: Original-Received: from monty-python.gnu.org ([199.232.76.173]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 18JOlQ-0008S5-01 for ; Wed, 04 Dec 2002 02:55:48 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 18JOme-0003vG-00; Tue, 03 Dec 2002 20:57:04 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 18JOmX-0003pS-00 for guile-devel@gnu.org; Tue, 03 Dec 2002 20:56:57 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 18JOmV-0003os-00 for guile-devel@gnu.org; Tue, 03 Dec 2002 20:56:57 -0500 Original-Received: from kvast.blakulla.net ([213.212.20.77]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 18JOmU-0003ml-00 for guile-devel@gnu.org; Tue, 03 Dec 2002 20:56:55 -0500 Original-Received: from barbara.blakulla.net ([213.212.21.238] helo=linnaeus) by kvast.blakulla.net with esmtp (Exim 3.36 #1 (Debian)) id 18JOjF-000386-00; Wed, 04 Dec 2002 02:53:33 +0100 Original-Received: from mdj by linnaeus with local (Exim 3.36 #1 (Debian)) id 18JOjE-0002QF-00; Wed, 04 Dec 2002 02:53:32 +0100 Original-To: Dirk Herrmann Original-Cc: djurfeldt@nada.kth.se In-Reply-To: (Dirk Herrmann's message of "Tue, 3 Dec 2002 21:36:57 +0100 (CET)") User-Agent: Gnus/5.090008 (Oort Gnus v0.08) Emacs/21.2 (i386-pc-linux-gnu) Errors-To: guile-devel-admin@gnu.org X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.0.11 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Developers list for Guile, the GNU extensibility library List-Unsubscribe: , List-Archive: Xref: main.gmane.org gmane.lisp.guile.devel:1782 X-Report-Spam: http://spam.gmane.org/gmane.lisp.guile.devel:1782 Dirk Herrmann writes: > Certainly this could be done as a temporary solution, but it is still not > a clean approach, since it still has dependencies on a lot of evaluator > internals. Even if my current implementation of the memoizer is closely > related to the old way the evaluator works, it may make sense to change > this in the future. That is, the representation of environment frames > during memoization and during execution may differ for good reasons. Certainly. As I've "confessed" in another letter, the assumptions on the environment representation is a dirty hack. One way of handling this is to use an abstraction such as "lookup": eval_environment_lookup (SYMBOL, ENV). [Excellent suggestions for using different env representation during memoization deleted.] > I disagree that the result of procedure-source is an excellent > representation of the logic of the method. procedure-source does not > provide the context of the lambda expression: > guile> (define foo (let ((bar 1)) (lambda () bar))) > guile> (procedure-source foo) > --> (lambda () bar) > That is, the source of the procedure foo does not provide all that is > needed to understand the logic of the method. This is a principal problem > of using procedure-source. In order to compile the output of > procedure-source correctly, you need more than the result of > procedure-source, namely something like procedure-environment-info. Well, I thought that was implicit in my argument. compile-method indeed uses the pair procedure-source, procedure-environment. Of course I never meant that the source without its environment is a valid representation of the logic. I hope the actual behavior of goops methods proves that. But I maintain that the pair of source and environment provides 100% of the semantics. > I have suggested this as a temporary fix, but not for performance > reasons: Goops does already interfere at some places with the > evaluator and the representation of the memoized code: First, there > are three memoizer codes that are specifically added for goops. > Second, the method dispatch code is copied verbatim into the > evaluator. Third, during execution of the code dispatch goops > modifies the memoized code for storing hash tables for faster > function dispatch. (...the reason being performance (measured in benchmarks, BTW), but I think you make it sound worse than it is. Two of the codes are very simple (object slot access). The dispatch form does contain dependencies on the evaluator for the evaluation of operand forms, but the rest is a section of self-contained code operating on a data structure which is entirely "owned" and handled by goops. It's not that a lot would need adaptation if the evaluator changes.) > A cleaner approach to optimization would be to allow compiler > optimizations in an earlier stage. Personally I'd prefer a solution with > five phases: 1) reading 2) macro expansion (scheme -> intermediate > format) 3) optimization (intermediate format -> intermediate format) 4) > memoization (intermediate format -> memoized code) 5) execution. For the > intermediate format I think of some very close to scheme format using > keywords like #:if and such (see new-model.text) as Marius has suggested. > This ought to be very stable. The memoized code, in contrast could be > somewhat less stable. Goops would add its optimizations functions to step > 3. Well, this all sounds good. But note that the use of procedure-source/environment in no way put serious constraints on the steps above. For example, steps 1, 2 and 4 could be performed when constructing the closure. Then an unmemoization step (U) followed by steps 3 and 4 would lead to 5. Basically, the inserted extra 4 and U are the only differences. There are a couple of constraints which I should mention here: 1. The MOP currently specifies that a method is created from a list of specializers and a closure by a call to `make'. The main reason for using a closure is that it is a nice way of capturing the lexical environment without the need to use a non-R5RS form and introducing an explicit representation of the environment. Another reason is that this pattern was used in Kiczales tiny-clos MOP---reflexive code which describes itself beautifully. Even though goops has dark corners "under the hood", the goops MOP provides a conceptually simple model for its semantics. The MOP also allows us to specialize `make' to a subclass of . If we try to get around the problem of capturing the environment by limiting ourselves to the special form `method', we loose this aspect of the MOP. So, the only alternatives I see are either a closure or passing the source expression together with the result of a call to (the-environment). 2. Goops method optimizations should not be done in advance. They are supposed to be performed at the exact instance when a generic function is called with a particular combination of argument types for the first time. That is of course possible to reconcile with the steps above, although steps 3-4 will be folded into the execution. > One could think of the possibility to add hooks for adding arbitrary > optimization functions, but this is just an idea. That is an exciting idea. It would be wonderful to be able to specify code re-writing rules such as: (map foo (map bar ls)) --> (map (lambda (x) (foo (bar x)))) > The reason why goops does not work with my changes is that goops > relies not only on the working of procedure-source Which should be of no problem to you unless you want to remove it. If you want that, there are other reasons than we discuss here for keeping it. > but also on the way environment frames are stored Yes. > and on the way the evaluator works. Not sure what you mean here. > As long as you don't have a perfect system, you have to build upon > the things that are available to you. Yes, unfortunately so. > With "interactions" I meant holes in the abstraction barrier, where > code relies on the behaviour of unmemoization It does in no way depend on the behaviour of unmemoization, unless the unmemoized source is semantically something different. The unmemoized source + the lexical environment is sufficient for full semantics. Sorry for repeating this statement, but it is important that you prove me wrong if I am wrong. > the internal representation of environments Yes, this is a hole. > It becomes more and more clear to my why so many previous approaches to > work on the evaluator have failed: The dependencies between the evaluator > and other parts of guile make it necessary to understand and work on quite > a lot of different parts of guile. You're absolutely right. It is very important to work towards good modularization, especially in this kind of collaborative project. Don't think I'm not trying... Although for many years there seemed to be higher priorities than taking on the monolithic SCM evaluator. Best regards, Mikael _______________________________________________ Guile-devel mailing list Guile-devel@gnu.org http://mail.gnu.org/mailman/listinfo/guile-devel