From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: summary: lilypond, lambda, and local-eval Date: Fri, 16 Dec 2011 02:35:48 -0500 Message-ID: <87pqfpj7e3.fsf@netris.org> References: <87r506uodd.fsf@pobox.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1324021056 2960 80.91.229.12 (16 Dec 2011 07:37:36 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 16 Dec 2011 07:37:36 +0000 (UTC) Cc: guile-devel To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Dec 16 08:37:32 2011 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RbSMG-0003bS-EG for guile-devel@m.gmane.org; Fri, 16 Dec 2011 08:37:32 +0100 Original-Received: from localhost ([::1]:42526 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RbSMF-0008Sk-Vk for guile-devel@m.gmane.org; Fri, 16 Dec 2011 02:37:31 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:37125) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RbSMC-0008Sc-Kd for guile-devel@gnu.org; Fri, 16 Dec 2011 02:37:30 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RbSMB-0007Pc-A4 for guile-devel@gnu.org; Fri, 16 Dec 2011 02:37:28 -0500 Original-Received: from world.peace.net ([96.39.62.75]:36307) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RbSMB-0007PW-46 for guile-devel@gnu.org; Fri, 16 Dec 2011 02:37:27 -0500 Original-Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=yeeloong) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.69) (envelope-from ) id 1RbSM4-0007hx-P5; Fri, 16 Dec 2011 02:37:21 -0500 In-Reply-To: <87r506uodd.fsf@pobox.com> (Andy Wingo's message of "Thu, 15 Dec 2011 11:21:18 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.92 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 96.39.62.75 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:13126 Archived-At: Hello all, Although it has not yet been decided whether `local-eval' will be accepted into Guile 2, I've decided to proceed with a proper implementation that is fully integrated into the compiler. I hope to demonstrate that this feature can be implemented easily without creating any significant maintenance burden. Here's an outline of my plan: How to compile (the-environment) ================================ Most passes of the compiler will pretend that (the-environment) is replaced by tree-il code corresponding to the following standard scheme code: (make-lexical-environment ;; a normal procedure (constructor) (case-lambda ;; general dispatcher to access or mutate any lexical (() #f) ((name-XXX) ;; name-XXX is a gensym (case name-XXX (() ) ;; one clause for each lexical ...)) ((name-XXX value-XXX) ;; name-XXX and value-XXX are gensyms (case name-XXX (() (set! value-XXX)) ;; one clause for each lexical ...))) (quote ) ;; simple list structure (quote ) ;; simple list structure ) ;; XXX not sure how best to represent this The is extracted from the tree-il node corresponding to (the-environment), created by the macro expander. I've already implemented this part in my previous patch. The would include a list of lexical variable names ( ...), which must exactly correspond to the closure slots of the `case-lambda', in order to implement `local-eval'. We _might_ also need to include some information about how those variables are stored, e.g. a flag telling whether they are boxed. (Usually they will all be boxed, but Note that general-purpose dispatcher will not actually be used by `local-eval'. Its purpose is primarily to force all of the visible lexicals to be boxed and included within the closure. They are also there to make any fancy optimizers do the right thing. KEY POINT: Since the general-purpose dispatcher would be sufficient to implement `local-eval' in standard scheme, it communicates exactly the right facts to the code analyzers, no matter how clever they are, and it does so without the analyzers having to know anything about these new primitives! How to implement `local-eval' ============================= When `local-eval' is called on a lexical environment that was created by compiled code, it will do the following: * Macroexpand the local expression within . * Compile the expanded expression within . (I'll explain how to do this below) * Make a copy of the closure from the lexical environment object, but replace its code (the dispatcher) with the newly compiled code. * Call the newly created closure. So, the trick is to make sure that the newly compiled code accesses the closure variables in exactly the same way as the general-purpose dispatcher. I haven't yet dug deeply enough into the compiler internals to know the best way to do this. Ideally I would create a new procedure to handle this simply and robustly, making use of . For now, I will describe a method that I suspect would do the right thing without any new compiler interfaces, though not as efficiently or robustly: Simply compile the same general-purpose dispatcher as before, except replace the #f (from the first case-lambda clause) with the expanded local expression: (lambda ( ...) ;; list of lexicals from (case-lambda (() ) ((name-XXX) ;; name-XXX is a gensym (case name-XXX (() ) ;; one clause for each lexical ...)) ((name-XXX value-XXX) ;; name-XXX and value-XXX are gensyms (case name-XXX (() (set! value-XXX)) ;; one clause for each lexical ...)))) and then extract the code from the inner `case-lambda'. As described above, this code would replace the code portion of the captured closure. NOTE: This assumes that case-lambda creates only one closure for all cases. I confess that I haven't yet verified this suspicion. If this is not true, one could replace the case-lambda with the equivalent using standard scheme. Again, I don't expect to actually use this hack. I expect to simply introduce a new internal compiler interface that essentially does the same thing, but in a less fragile way. However, I hope that these draft notes will give confidence that it is possible to implement `the-environment' and `local-eval' without adding any additional complexity to the compiler. No changes should be needed to the analysis or optimization passes, aside from adding some clauses to `record-case' for `the-environment' tree-il nodes, and a new procedure or two to handle `local-eval'. I intend to implement this soon. Comments and suggestions solicited. Best, Mark