From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: summary: lilypond, lambda, and local-eval Date: Sun, 18 Dec 2011 02:11:41 -0500 Message-ID: <878vmaicb6.fsf@netris.org> References: <87r506uodd.fsf@pobox.com> <87pqfpj7e3.fsf@netris.org> <87aa6skika.fsf@netris.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1324192427 13424 80.91.229.12 (18 Dec 2011 07:13:47 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sun, 18 Dec 2011 07:13:47 +0000 (UTC) Cc: guile-devel To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Dec 18 08:13:43 2011 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RcAwI-0008FV-RI for guile-devel@m.gmane.org; Sun, 18 Dec 2011 08:13:43 +0100 Original-Received: from localhost ([::1]:32893 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RcAwH-0004PF-Hs for guile-devel@m.gmane.org; Sun, 18 Dec 2011 02:13:41 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:50272) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RcAwE-0004Nz-GJ for guile-devel@gnu.org; Sun, 18 Dec 2011 02:13:39 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RcAwA-00076S-Mi for guile-devel@gnu.org; Sun, 18 Dec 2011 02:13:38 -0500 Original-Received: from world.peace.net ([96.39.62.75]:55895) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RcAwA-0006x6-IO for guile-devel@gnu.org; Sun, 18 Dec 2011 02:13:34 -0500 Original-Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=yeeloong) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.69) (envelope-from ) id 1RcAvt-00053V-E1; Sun, 18 Dec 2011 02:13:17 -0500 In-Reply-To: <87aa6skika.fsf@netris.org> (Mark H. Weaver's message of "Fri, 16 Dec 2011 03:49:09 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.92 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 96.39.62.75 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:13151 Archived-At: I wrote: >> Most passes of the compiler will pretend that (the-environment) is >> replaced by tree-il code corresponding to the following standard scheme >> code: > > I should also mention that perhaps, instead of simply "pretending", it > might make sense to actually replace (the-environment) with the standard > scheme code I gave as early as possible, so that later passes will never > even see `the-environment' tree-il nodes. It need only be late enough > so that the list of visible lexical variable names is known at that > point. So, it turns out that the best place to transform (the-environment) into standard scheme code is in the macro expander itself. Indeed, only the macro expander has enough information to generate an optimal list of "reachable lexicals", i.e. lexical variables that are accessible using normal symbols (as opposed to syntax objects) [more on this below]. This is great news, because it means that `the-environment' will no longer require its own tree-il type, and the compiler will only see the standard scheme code that it expands into. However, we will need a robust way to either (A) specify, (B) discover, or (C) predict the order of the captured lexicals in a closure. Option A would be ideal, because it would allow `local-eval' to easily and robustly create a new closure that is "compatible" with the general-dispatcher closure created by (the-environment), using only standard compiler interfaces. By "compatible", I mean that their captured lexicals have the same names and are stored in the same order in the closure, so that code from the new closure can be spliced into the existing general-dispatcher closure. Option B would require `local-eval' to use a special interface to the compiler to compile new code that is compatible with the existing general-dispatcher closure. Option C is similar to option B but probably more fragile, because future changes to the compiler might invalidate the predictions and thus break `local-eval'. However, this fragility could perhaps be mitigated by prominent warnings in the comments of the code that determines closure slot order. I have yet to decide which option to take. Suggestions welcome. * * * * * There's also another issue that has come to my attention. If we must support arbitrary syntax-objects to be passed to `local-eval', in many (most?) cases this will greatly increase the number of lexicals that must be captured, and thus boxed, by (the-environment). Consider the following example: (let* ((t 42) (t (1+ t)) (t (1+ t)) (t (1+ t))) (or #f (the-environment)))) If syntax-objects are never passed to `local-eval', then only one lexical variable is reachable from (the-environment), namely the last `t' in the `let*', and therefore that's the only variable that must be boxed. However, if we must support arbitrary syntax-objects, then not only are all four of the above `t's reachable, but the hidden `t' from the expansion of the `or' is reachable as well! (or at least I see no way for the implementation of `the-environment' to prove that the user does not possess such syntax-objects, given the way psyntax is implemented). It would be a shame to force all of these shadowed and hidden variables to be boxed. If we can minimize the number of captured variables, we can reduce the performance impact of `the-environment'. So, I'm thinking that (the-environment) should only capture lexical variables that are reachable using normal symbols. I've already written code for psyntax that generates this list of reachable lexicals. This should suffice in the overwhelming majority of cases. If needed, we could easily support a variant: (the-full-environment) or (the-environment #:full), that captures _all_ lexicals reachable using arbitrary syntax-objects. What do you think? Mark