Re: summary: lilypond, lambda, and local-eval

[Andy Wingo <wingo@pobox.com>]

On Sun 18 Dec 2011 08:11, Mark H Weaver <mhw@netris.org> writes:

> So, it turns out that the best place to transform (the-environment) into
> standard scheme code is in the macro expander itself.

Are you certain that it is desirable to transform it into standard
Scheme code?

The statement that the-environment acts "as if it were the expression,
(case-lambda ....)" doesn't mean that you have to implement it that
way.

> Indeed, only the macro expander has enough information to generate an
> optimal list of "reachable lexicals", i.e. lexical variables that are
> accessible using normal symbols (as opposed to syntax objects) [more
> on this below].

Are you certain that you want to restrict the set of identifiers?

To me it sounds like an optimization, perhaps premature.

What do you think about having a <the-environment> tree-il form have a
field for the names and a field for the gensyms of captured lexicals?

> This is great news, because it means that `the-environment' will no
> longer require its own tree-il type, and the compiler will only see the
> standard scheme code that it expands into.
>
> However, we will need a robust way to either (A) specify, (B) discover,
> or (C) predict the order of the captured lexicals in a closure.

We could compile `the-environment' so that at runtime it yields a record
containing a vector of syntax objects and a vector of corresponding
variable objects.  (When the compiler boxes a variable, it does so in a
variable object, as from make-variable.)  Then you don't introduce
cross-cutting assumptions to the compiler and runtime.

> I have yet to decide which option to take.  Suggestions welcome.

WDYT about mine? :)

> There's also another issue that has come to my attention.  If we must
> support arbitrary syntax-objects to be passed to `local-eval', in many
> (most?) cases this will greatly increase the number of lexicals that
> must be captured, and thus boxed, by (the-environment).

If a tree-il `the-environment' form takes a list of names and gensyms,
then we can provide the possibility in the future to limit the set of
captured bindings.

> So, I'm thinking that (the-environment) should only capture lexical
> variables that are reachable using normal symbols.

I think I disagree here.  It is strictly less useful to capture a subset
of bindings, and it would only be done for efficiency, and it probably
doesn't matter.

So yeah, I guess my arguments here depend on a tree-il the-environment
form.  I wonder if that is the right thing, though; maybe there is a
lower-level concept at work.  The only thing that you need that tree-il
doesn't give you right now is the ability to declare a variable as
boxed, and to capture its identity.

Maybe what we need is a <lexical-capture> form that evaluates to the
variable corresponding to a bound lexical.  Then `the-environment' could
expand out to

   (make-struct/no-tail <lexical-environment>
                        '(name ...)
                        (list (capture-lexical name) ...))

You would still need support from the expander to get the set of
currently-bound names, but maybe that is a new primitive that we could
add.

Could we do it all with two new low-level primitives?  And then, could
we actually put `the-environment', environment accessors, and everything
else into a module?

Andy
-- 
http://wingolog.org/