Re: Anything better for delayed lexical evaluation than (lambda () ...)?

[Mark H Weaver <mhw@netris.org>]

David Kastrup <dak@gnu.org> writes:
>> In general, the _right_ way to build a custom extension language using
>> Guile 2 is to write a compiler that converts your language into one of
>> the other languages that Guile 2 supports.
>
> Lilypond is not Scheme.  It has syntax ambiguities that are resolved by
> lexical tie-ins and thus depend on the context.  You can't easily
> compile it in advance.

Lexical tie-ins require the use of a context-sensitive parser, but how
does it prevent compilation in advance?  Guile 2 places no constraints
whatsoever on the parser used to compile your language to Guile.  You
could use the exact same Bison parser you are currently using, but with
different actions.

>> If there's something about Lilypond's language that you believe would
>> make compilation impractical, let's talk about it.
>
> Its syntax and semantics.
>
> <URL:http://git.savannah.gnu.org/cgit/lilypond.git/tree/lily/parser.yy>
>
> If I call a function with an optional argument of type integer? before
> an argument of type ly:music? and I encounter #x, then the value of x
> decides whether this argument will be used as the optional argument or
> as the following argument.  The rest of the parsing has to follow.

I don't see a serious problem here.  In general, anything that can't be
done at compile time can be postponed to runtime easily enough.  To
address the specific example you give above, the compiled Lilypond
procedure could look something like this:

(define (myproc . args)
  (extract-lyargs args `((#:optional ,integer?) (#:required ,ly:music?))
    (lambda (x music)
      body ...)))

where `extract-lyargs' is a procedure (part of the runtime environment)
that takes the list of arguments and a formal-parameter specification,
and does the runtime tests needed to decide how the arguments should be
put into `x' and `music'.

> And you are _totally_ putting the cart before the horse here.  Lilypond
> is not supposed to be an extension language for Guile, but Guile is
> supposed to be an extension language for Lilypond.  The acronym Guile
> stands for "GNU's Ubiquitous Intelligent Language for Extension".  You
> are losing sight of what Guile is supposed to be.

I don't know about that.  You seem to imply that Lilypond's use of Guile
is very typical, and that other programs that use Guile for extension
will run into similar difficulties, but as far as I can tell Lilypond is
quite unique here.

Typically, an application using libguile (or any other language library)
allows the library to handle all aspects of parsing and running the
supported extension language(s).  In Guile's case, the idea is that
whenever a new language is added to Guile, applications using libguile
can automatically make use of those new languages.

You are using Guile in a very unusual way.  You have constructed a
hybrid language of both Scheme and Lilypond, where each can be nested
within the other (so far so good), but -- and here's the kicker -- you
apparently want to implement this hybrid language using two separate
interpreters maintained by two separate groups that are each able to run
code within lexical environments established by the other one.

This is a fundamentally bad idea, because this structure makes it
impossible for either of these language implementations to evolve in any
significant way.  It forces them both to remain simple interpreters.

> You are working from the premise that Guile should govern the
> architecture of the system it is supposed to be extending.

I can understand why it appears that way to you, but this is only
because you have built a system on Guile that places unreasonable
constraints upon the internal workings of Guile.

Please try to look at it from our perspective, and also from the
perspective of other programs that use Guile in a more typical way.
Most users of Guile 2 benefit from the architectural and efficiency
improvements, and are not harmed by them.

We are not trying to impose any particular architecture on your system,
only on the way the language implementation itself works.  Is it really
so unreasonable that the language implementation should be under Guile's
control?  Is this really a betrayal of the original vision of what Guile
is "supposed to be", as you wrote above?  If you think so, can you
please back this up with some references?

     Mark