bug#10132: Help lilypond interleave scheme and lilypond code in guile 2.x

[David Kastrup <dak@gnu.org>]

Andy Wingo <wingo@pobox.com> writes:

> I am going to be away from the machine for the weekend, but before I
> headed out, I just wanted to put out one idea:
>
> On Fri 25 Nov 2011 14:35, David Kastrup <dak@gnu.org> writes:
>
>>> What do you use to parse the lilypond code?  What does it parse to?
>>
>> Classical Bison/Flex parser/scanner.  There is no "what does it parse
>> to" since the Bison rules execute the actions on the fly: it is a
>> classical interpreter.  With a number of lexical and semantical tie-ins,
>> it would be non-trivial to actually create an intermediate
>> representation.
>
> Have you considered using silex or some other tokenizer in scheme,
> combined with the lalr parser from (system base lalr)?  See "LALR(1)
> Parsing" in the manual for Guile 2.0.

Lilypond is not yet capable of running under Guile 2.0 (and needs to
stay 1.8-compatible for a considerable time span), so it makes no sense
to think about using 2.0 features for core parts of it.

The core of Lilypond is quite C++-centric.  Parsing takes a
non-negligible amount of runtime already.  Switching to a different
system likely to be slower and less directly interfacing with C++ is not
going to be on the agenda anytime soon, and I am less than convinced
that this total change of playground would actually make a qualitative
difference regarding the implementation of this particular scoping task.

>> The procedure-environment approach was elegant and minimally complex.
>> The question is how feasible it is for the Guile compiler to capture
>> an environment in a form that can be used even after compilation.
>> Like taking the address of a variable in C, the export of such an
>> environment interferes with a number of static optimizations.  For
>> our particular application, readonly access to the symbols in the
>> environment should be quite sufficient, but of course I can't vouch
>> for other potential uses.
>
> If this is the answer, then we can figure out a way to implement it in
> Guile 2.0.x as well.

"Readonly access" in this context does not mean "separate copies" but
rather "access to the original variables without the need to change them
via this access path".  If such a closure variable is changed by a
function "properly" compiled in the lexical closure (rather than
artificially relying on process-environment), we would likely still need
to see this change.

> But if you are amenable to it, implementing the parser in Scheme would
> be another attractive option -- though, it would be a change, and that
> has costs.

That's not on the table right now in my opinion, and I actually don't
see that it would help to a relevant degree since, as I said, lexical
and semantic tie-ins require the parser to work _progressively_ instead
of being able to compile an intermediate Scheme representation, and
without an intermediate representation, I don't see how we could exploit
the "natural" implicit implementation of closures in the Scheme
compiler.

-- 
David Kastrup