From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.devel Subject: Re: Anything better for delayed lexical evaluation than (lambda () ...)? Date: Wed, 14 Dec 2011 20:14:54 +0100 Message-ID: <8739cn7yoh.fsf@fencepost.gnu.org> References: <87liqtpsl9.fsf@fencepost.gnu.org> <87mxaxihnw.fsf@pobox.com> <87obvclu92.fsf@fencepost.gnu.org> <87aa6wbp0w.fsf@pobox.com> <87fwgolgm5.fsf@fencepost.gnu.org> <8762hkbkwi.fsf@pobox.com> <87borclcem.fsf@fencepost.gnu.org> <87zkewa2vy.fsf@pobox.com> <87zkewjvyz.fsf@fencepost.gnu.org> <87vcpka13n.fsf@pobox.com> <87zkewnzy7.fsf@netris.org> <87r5089ui3.fsf@pobox.com> <87r508nv0o.fsf@netris.org> <87fwgondme.fsf@netris.org> <87borboalb.fsf@netris.org> <87liqf5uty.fsf@pobox.com> <877h1zbg7s.fsf@fencepost.gnu.org> <8739cn5la2.fsf@pobox.com> <87r507m5dg.fsf@netris.org> <87k45zm21r.fsf@netris.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1323890109 30152 80.91.229.12 (14 Dec 2011 19:15:09 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 14 Dec 2011 19:15:09 +0000 (UTC) Cc: Andy Wingo , guile-devel@gnu.org To: Mark H Weaver Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed Dec 14 20:15:04 2011 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RauIC-0004Bb-Cp for guile-devel@m.gmane.org; Wed, 14 Dec 2011 20:15:04 +0100 Original-Received: from localhost ([::1]:52233 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RauIB-0002LK-FP for guile-devel@m.gmane.org; Wed, 14 Dec 2011 14:15:03 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:50879) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RauI7-0002KF-5H for guile-devel@gnu.org; Wed, 14 Dec 2011 14:15:00 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RauI5-0005MR-3C for guile-devel@gnu.org; Wed, 14 Dec 2011 14:14:59 -0500 Original-Received: from fencepost.gnu.org ([140.186.70.10]:55208) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RauI4-0005MN-MD for guile-devel@gnu.org; Wed, 14 Dec 2011 14:14:56 -0500 Original-Received: from localhost ([127.0.0.1]:36895 helo=lola) by fencepost.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RauI3-0001t4-OA; Wed, 14 Dec 2011 14:14:56 -0500 Original-Received: by lola (Postfix, from userid 1000) id 00644E5092; Wed, 14 Dec 2011 20:14:54 +0100 (CET) In-Reply-To: <87k45zm21r.fsf@netris.org> (Mark H. Weaver's message of "Wed, 14 Dec 2011 13:38:24 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.92 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.10 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:13101 Archived-At: Mark H Weaver writes: > Replying to myself: > >> Andy Wingo writes: >>> It is my opinion -- and I could be wrong here, either by >>> misunderstanding (again!) the issues, or for whatever reason -- that >>> closures are the best solution to the #{#} problem. >> >> I would love to use closures for this, but they simply are not powerful >> enough to implement `local-eval' properly. >> >> First of all, closures cannot capture the syntactic environment, so >> things like (let-syntax ((foo ...)) (the-environment)) would not work. > > This statement (and the rest of my previous email), may have been based > on a mistaken impression of your "closure" proposal. I assumed that you > meant that (the-environment) should essentially implemented as a macro > that expands into a (lambda () ...) that would handle every possible > lexical variable access or mutation. > > However, based on the IRC logs, now I think you may have meant that > Lilypond should _scan_ the Lily code between #{ and #} looking for > embedded Scheme code, _before_ evaluating anything. > > If this could be done reliably, it would certainly make our lives > easier. Lilypond could create the entire Scheme expression with all the > embedded bits of Scheme as closures (not general purpose ones, but with > the exact right code within). We could compile the resulting code, and > life would be good. To be fair: this is what we currently do, and we only actually call those lambdas that end up actually being recognized by the grammar. So as long as (primitive-eval `(lambda () ,(read))) is guaranteed to not ever choke, the potential for error is limited. And to be fair: we already need to use the Scheme reader for skipping stuff behind # and $ or it would be hard to reliably detect the matching #} for the #{ construct when it occurs nested within Scheme again. It took a number of iterations to make all this robust and dependable, and just recently a Lilypond comment of the form % (this can be either #UP or #DOWN) caused trouble since the Scheme reader not just read #DOWN but also ) and then unread it again, making it end up at the start of the _next_ embedded expression for which the Scheme reader was asked for its opinion. > I don't know enough about Lilypond's language to know whether it is > feasible to do this robustly. So-so, but then we don't get around _scanning_ (and thus reading) the potential sexps in a dry run anyway with the current design. This would be different if Lilypond employed a different delimiter strategy. > We have already established that Lily cannot be parsed without runtime > information. Furthermore, their language needs to use lexical > tie-ins, which means that the lexer needs contextual information from > the parser. Therefore, they cannot even produce a stream of lexical > tokens prior to execution. > > So, with no help from either the parser or scanner, it sounds like > they'd be stuck with something like the usual Emacs strategy of > scanning source code with regexps to do syntax highlighting: it works > in practice most of the time, but it will fail sometimes for unusual > inputs. That's fine for a syntax highlighter, but a language > implementation should be more robust. Because of the delimiting issue, there is actually not much of a gain to be expected. Results would improve in cases like #{ % A Lilypond comment before a Scheme element #( #y %) and the comment ends again #} In this case, the current behavior will be that #y has no (lambda ()...) associated with it, and consequently evaluation will be done without a lexical environment, and fail. The old version using local-eval would have worked, but it would still have required the sexp starting at #( to end somewhere before #}. > In summary, it makes my head hurt to even think about how I would > implement and maintain Lilypond on Guile 2.0 using this strategy. I posted a link to the working code already. It made my head hurt. > It shouldn't be this difficult. This case with Lilypond has convinced > me that `the-environment' and `local-eval' can be very useful, and > sometimes they cannot be easily replaced with our other facilities. I think that it is a good thought experiment to assume Lilypond _could_ be properly compiled. To get lexical closure, we would have to bounce _everything_ around in uncompiled or macro form until the top-level form got assembled. For debugging and error messages, that would be inconvenient. Again, to be fair: the current code has to associate file names and line numbers with the string in order to get useful error messages. So it took work getting errors to throw where they belong. So at its current state, Lilypond itself does not much more than lukewarmly ask for local-eval. But for the sake of GUILE, I would consider it a mistake to lose sight of it. It is a powerful tool in the language embedding and self-awareness department. -- David Kastrup