From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.lisp.guile.bugs Subject: bug#10099: Null (begin) blocks - V2.0.3 reports error was OK in V2.0.2 Date: Fri, 25 Nov 2011 11:37:38 +0100 Message-ID: <87aa7kqyh9.fsf@fencepost.gnu.org> References: <4ECA89A5.5090202@hulin.org.uk> <871usznhvv.fsf@pobox.com> <4ECED81C.6030606@hulin.org.uk> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1322217526 19739 80.91.229.12 (25 Nov 2011 10:38:46 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 25 Nov 2011 10:38:46 +0000 (UTC) Cc: 10099@debbugs.gnu.org To: Ian Hulin Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Fri Nov 25 11:38:38 2011 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RTtAy-0004xd-0E for guile-bugs@m.gmane.org; Fri, 25 Nov 2011 11:38:36 +0100 Original-Received: from localhost ([::1]:36475 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RTtAx-00063g-Kn for guile-bugs@m.gmane.org; Fri, 25 Nov 2011 05:38:35 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:49064) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RTtAv-00063b-1I for bug-guile@gnu.org; Fri, 25 Nov 2011 05:38:34 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RTtAo-0005RL-Kd for bug-guile@gnu.org; Fri, 25 Nov 2011 05:38:32 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:36412) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RTtAo-0005RG-F1 for bug-guile@gnu.org; Fri, 25 Nov 2011 05:38:26 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.69) (envelope-from ) id 1RTtCM-0002XW-8L for bug-guile@gnu.org; Fri, 25 Nov 2011 05:40:02 -0500 X-Loop: help-debbugs@gnu.org In-Reply-To: <87d3cl13lg.fsf@pobox.com> Resent-From: David Kastrup Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-guile@gnu.org Resent-Date: Fri, 25 Nov 2011 10:40:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10099 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 10099-submit@debbugs.gnu.org id=B10099.13222175679711 (code B ref 10099); Fri, 25 Nov 2011 10:40:02 +0000 Original-Received: (at 10099) by debbugs.gnu.org; 25 Nov 2011 10:39:27 +0000 Original-Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RTtBn-0002WZ-De for submit@debbugs.gnu.org; Fri, 25 Nov 2011 05:39:27 -0500 Original-Received: from fencepost.gnu.org ([140.186.70.10]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RTtBi-0002WL-Kc for 10099@debbugs.gnu.org; Fri, 25 Nov 2011 05:39:26 -0500 Original-Received: from localhost ([127.0.0.1]:46126 helo=lola) by fencepost.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RTtA4-0003Ez-Lq; Fri, 25 Nov 2011 05:37:42 -0500 Original-Received: by lola (Postfix, from userid 1000) id DFAA5E050C; Fri, 25 Nov 2011 11:37:38 +0100 (CET) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.90 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Resent-Date: Fri, 25 Nov 2011 05:40:02 -0500 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:5944 Archived-At: Ian Hulin writes: > Hi Andy, > On 23/11/11 12:27, Andy Wingo wrote: >> Hi Ian, >> >> Did it really work in v2.0.2, I wonder? >> > Apparently not, see below for explanation. This mail addresses two different issues, mixing them in somewhat surprising ways. I'll try to split the issues again. >> On Tue 22 Nov 2011 00:02, Andy Wingo writes: >>> On Mon 21 Nov 2011 18:25, Ian Hulin writes: >>>> (define-public (void? x) (eq? x (begin))) This works in V1.8, >>>> and apparently used to work in 2.0.2 (no errors), >>> >>> Interesting. If it changed incompatibly in 2.0.x, that is a >>> Guile bug. Sorry about that! We'll fix it. >> >> I cannot reproduce this issue with 2.0.2. I suspect that this code >> has been this way since early 1.9 releases, and is a consequence of >> switching to the new syntax expander. >> > Looks like the change was already in 2.0.2. Another LilyPond developer > (David Kastrup, dak@gnu.org) noticed the advertised API and decided to > use it in part of an ongoing clean-up of the Lily parser. Meanwhile I > was working on the Guile V2 migration stuff and it broke. More concretely regarding "the advertised API": the Guilev1.8 documentation does neither mention `*unspecified*', nor `unspecified?'. On the C side, it documents SCM_UNSPECIFIED. On the Scheme side, the only available information is the documentation for begin: -- syntax: begin expr1 expr2 ... The expression(s) are evaluated in left-to-right order and the value of the last expression is returned as the value of the `begin'-expression. This expression type is used when the expressions before the last one are evaluated for their side effects. Guile also allows the expression `(begin)', a `begin' with no sub-expressions. Such an expression returns the `unspecified' value. So it is somewhat disingenuous to laugh off such uses of begin. At the very least you should add documentation for *unspecified* and unspecified? to the documentation of the 1.8 branch. That way, you at least can point people to documentation relevant for the migration. There _are_ a few problems even in Guilev1.8: v1.8 refuses to accept some forms that should likely be *unspecified*: guile> (lambda ()) Backtrace: In standard input: 1: 0* (lambda ()) standard input:1:1: In procedure memoization in expression (lambda ()): standard input:1:1: In file "standard input", line 0: Missing expression in (lambda ()). ABORT: (syntax-error) guile> ((lambda () (begin))) Backtrace: In standard input: 4: 0* [#] standard input:4:1: In procedure memoization in expression ((lambda () #)): standard input:4:1: Missing body expression in ((begin)). ABORT: (syntax-error) guile> ((lambda () (begin (begin)))) Backtrace: In standard input: 6: 0* [#] standard input:6:1: In procedure memoization in expression ((lambda () #)): standard input:6:1: Missing body expression in ((begin (begin))). ABORT: (syntax-error) guile> ((lambda () #f (begin))) guile> I have switched to using *unspecified* and unspecified? by now, but it might be worth ironing out the strange behaviors of (begin) nevertheless in particular where they conflict with the behavior to be expected from the documentation (preferably both in v1.8 as well as v2), in addition to mentioning the more straightforward commands. So much for that. The next quote is for a totally different issue, the availability of local environments and evaluation in them. Lilypond has an input syntax of its own, and it allows interspersing Scheme code. $ or # switches to the Scheme interpreter (for one sexp) when in Lilypond syntax, and #{ ... #} switches to Lilypond inside. We have constructs like spacingTweaks = #(define-music-function (parser location parameters) (list?) (_i "Set the system stretch, by reading the 'system-stretch property of the `parameters' assoc list.") #{ \overrideProperty #"Score.NonMusicalPaperColumn" #'line-break-system-details #(list (cons 'alignment-extra-space (cdr (assoc 'system-stretch parameters))) (cons 'system-Y-extent (cdr (assoc 'system-Y-extent parameters)))) #}) As you can see, the music function argument "parameters" is used in Scheme code embedded in Lilypond code. Previously, #{ (implemented in the Scheme reader) captured the enclosed material at read time as a string for interpretation (using the Scheme reader to skip over any #sexp or $sexp inside) as well as (procedure-environment (lambda () '())), then called the Lilypond parser at execution time, and the parser executed all embedded Scheme in the procedure-environment captured in the lexical scope of the define-music-function text. My _current_ implementation can't capture an execution environment, so I need to precompile a whole host of closures, and later correlate them with input source positions. This is somewhat error-prone since the full parsing of Lilypond might not agree with the cursory scan of the #{ ... #} body. Since the parser and lexer are context-dependent and can switch behavior according to variables at execution time, a more complete parse at compile time is not feasible. To give an example, the above #{ ... #} nowadays gets translated into (let* ((clone (ly:parser-clone parser (list (lambda () "Score.NonMusicalPaperColumn") (lambda () (quote line-break-system-details)) (lambda () (list (cons (quote alignment-extra-space) (cdr (assoc (quote system-stretch) parameters))) (cons (quote system-Y-extent) (cdr (assoc (quote system-Y-extent) parameters)))))))) (result (ly:parse-string-expression clone " \\overrideProperty #\"Score.NonMusicalPaperColumn\" #(quote line-break-system-details) #(list (cons (quote alignment-extra-space) (cdr (assoc (quote system-stretch) parameters))) (cons (quote system-Y-extent) (cdr (assoc (quote system-Y-extent) parameters)))) "))) (if (ly:parser-has-error? clone) (ly:parser-error parser (_ "error in #{ ... #}"))) result) Namely, all contained # expressions are precompiled into (list (lambda () "Score.NonMusicalPaperColumn") (lambda () (quote line-break-system-details)) (lambda () (list (cons (quote alignment-extra-space) (cdr (assoc (quote system-stretch) parameters))) (cons (quote system-Y-extent) (cdr (assoc (quote system-Y-extent) parameters)))))) Previously, all that was needed to record instead was (procedure-environment (lambda () '())) (again note the useless '() to bypass the issue of complaints about a missing body) and just use that for feeding local-eval inside of ly:parse-string-expression. This results in quite heavy storage requirements and will cause problem if the full parsing does not use the same # $ starting points as the preliminary scanning, like when # or $ are inside of Lilypond comments: they will get turned into lambdas at compile time, but not regarded at execution time. I have a patch in the queue for Lilypond that also records the starting positions in the string, and skips "obvious" constants, only wrapping symbols and pairs with a car different from 'quote into lambdas. That reduces the stored amounts of data and improves reliability, at the cost of considerable complexity. > I'm sure David will let me know if I've over-simplified stuff or got > things wrong here. > > However, it boils down to lousy timing for LilyPond finding a use for > this bit of the Guile API, which you guys had decided to withdraw > because no-one apparently had a use for it :-{ The use would be in implementing embedded lexical environments not exclusively written in Scheme. The thrust of Guilev2 to completely abandon support of interpreted and nested environments in favor of getting speedups for compiled applications casts doubt on its usefulness as an _extension_ language, and that was the principal aim distinguishing it from compiled Scheme variants like Chicken which, I would guess, have enough of a headstart compared to Guile that abandoning Guile's original goals and trying to compete with the established compilers on their home ground is not likely to increase the core audience of Guile. As I said: for this particular application, I have coded a rather inelegant and resource-grabbing workaround that really is not going to help performance since the intertwined Lilypond interpreter does not benefit from precompilation of mostly trivial lambda functions when the actual procedure-environment is unlikely to ever reference more than five variables. So for smooth integration into an interpretative environment, this is a definite step for the worse. It was perhaps a reasonably easy decision to make since the original functionality was heavily underused, likely also due to abysmally bad documentation that provides not much more than the motivation to hunt through source code and search the web for other people who actually managed to make something work. Please feel free to split this twofold mail into separate parts and answer in appropriate lists. I apologize for not having a clue what belongs where myself. -- David Kastrup