From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: Re: local-eval on syntax-local-binding, bound-identifiers Date: Mon, 16 Jan 2012 16:17:13 +0100 Message-ID: <87ipkbisna.fsf@pobox.com> References: <87sjjg7gqm.fsf@pobox.com> <87zkdo2ukg.fsf@netris.org> <87r4yzj4gu.fsf@pobox.com> <8762gb3hfv.fsf@netris.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1326727152 23154 80.91.229.12 (16 Jan 2012 15:19:12 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 16 Jan 2012 15:19:12 +0000 (UTC) Cc: guile-devel To: Mark H Weaver Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Jan 16 16:19:06 2012 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RmoJV-00052t-SI for guile-devel@m.gmane.org; Mon, 16 Jan 2012 16:17:38 +0100 Original-Received: from localhost ([::1]:50898 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RmoJU-00018L-Vi for guile-devel@m.gmane.org; Mon, 16 Jan 2012 10:17:36 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:54424) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RmoJI-00016c-T4 for guile-devel@gnu.org; Mon, 16 Jan 2012 10:17:34 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RmoJD-00089T-Dt for guile-devel@gnu.org; Mon, 16 Jan 2012 10:17:24 -0500 Original-Received: from a-pb-sasl-sd.pobox.com ([74.115.168.62]:43407 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RmoJC-00088y-Lk for guile-devel@gnu.org; Mon, 16 Jan 2012 10:17:19 -0500 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTP id 0D69E6A3C; Mon, 16 Jan 2012 10:17:18 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=nKAHUxf/1tSA6mBykscfOGrLqxo=; b=M4Vn67 B/9ePPY6p/1xUnHtfaFCUIxqFI9EeI2hgeSGP0NDQRPpto/8orBKJr3liLfCxe9c r90CZABNkCqr1rU6aldfHHQ/riw+ScxP17h0N50cTI8PAef2KyeKs3KdREk5ET/4 Gc5y1NIzagcXi+MF/Gz3Uf1J/oQTruC+l1lGM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=lwlHJ/TTXuDHlYU2ff/xckPIpolu5EqC 5tzid64JUqddtuUwQBFmGfwa2qTfoOhP9802pppRqYi+OY+Z5Uhih+10p3xEhxyF xOfjNPPZ6q5hrRNxFO2ADzXlCGBDxGtSUILvSX+2lW9xUOLhY/Tn1Mr5Svvet1Mq W2eJqSTZTkg= Original-Received: from a-pb-sasl-sd.pobox.com (unknown [127.0.0.1]) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTP id 05AE66A3B; Mon, 16 Jan 2012 10:17:18 -0500 (EST) Original-Received: from badger (unknown [90.164.198.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTPSA id 30AA96A3A; Mon, 16 Jan 2012 10:17:17 -0500 (EST) In-Reply-To: <8762gb3hfv.fsf@netris.org> (Mark H. Weaver's message of "Mon, 16 Jan 2012 08:28:20 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) X-Pobox-Relay-ID: 2D424D2A-4055-11E1-813D-65B1DE995924-02397024!a-pb-sasl-sd.pobox.com X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-Received-From: 74.115.168.62 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:13543 Archived-At: Hi Mark, Thanks for the comments :) On Mon 16 Jan 2012 14:28, Mark H Weaver writes: > Andy Wingo writes: >>>> + (cons (wrap (car symnames) >>>> + (anti-mark (make-wrap (car marks) subst)) >>> >>> ***** Why are you adding anti-marks here? >> >> As the changelog noted (and a comment should have noted ;), the >> identifiers are anti-marked so that syntax transformers can introduce >> them, as-is. >> >> The purpose of this procedure is to get a list of identifiers, and to >> capture some subset of them. It will do so by introducing references to >> them in the expansion of some macro. However they are not introduced >> identifiers: they come from the code itself. They are input the macro, >> and as such need an anti-mark. >> >> The anti-mark will be stripped from the expansion when the transformer >> that called `bound-identifiers' returns. > > Does this mean that `bound-identifiers' will not function properly when > used outside of a macro? What about if it's used within a macro that > was generated by another macro (or things of that nature)? Are there > cases where you might need to strip more than one anti-mark? Well, bound-identifiers is a procedure, so if you are using it outside the dynamic extent of a transformer procedure, that means that you have a syntax object that you squirreled away from somewhere, so already we're in somewhat uncharted territory. Once you introduce it somewhere else (in another macro), the anti-mark would be stripped though. The only other interface to the expander is `macroexpand', where we should probably also strip the anti-mark, if we are allowing anti-marks to escape. Macro-generating macros should be fine, here. `expand-macro' is iterative, not recursive, so you don't need to strip anti-marks twice. I agree that this anti-mark has a bad smell, but the idea of a `bound-identifiers' procedure or form sounds like a good idea, so if you have any suggestions for improvement here, they are most welcome. >> The forgeable gensym issue is something we have in Guile, more >> generally, that needs a broader solution. > > Ah, good point! Macros already serialize syntax-objects to disk. > psyntax wraps are already part of our ABI, so nothing new there. The issue is not in the uniqueness of gensyms associated with lexical variables, by the way: it's more in the uniqueness of the labels and the marks. See psyntax-pp.scm, for example, or commit fd5985271fee3bcb6a290b6ad10525980a97ef8d. > [W]ith `local-eval', it seems to me quite plausible that gensym > collisions might occur. Suppose in one Guile session you compile a > procedure (foo) that uses (the-environment), and then in another Guile > session, you call (foo) and then `local-eval' with the environment > returned by (foo). Now the wrapper procedure splices together syntax > objects from two different Guile sessions into a single top-level > form, where (unlike in the macro case) all of these syntax objects are > lexicals, and thus depend on the gensyms and the labels. > > See how this is a problem now where it wasn't before? > Or am I missing something? To be perfectly honest, this stuff is very confusing to me, but I think I can see how this can happen, yes. I do think that it's important to fix this bug at some point, but IMO it is not a blocker for local-eval, much less 2.0.4. >> (Note though that (eval X e) does indeed evaluate X in tail >> position.) > > Looks to me like `eval' is initially bound to the C function scm_eval. > Is it later rebound to a Scheme procedure? If so, where? Ah, you're right, I was thinking of primitive-eval. Our scm_eval is buggy wrt the spec (r5rs at least, which says): Certain built-in procedures are also required to perform tail calls. The first argument passed to `apply' and to `call-with-current-continuation', and the second argument passed to `call-with-values', must be called via a tail call. Similarly, `eval' must evaluate its argument as if it were in tail position within the `eval' procedure. So we need to override eval to a Scheme procedure. >> `the-environment' is not fundamental: it can be implemented in terms of >> simpler primitives. > > The same can be said of `lambda' or `syntax-case', but that's not the > appropriate way to choose primitives in a language. John Shutt would argue with you here, but I digress :) (http://web.cs.wpi.edu/~jshutt/kernel.html) > BTW, did you see my most recent model for thinking about `local-eval'? > > (the-environment) expands to (list (lambda () ) ...), with one > element for every possible expression: a countably infinite list that > could be built lazily. `local-eval' simply chooses the appropriate > procedure from the list and calls it. A poor implementation strategy, > but the semantic meaning is quite clear, no? It sounds clear, but does it have any explanatory power? It sounds like it could apply just as well to any other computation... > Regardless, I hope it's safe to say that `the-environment' and > `local-eval' are far closer to the spirit of primitives one finds in the > Scheme standards than are the ones you're proposing. Do you not > agree? Yes, I grant you that the meanings of the-environment and local-eval seem more clear than those of their constituent parts (boxes, identifier syntax, etc). I'm certainly not arguing that lilypond should be using bound-identifiers somehow! But I'm not sure that this says something about what psyntax should have in it (especially if it is to have syntax-local-value or something like it anyway). >> Implementing the-environment in psyntax also poses implementation >> layering problems: it uses wrap hacks to expand out to private forms >> from ice-9 local-eval. > > I don't see how that's a "hack" at all. That's exactly what wraps are > for: to specify the expander's lexical environment in which to look up > the symbol. I'm using them in a completely straightforward way to > create syntax-objects for identifiers in ice-9/local-eval. This within > psyntax, where wraps are created all over the place. Creating wraps is not the hack. It's creating wraps that are scoped in another specific module. With the-environment in psyntax, psyntax depends on (ice-9 local-eval). With the-environment in (ice-9 local-eval), psyntax does not depend on (ice-9 local-eval). I *very* much prefer a psyntax that does not depend on other modules. > A `syntax-rules' macro can be _perfectly_ represented as a > syntax-object of the `syntax-rules' form itself. Therefore, I would > like to represent them this way in the expansion of (the-environment), > so that `syntax-rules' macros _can_ be serialized to disk. All this for local-eval? And you don't get procedural macros? And what would it do to psyntax? But OK. I still think you should be able to do this in a module! Ultimately, Guile should be a great platform for language research and experimentation. Like Racket is, in many ways. In order to support their multitudinous syntactic rackets, Racket gives the macro writer lots of power -- lots of accessors, lots of insight into what the expander is up to. We should do the same, where it makes sense to allow access to that information. We shouldn't restrict these sorts of experiments to the kinds of (grumpy, crotchety) people that maintain psyntax.scm ;-) > However, this requires re-expanding the bodies of the `syntax-rules' > forms, so now the environment structures _do_ matter. Ultimately, this > means that a set of mutually recursive local macros must be recreated in > `local-eval' as a `letrec-syntax'. > > Does that make sense? Yes I see where you're going here. IMO, we should be providing the tools to let you do these experiments in parallel, without delaying the release of other bugfixes. Regards, Andy -- http://wingolog.org/