From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: Re: request review & testing: syncase Date: Wed, 15 Apr 2009 17:23:56 +0200 Message-ID: References: <87ab6krkji.fsf@arudy.ossau.uklinux.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1239809235 9452 80.91.229.12 (15 Apr 2009 15:27:15 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 15 Apr 2009 15:27:15 +0000 (UTC) Cc: guile-devel To: Neil Jerram Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed Apr 15 17:28:34 2009 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1Lu72G-0006Fb-LO for guile-devel@m.gmane.org; Wed, 15 Apr 2009 17:28:25 +0200 Original-Received: from localhost ([127.0.0.1]:51236 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Lu70r-0000Yd-QB for guile-devel@m.gmane.org; Wed, 15 Apr 2009 11:26:57 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Lu6y8-0005Fn-ES for guile-devel@gnu.org; Wed, 15 Apr 2009 11:24:08 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Lu6y3-00058Z-EY for guile-devel@gnu.org; Wed, 15 Apr 2009 11:24:07 -0400 Original-Received: from [199.232.76.173] (port=33256 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Lu6y2-00058B-RH for guile-devel@gnu.org; Wed, 15 Apr 2009 11:24:02 -0400 Original-Received: from a-sasl-fastnet.sasl.smtp.pobox.com ([207.106.133.19]:47646 helo=sasl.smtp.pobox.com) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1Lu6y2-0002pN-Fz for guile-devel@gnu.org; Wed, 15 Apr 2009 11:24:02 -0400 Original-Received: from localhost.localdomain (unknown [127.0.0.1]) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTP id 72A19AA04D; Wed, 15 Apr 2009 11:24:01 -0400 (EDT) Original-Received: from unquote (unknown [82.123.180.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-sasl-fastnet.sasl.smtp.pobox.com (Postfix) with ESMTPSA id 81ED2AA04C; Wed, 15 Apr 2009 11:23:59 -0400 (EDT) In-Reply-To: <87ab6krkji.fsf@arudy.ossau.uklinux.net> (Neil Jerram's message of "Mon, 13 Apr 2009 10:05:05 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (gnu/linux) X-Pobox-Relay-ID: 728D3F68-29D1-11DE-8494-C121C5FC92D5-02397024!a-sasl-fastnet.pobox.com X-detected-operating-system: by monty-python.gnu.org: Solaris 10 (beta) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:8411 Archived-At: Hi Neil, Thanks for the review! On Mon 13 Apr 2009 11:05, Neil Jerram writes: > "serialize module information into syncase's output -- getting ready > for hygiene" > > Renaming (ice-9 annotate) to (ice-9 expand-support)... is that really > compelling enough for the risk of upsetting someone (if there is > anyone!) who is using (ice-9 annotate) ? (ice-9 annotate) is new, it's only been in for a month or two. So it's not possible that anyone uses it :) > So far, not understanding why we don't generate (@ ...) or (@@ > ... directly), but perhaps that will become clear. That's because syntax-case operates natively on "syntax objects", which already carry information with them regarding what piece of code introduced them into an expansion. Adding modules to that information was a natural fit, and allowed existing predicates to keep on working (e.g. identifier?). Also, using a part of existing Scheme (namely, @ and @@) introduces hygiene problems of its own. One can imagine sandbox modules in which certain imported macros are available, but @ and @@ are not available. (This latter point is moot in the current implementation, but that will change; the expansion stripper will do different, appropriate things for the memoizer and for the compiler.) > "add modules to syntax objects (part 1, intermediate step)" > > Out of interest, why use a vector here, whereas in the previous commit > you used structs? Structs are actually vectors, within syncase at least. (This is all within a big letrec IIRC, so these definitions don't escape the expansion definitions.) > "finish bootstrap to syntax-objects with modules" > > OK. I see now why the vector representation didn't matter. I hadn't > realized before that psyntax.scm required itself to bootstrap. That's the magical thing about psyntax.scm, that it's an expander that expands itself. Also the source of much consternation ;-) > "thread the module through syntax-case's expansion" > > +SCM_DEFINE (scm_procedure_module, "procedure-module", 1, 0, 0, > + (SCM proc), > + "Return the module that was current when this procedure was defined.\n" > + "Free variables in this procedure are resolved relative to the\n" > + "procedure's module.") > > Can you delete the second sentence? Sure. > +#define FUNC_NAME s_scm_procedure_module > +{ > + SCM_VALIDATE_PROC (SCM_ARG1, proc); > > Could you use scm_env_module () here to simplify the code? Ooh, good point. Done. > "more work on modules and hygiene, not finished yet, alas." > > Hmm, "eval closures are so 1990s": I think I agree, but I believe the > idea of eval closures was to allow other top levels than modules (and > the root). In case anyone out there is actually using a non-module > eval closure, are we inadvertently removing support for using syncase > with such top levels? Hm, I think so. Given that Guile assumes that eval closure lookup is idempotent, module binder procedures should have expressive power similar to that of eval closures. I would like to remove eval closures altogether. > + ;(debug-disable 'debug 'procnames) > + ;(read-disable 'positions) > > Is that temporary? It was intended to be ;-) Good catch, fixed locally. > -;; The following lines are necessary only if we start making changes > -;; (use-syntax sc-expand) > -;; (load-from-path "ice-9/psyntax") > > Am I correct in thinking that we have Makefile rules to automatically > do the needed generation when psyntax.scm changes? Yes. If you touch psyntax.scm and then make, you'll see it. > - (build-global-reference (source-annotation (car e)) value mod) > + (build-global-reference (source-annotation (car e)) value > + (if (syntax-object? (car e)) > + (syntax-object-module (car e)) > + mod)) > > Didn't understand this change, and I'm not sure if the changelog > covers it. Can you explain more? Sure. Consider a form produced by an expansion: `(a b c)'. The origin of the list structure could be different than the origin of the parts; for example a pmatch macro expansion would produce a (ppat ...) form whose contents depend on the source text, and thus the expanded form "belongs" to the source text, but the ppat identifier was introduced by the expansion, and thus "belongs" to the module that pmatch was defined in. The macro the above diff comes from, syntax-type, operates both on leaf nodes and at one level removed from the leaf node. That is to say, it can process identifiers, for dealing with identifier-syntax, and forms, for dealing with macro expansion. The above diff deals with a form like: (foo . rest) which should be expanded out if `foo' is a macro. If `foo' is not a macro, and not lexically bound, it is a call to a global procedure. But that global procedure should be looked up to relative to the macro that introduced it, not to the macro that introduced the enclosing form. The diff says, if `foo' was introduced via hygienic expansion, scope it relative to the macro that introduced it. (It could have been introduced via datum->syntax-object, in which case we just give it the scope of the containing expression, if any.) Note that this case would fall through nicely if syntax-type recursed on its car, but for various reasons (efficiency, clarity regarding macro expansions) it doesn't, so we have to put in the special case. > "hygienic compilation" > > OK. Am I correct in thinking that this is just an optimization - > i.e. that the VM would correctly interpret the @ and @@ macros later > on if they weren't intercepted here? No, the VM would compile to a call to the result of doing the runtime lookup. If the thing being looked up is actually a macro, you need to look it up at compile time -- at runtime a macro is not a procedure, so you would get an error. Like this, with today's guile: scheme@(guile-user)> ((lambda (x) (x 1 2)) if) ERROR: In procedure vm-run: ERROR: VM: Wrong type to apply: # [IP offset: 7] Contrast to the behavior under the interpreter: scheme@(guile-user)> ,o interp #t scheme@(guile-user)> ((lambda (x) (x 1 2)) if) $2 = 2 In the future, I'm not sure what to do -- if we always use the syncase expander, a bare "if" will yield a syntax error at expansion time. Scheme people claim this is the correct behavior, though I would prefer that macros have first-class values. > "@ and @@ as primitive macros" > > + ASSERT_SYNTAX (scm_ilength (expr) == 3, s_bad_expression, expr); > + ASSERT_SYNTAX (scm_ilength (scm_cadr (expr)) > 0, s_bad_expression, expr); > > I think it would be helpful to also assert that the caddr is a > symbol. (Also in @@) Done. > "fix hygiene + modules + local macros" > > - #:export (pmatch ppat)) > + #:export (pmatch)) > ;; FIXME: shouldn't have to export ppat... > > Can remove the FIXME too now! Indeed! > I believe this implementation confirms the point that we're relying on > (define-module ...) to have a lexical effect, and hence (given that > define-module isn't actually syntax) on the fact that Guile reads and > evaluates top-level forms from a file or the REPL sequentially. I > think we should just make that explicit somewhere, for anyone looking > in future at a different way of reading files. That is to say, the compiler has R5RS semantics: expressions are logically evaluated one-by-one. Adding compilation to this mix should not affect the semantics. (That's why the compiler can compile most all existing Guile code -- with a couple of caveats below.) I'm not sure exactly what you mean by "lexical" effect; if you mean that that compiling one expression can affect the next in the read sequence. This is true. In the define-module case, what happens is that you have code that is evaluated both at expansion time and run time -- so the expansion-time code (that does a set-current-module) gets run before the next expression is compiled. The compiler doesn't need to scan for define-module; it can primitive-eval forms at expansion time. But there is a catch, a bug really: currently we read in a whole file at once. This is the wrong thing, because expansion of a form may alter the current reader. We should read, expand, and compile forms one-by-one. Cheers, Andy -- http://wingolog.org/