From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: =?UTF-8?Q?Marc_Nieper=2DWi=C3=9Fkirchen?= Newsgroups: gmane.lisp.guile.devel Subject: Re: Feature request: Expose `ellipsis?' from psyntax.ss Date: Fri, 16 Nov 2018 14:37:31 +0100 Message-ID: References: <875zwzmq4n.fsf@netris.org> <87pnv6iss7.fsf@netris.org> <87k1leip2i.fsf@netris.org> <124085ab-2a76-4892-e790-d58b07bcb3fc@nieper-wisskirchen.de> <87lg5tj3gn.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000009a0ecb057ac84567" X-Trace: blaine.gmane.org 1542375357 29527 195.159.176.226 (16 Nov 2018 13:35:57 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 16 Nov 2018 13:35:57 +0000 (UTC) Cc: guile-devel@gnu.org To: mhw@netris.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Nov 16 14:35:53 2018 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gNeHp-0007Yf-0j for guile-devel@m.gmane.org; Fri, 16 Nov 2018 14:35:53 +0100 Original-Received: from localhost ([::1]:44906 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gNeJv-0006j8-7K for guile-devel@m.gmane.org; Fri, 16 Nov 2018 08:38:03 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:45626) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gNeJm-0006iz-7U for guile-devel@gnu.org; Fri, 16 Nov 2018 08:37:58 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gNeJg-0007rZ-7J for guile-devel@gnu.org; Fri, 16 Nov 2018 08:37:54 -0500 Original-Received: from mail-pg1-f176.google.com ([209.85.215.176]:45578) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gNeJd-0007qJ-1t for guile-devel@gnu.org; Fri, 16 Nov 2018 08:37:46 -0500 Original-Received: by mail-pg1-f176.google.com with SMTP id y4so10622043pgc.12 for ; Fri, 16 Nov 2018 05:37:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tErE+qHM8FXvjZs5bBMErSABihYlPtcNvYir02NwhuQ=; b=cur8+SCV8A6oOAvYSVKtdKotug0KTs7MpoQoBLdMVD3nlHwDBYfCeTQRHReT7OL903 1viSOJvWgF/LyuPUjWxajGG2P8ptfJEEwFdChX8P5kvOWF/ihyH/HZl5L2xd0SlFtj8J 2GjnLLdgbptQR1WadE8hWH3Fbl4QAwVcx+t+TVJaTP7rL7eUC1O+WTlMf5T5FCOquXft ADBS6q1HOkk5CiGuC36a5E1v5VcN87SVv7dGU73QK04JxESRqCloMrn3ihHOESOqqdrd F1rjoDONSNhOW40d7JCfVxZltx4wYbDaIYipliGWuCM830yGpNWpnFobx0H3lQX19asY qSYA== X-Gm-Message-State: AGRZ1gLZqQFtcZ3WRAjfJzAGRNi1KFGrp7FVdEemxl8XUgicKiOdnCSk lBVWhNdoaNmoaiTe+guHvCc/p4AdtCapb0VtY7nXEA== X-Google-Smtp-Source: AJdET5egk2PNSwREoFFFEHOQFIcIxXZbTgL4Uf8QB45+TKbd4kWA8tN+88P0sTXZV/BgSyiecIeGp+A5DsBG3lu70CI= X-Received: by 2002:a63:6f0d:: with SMTP id k13mr9893733pgc.42.1542375463323; Fri, 16 Nov 2018 05:37:43 -0800 (PST) In-Reply-To: <87lg5tj3gn.fsf@netris.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.215.176 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19744 Archived-At: --0000000000009a0ecb057ac84567 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Am Fr., 16. Nov. 2018 um 01:01 Uhr schrieb Mark H Weaver : > Hi Marc, > > Marc Nieper-Wi=C3=9Fkirchen writes: > > > > Let's assume we are writing a macro that reimplements syntax (or som= e > > > variation thereof) and which has to check whether identifiers are > > > ellipses. For example, the following could be given: > > > > > > (with-ellipsis e > > > (my-syntax a e) > > > > > > Now, this could be a result of a macro expansion and e could carry > > > different marks than with-syntax or my-syntax. This is why I have be= en > > > thinking that one also needs the lexical context of my-syntax and no= t > > > only the context of e. > > > > I don't see what problem would be caused by 'e' carrying different mar= ks > > than 'my-syntax'. > > > > As far as I can tell, in the end, the two instances of 'e' above will > > effectively be compared to one another using 'bound-identifier=3D?'. = They > > must have the same name and the same marks to match. The marks on > > 'my-syntax' are irrelevant here. > > > > I have been thinking of the scope in which $sc-ellipsis is bound by > > `with-syntax'. > > You've written 'with-syntax' is several places, in both this email and > in your previous email, and I'm guessing that you meant to write > 'with-ellipsis' in each of those places. Is that right? > Yes! I shouldn't do things in parallel (writing emails for this list and programming syntax-case transformers, which use with-syntax but not with-ellipsis, at the same time). I hope it didn't cause too much confusion= . > > If `my-syntax' is within the scope of `with-ellipsis', the binding of > > $sc-ellipsis introduced by this `with-syntax' will be relevant; if > > `my-syntax' is not in the lexical scope of `with-ellipsis', the > > binding should be irrelevant; thus my thought that we need the lexical > > information of my-syntax as well. > > > > Operationally, when (with-ellipsis e (my-syntax a e)) is expanded, 'e' > > will be added to the macro expansion environment as the innermost > > binding of the ellipsis identifier, and then (my-syntax a e) will be > > expanded within that new expansion environment. That is the expansion > > environment that will be consulted by the 'ellipsis-identifier?' > > predicate to find the current ellipsis identifier, which is compared > > with its argument (after stripping its anti-mark) using > > 'bound-identifier=3D?'. > > > > Aha, so maybe I have misunderstood the scope of `with-syntax'. Please > > consider the following example: > > > > (define-syntax foo > > (lambda (stx) > > (with-ellipsis e > > (syntax-case stx () > > ((_ x e) (bar #'(x e))))))) > > > > (eval-when (expand) > > (define (bar x*) > > (syntax-case x* () > > ((x ...) ---)))) > > > > I would have thought that the `...' identifier in `bar' is recognized > > as an ellipsis, > > It is. > > > but from what you are saying it seems that the binding `with-syntax' > > is dynamic with respect to macro expansion (like syntax > > parameters). Is this really what we want? > > I agree that it's not what we want, and if I understand correctly, it's > not what we have in Guile. > > In Psyntax, lexical lookups of identifiers are done in two steps, using > two different data structures. First, the deferred substitutions in the > wrap are applied to the identifier, which yields a gensym if the > identifier is lexically bound. Next, the gensym is looked up in the > expansion environment 'r' to find the actual binding. > > The deferred substitutions are applied to the inner bodies of each core > binding construct. When the macro expander encounters a core binding > construct, a fresh gensym is created for the binding, and that gensym is > effectively substituted for all free occurrences of the identifier > within the inner body. Mostly for efficiency reasons, this substitution > is done lazily, by adding it to the wrap. The expansion environment is > also extended each time the macro expander encounters a core binding > construct. > > With this in mind, let's examine your example above more closely. The > ellipsis binding for 'e' is only in the transformer environment when the > 'syntax-case' form is expanded. It is _not_ in the transformer > environment when your 'foo' macro is later used. > I agree and I see that my example doesn't demonstrate what it should have demonstrated because `bar' is not executed before `foo' is used as a macro. The example should have been more like the following: (define-syntax foo (lambda (stx) (with-ellipsis e (syntax-case (third-party-macro-transformer-helper-macro stx) () ---)))) Here, the helper macro may expand into another instance of syntax-case. That instance should not recognize `e' as the ellipsis but whatever the ellipsis was where the helper macro was defined. > But let's go one step further. Let's consider what will happen if 'foo' > is used within 'with-ellipsis': > > (with-ellipsis --- (foo a b)) > > When this is expanded, a fresh gensym will be generated, and an ellipsis > binding will be added for that gensym in the expansion environment 'r'. > Also, a substitution from #{ $sc-ellipsis }# to that gensym will be > added to the wrap of (foo a b). > > Now let's consider how 'bar' will be affected by this. In the example > you give, where 'bar' uses 'syntax-case', the ellipsis identifier will > be looked up in the transformer environment where 'bar' is *defined*, > not the transformer environment where 'bar' is called. > > But let's suppose that we change 'bar' to use 'ellipsis-identifier?' at > run-time, like this: > > (define-syntax foo > (lambda (stx) > (with-ellipsis e > (syntax-case stx () > ((_ x e) (bar #'(x e))))))) > > (eval-when (expand) > (define (bar x*) > (syntax-case x* () > ((x dots) > (ellipsis-identifier? #'dots) > #'#true) > (_ > #'#false)))) > > We now see this behavior with my draft patch: > > (with-ellipsis --- (foo a b)) =3D> #false > (with-ellipsis --- (foo a ...)) =3D> #false > (with-ellipsis --- (foo a ---)) =3D> #true > > I think this is what we want, right? > I think it looks correct. > When 'bar' is called, there will be a binding in the transformer > environment 'r' that maps a gensym to an ellipsis binding, which > specifies '---' as the ellipsis identifier. However, that binding will > only apply when testing identifiers that have been wrapped to include a > substitution from #{ $sc-ellipsis }# to the same gensym, so it will only > apply to identifiers that are in body of the same 'with-ellipsis' form. > > > Therefore I think, we want `with-ellipsis' to be lexically scoped (in > > the macro transformer code). > > Yes, that was certainly my intent. > Let's run the following example: (eval-when (expand) (define-syntax bar (syntax-rules () ((_ stx) (syntax-case stx () ((_ a (... ...)) #'#t) ((_ a b c) #'#f)))))) (define-syntax foo (lambda (stx) (with-ellipsis e (bar stx)))) (display (foo 1 2 3)) (newline) This one displays `#t' in Guile, which is exactly what we want. I guess the reason is that the macro invocation `(bar stx)' creates a new transformer environment, in which `{# $sc-ellipsis #}' becomes unbound again. Now, why does the following work (i.e. why does it print `#t')? (eval-when (expand) (define-syntax bar2 (syntax-rules () ((_ e body) (with-ellipsis e body))))) (define-syntax foo2 (lambda (stx) (bar2 f (syntax-case stx () ((_ a ...) #'#t) ((_ a b c) #'#f))))) (display (foo2 1 2 3)) (newline) If I change `f' to `e' in `foo2', `#f' will be printed, so the `with-ellipsis' effects the body `body' if and only if `e' has the same marks as the body (or, rather, `...' in the body). Is this the right semantics? I incline to think so but I am not sure. > > Thanks for the explanation. I have been toying with my own > > > implementation of the syntax-case system. In my implementation the > > > (shared) lexical environments are part of the wraps (so the > > > identifiers are in some way self-contained). > > > > Interesting. Are locally-bound macro transformers included in those > > lexical environments? If so, how do you implement 'letrec-syntax'? > > > > My environments are lists of ribs where each rib corresponds to a > > lexical frame. Given the form > > > > (letrec-syntax ((var init)) body ...) > > > > I create a new rib that contains the binding of var to init and add a > > wrap around each expression in body ... that contains the new rib but > > no new marks. > > I think that you also need to apply the same wrap to 'init', no? > Yes, for `letrec-syntax'. For `let-syntax', I don't add the new rib to each `init'. > > When the body is examined by the expander, the wraps are > > gradually pushed down (like in the original description of > > `syntax-case' by Dybvig and Hieb) so that eventually the environments > > stored with the identifiers in body gain another rib. > > > > > Will ellipsis? also work outside of macros? Say, what would be the > > > result of the following (run-time) code? > > > > > > (with-syntax e > > > (ellipsis? #'e) > > > > No, this is an error. Like 'syntax-local-binding', the > > 'ellipsis-identifier?' predicate must be called within the dynamic > > extent of a macro transformer call by the macro expander. > > > > Is this related to the question above of whether `with-syntax' has > > lexical or dynamic scope? In the former case I don't see a theoretical > > reason why it has to be restricted to the dynamic extent of a macro > > transformer call. > > I'll assume that you meant to write 'with-ellipsis' above, not > 'with-syntax', as I did in my earlier responses. > Yep. > The reason this can't work is ultimately because the ellipsis bindings > are stored in the transformer environment, which simply does not exist > at run time when 'ellipsis?' is called here. > > It might be possible to make this work with your approach to storing > full binding information in the syntax objects, but that's not how > Psyntax works. > > FWIW, I will say that in Guile, the size of syntax objects in Psyntax is > already quite significant in practice, and most of that information is > never used unless the syntax objects are passed as the first argument to > 'datum->syntax'. Many years ago, I reduced the size of 'psyntax-pp.scm' > by an order of magnitude by stripping out most of that information from > the syntax objects in the expanded code. > I should also add that my syntax-case expander is still work-in-progress and I haven't profiled the code yet. In particular, I may have to improve structure sharing between syntax objects to make them less fat. My eventual goal is to write a Scheme compiler in Guile, but it will still take some time. > Therefore, I would be reluctant to make the syntax objects any larger > than they already are in Psyntax. > That's reasonable. I don't think that one actually needs to use `ellipsis-identifier?' in run-time code. > > > P.S.: By the way, the module (system syntax) and in particular the > > > procedure syntax-local-binding has already helped me a lot because I > > > needed to attach extra information to symbols and Guile doesn't (yet= ) > > > support Chez's define-property (well, this would be another feature > > > request). > > > > Hmm. Can you tell me more specifically how you are using > > 'syntax-local-binding' to accomplish this? As the Guile manual warns, > > those interfaces are subject to change in future versions of Guile, an= d > > therefore it is best to avoid them where possible. > > > > What I have been implementing is a pattern matcher and rewriter as a > > macro in Guile that works much like syntax-case/syntax. Let's call it > > my-syntax-case/my-syntax. When `my-syntax' is given a template, it has > > to check whether an identifier appearing in the template is a > > "my-"pattern variable or not. For that, `my-syntax-case' introduces > > (via `let-syntax') lexical bindings of the identifiers that are used > > as pattern variables. The associated syntax transformer just outputs > > an error (namely that the pattern variable is used outside of > > `my-syntax'). However, I also attach a custom property (with > > `make-object-property`) to this syntax transformer that holds > > information about the match and the nesting depth of the pattern > > variable. In order to retrieve this information in `my-syntax', I use > > `syntax-local-binding' to get hold of the associated syntax > > transformer. > > Okay. I would suggest another approach that is more portable: instead > of having the associated syntax transformers always return an error, add > a clause so that when they are applied to a special keyword, they expand > into something that includes the information about the match. > > For example, you might take a look at 'define-tagged-inlinable' in > Guile's implementation of srfi-9.scm, where I did something like this. > The problem is that I have to be able to test arbitrary identifiers for whether they are pattern variables. Now if `x' is any identifier and I expand something like `(x :keyword k ...)` and `x' is not a pattern variable that I control, anything may happen. Another (minor) problem would be that the macro expansion (x :keyword k ...) would create new marks so I would have to pass many identifiers in `k ...' to the continuation. Contrary to that, calling a procedure during transformation does not introduce fresh marks. > > In Chez Scheme, I would have used `define-property' to define my > > custom property directly on the identifier standing for the pattern > > variable. I haven't found an equivalent feature in Guile. I don't know > > how to nicely code my-syntax-case/my-syntax in standard R6RS. > > Sure, that sounds like a nice feature. I'll add it to my TODO list :) > That would be great! :-) All the best, Marc --0000000000009a0ecb057ac84567 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Am Fr., 16. Nov. 2018 um= 01:01=C2=A0Uhr schrieb Mark H Weaver <mhw@netris.org>:
Hi Marc,

Marc Nieper-Wi=C3=9Fkirchen <marc@nieper-wisskirchen.de> writes:

>=C2=A0 > Let's assume we are writing a macro that reimplements s= yntax (or some
>=C2=A0 > variation thereof) and which has to check whether identifie= rs are
>=C2=A0 > ellipses. For example, the following could be given:
>=C2=A0 >
>=C2=A0 > (with-ellipsis e
>=C2=A0 >=C2=A0 =C2=A0(my-syntax a e)
>=C2=A0 >=C2=A0
>=C2=A0 > Now, this could be a result of a macro expansion and e coul= d carry
>=C2=A0 > different marks than with-syntax or my-syntax. This is why = I have been
>=C2=A0 > thinking that one also needs the lexical context of my-synt= ax and not
>=C2=A0 > only the context of e.
>
>=C2=A0 I don't see what problem would be caused by 'e' carr= ying different marks
>=C2=A0 than 'my-syntax'.
>
>=C2=A0 As far as I can tell, in the end, the two instances of 'e= 9; above will
>=C2=A0 effectively be compared to one another using 'bound-identifi= er=3D?'.=C2=A0 They
>=C2=A0 must have the same name and the same marks to match.=C2=A0 The m= arks on
>=C2=A0 'my-syntax' are irrelevant here.
>
> I have been thinking of the scope in which $sc-ellipsis is bound by > `with-syntax'.

You've written 'with-syntax' is several places, in both this em= ail and
in your previous email, and I'm guessing that you meant to write
'with-ellipsis' in each of those places.=C2=A0 Is that right?

Yes! I shouldn't do things in parallel (= writing emails for this list and programming syntax-case transformers, whic= h use with-syntax but not with-ellipsis, at the same time). I hope it didn&= #39;t cause too much confusion.
=C2=A0
> If `my-syntax' is within the scope of `with-ellipsis', the bin= ding of
> $sc-ellipsis introduced by this `with-syntax' will be relevant; if=
> `my-syntax' is not in the lexical scope of `with-ellipsis', th= e
> binding should be irrelevant; thus my thought that we need the lexical=
> information of my-syntax as well.
>=C2=A0
>=C2=A0 Operationally, when (with-ellipsis e (my-syntax a e)) is expande= d, 'e'
>=C2=A0 will be added to the macro expansion environment as the innermos= t
>=C2=A0 binding of the ellipsis identifier, and then (my-syntax a e) wil= l be
>=C2=A0 expanded within that new expansion environment.=C2=A0 That is th= e expansion
>=C2=A0 environment that will be consulted by the 'ellipsis-identifi= er?'
>=C2=A0 predicate to find the current ellipsis identifier, which is comp= ared
>=C2=A0 with its argument (after stripping its anti-mark) using
>=C2=A0 'bound-identifier=3D?'.
>
> Aha, so maybe I have misunderstood the scope of `with-syntax'. Ple= ase
> consider the following example:
>
> (define-syntax foo
>=C2=A0 =C2=A0(lambda (stx)
>=C2=A0 =C2=A0 =C2=A0(with-ellipsis e
>=C2=A0 =C2=A0 =C2=A0 =C2=A0(syntax-case stx ()
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0((_ x e) (bar #'(x e)))))))
>
> (eval-when (expand)
>=C2=A0 =C2=A0(define (bar x*)
>=C2=A0 =C2=A0 =C2=A0(syntax-case x* ()
>=C2=A0 =C2=A0 =C2=A0 =C2=A0((x ...) ---))))
>
> I would have thought that the `...' identifier in `bar' is rec= ognized
> as an ellipsis,

It is.

> but from what you are saying it seems that the binding `with-syntax= 9;
> is dynamic with respect to macro expansion (like syntax
> parameters). Is this really what we want?

I agree that it's not what we want, and if I understand correctly, it&#= 39;s
not what we have in Guile.

In Psyntax, lexical lookups of identifiers are done in two steps, using
two different data structures.=C2=A0 First, the deferred substitutions in t= he
wrap are applied to the identifier, which yields a gensym if the
identifier is lexically bound.=C2=A0 Next, the gensym is looked up in the expansion environment 'r' to find the actual binding.

The deferred substitutions are applied to the inner bodies of each core
binding construct.=C2=A0 When the macro expander encounters a core binding<= br> construct, a fresh gensym is created for the binding, and that gensym is effectively substituted for all free occurrences of the identifier
within the inner body.=C2=A0 Mostly for efficiency reasons, this substituti= on
is done lazily, by adding it to the wrap.=C2=A0 The expansion environment i= s
also extended each time the macro expander encounters a core binding
construct.

With this in mind, let's examine your example above more closely.=C2=A0= The
ellipsis binding for 'e' is only in the transformer environment whe= n the
'syntax-case' form is expanded.=C2=A0 It is _not_ in the transforme= r
environment when your 'foo' macro is later used.

I agree and I see that my example doesn't demonstrate= what it should have demonstrated because `bar' is not executed before = `foo' is used as a macro. The example should have been more like the fo= llowing:

(define-syntax foo
=C2=A0 (= lambda (stx)
=C2=A0=C2=A0=C2=A0 (with-ellipsis e
=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 (syntax-case (third-party-macro-transformer-helper= -macro stx) ()
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ---))))=

Here, the helper macro may expand into another in= stance of syntax-case. That instance should not recognize `e' as the el= lipsis but whatever the ellipsis was where the helper macro was defined.
=C2=A0
But let's go one step further.=C2=A0 Let's consider what will happe= n if 'foo'
is used within 'with-ellipsis':

=C2=A0 (with-ellipsis --- (foo a b))

When this is expanded, a fresh gensym will be generated, and an ellipsis binding will be added for that gensym in the expansion environment 'r&#= 39;.
Also, a substitution from #{ $sc-ellipsis }# to that gensym will be
added to the wrap of (foo a b).

Now let's consider how 'bar' will be affected by this.=C2=A0 In= the example
you give, where 'bar' uses 'syntax-case', the ellipsis iden= tifier will
be looked up in the transformer environment where 'bar' is *defined= *,
not the transformer environment where 'bar' is called.

But let's suppose that we change 'bar' to use 'ellipsis-ide= ntifier?' at
run-time, like this:

=C2=A0 (define-syntax foo
=C2=A0 =C2=A0 (lambda (stx)
=C2=A0 =C2=A0 =C2=A0 (with-ellipsis e
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (syntax-case stx ()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ((_ x e) (bar #'(x e)))))))

=C2=A0 (eval-when (expand)
=C2=A0 =C2=A0 (define (bar x*)
=C2=A0 =C2=A0 =C2=A0 (syntax-case x* ()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ((x dots)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(ellipsis-identifier? #'dots)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0#'#true)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (_
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0#'#false))))

We now see this behavior with my draft patch:

=C2=A0 (with-ellipsis --- (foo a b))=C2=A0 =C2=A0 =3D> #false
=C2=A0 (with-ellipsis --- (foo a ...))=C2=A0 =3D> #false
=C2=A0 (with-ellipsis --- (foo a ---))=C2=A0 =3D> #true

I think this is what we want, right?

I = think it looks correct.
=C2=A0
When 'bar' is called, there will be a binding in the transformer environment 'r' that maps a gensym to an ellipsis binding, which specifies '---' as the ellipsis identifier.=C2=A0 However, that bin= ding will
only apply when testing identifiers that have been wrapped to include a
substitution from #{ $sc-ellipsis }# to the same gensym, so it will only apply to identifiers that are in body of the same 'with-ellipsis' f= orm.

> Therefore I think, we want `with-ellipsis' to be lexically scoped = (in
> the macro transformer code).

Yes, that was certainly my intent.

Let&= #39;s run the following example:

(eval-when (expan= d)
=C2=A0 (define-syntax bar
=C2=A0=C2=A0=C2=A0 (syntax-rules ()
= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ((_ stx)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 (syntax-case stx ()
=C2=A0=C2=A0=C2=A0 =C2=A0((_ a (... ...))
= =C2=A0=C2=A0=C2=A0 =C2=A0 #'#t)
=C2=A0=C2=A0=C2=A0 =C2=A0((_ a b c)<= br>=C2=A0=C2=A0=C2=A0 =C2=A0 #'#f))))))

(define-syntax foo
= =C2=A0 (lambda (stx)
=C2=A0=C2=A0=C2=A0 (with-ellipsis e (bar stx))))
(display (foo 1 2 3))
(newline)

This one displays= `#t' in Guile, which is exactly what we want. I guess the reason is th= at the macro invocation `(bar stx)' creates a new transformer environme= nt, in which `{# $sc-ellipsis #}' becomes unbound again.

Now, why d= oes the following work (i.e. why does it print `#t')?

(eval-when (expand)
=C2=A0 (define-syntax bar2=C2=A0=C2=A0=C2=A0 (syntax-rules ()
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ((_ = e body)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (with-ellipsis e body)))))<= br>
(define-syntax foo2
=C2=A0 (lambda (stx)
=C2=A0=C2=A0=C2=A0 (b= ar2 f (syntax-case stx ()
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 ((_ a ..= .)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0 #'#t)
=C2=A0=C2=A0= =C2=A0 =C2=A0=C2=A0=C2=A0 ((_ a b c)
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2= =A0=C2=A0 #'#f)))))

(display (foo2 1 2 3))
(newline)

If I change `f' to `e&= #39; in `foo2', `#f' will be printed, so the `with-ellipsis' ef= fects the body `body' if and only if `e' has the same marks as the = body (or, rather, `...' in the body). Is this the right semantics? I in= cline to think so but I am not sure.

>=C2=A0 > Thanks for the explanation. I have been toying with my own<= br> >=C2=A0 > implementation of the syntax-case system. In my implementat= ion the
>=C2=A0 > (shared) lexical environments are part of the wraps (so the=
>=C2=A0 > identifiers are in some way self-contained).
>
>=C2=A0 Interesting.=C2=A0 Are locally-bound macro transformers included= in those
>=C2=A0 lexical environments?=C2=A0 If so, how do you implement 'let= rec-syntax'?
>
> My environments are lists of ribs where each rib corresponds to a
> lexical frame. Given the form
>
> (letrec-syntax ((var init)) body ...)
>
> I create a new rib that contains the binding of var to init and add a<= br> > wrap around each expression in body ... that contains the new rib but<= br> > no new marks.

I think that you also need to apply the same wrap to 'init', no?

Yes, for `letrec-syntax'. For `let-sy= ntax', I don't add the new rib to each `init'.
= =C2=A0
> When the body is examined by the expander, the wraps are
> gradually pushed down (like in the original description of
> `syntax-case' by Dybvig and Hieb) so that eventually the environme= nts
> stored with the identifiers in body gain another rib.
>=C2=A0
>=C2=A0 > Will ellipsis? also work outside of macros? Say, what would= be the
>=C2=A0 > result of the following (run-time) code?
>=C2=A0 >
>=C2=A0 > (with-syntax e
>=C2=A0 >=C2=A0 =C2=A0(ellipsis? #'e)
>
>=C2=A0 No, this is an error.=C2=A0 Like 'syntax-local-binding',= the
>=C2=A0 'ellipsis-identifier?' predicate must be called within t= he dynamic
>=C2=A0 extent of a macro transformer call by the macro expander.
>
> Is this related to the question above of whether `with-syntax' has=
> lexical or dynamic scope? In the former case I don't see a theoret= ical
> reason why it has to be restricted to the dynamic extent of a macro > transformer call.

I'll assume that you meant to write 'with-ellipsis' above, not<= br> 'with-syntax', as I did in my earlier responses.

Yep.
=C2=A0
The reason this can't work is ultimately because the ellipsis bindings<= br> are stored in the transformer environment, which simply does not exist
at run time when 'ellipsis?' is called here.

It might be possible to make this work with your approach to storing
full binding information in the syntax objects, but that's not how
Psyntax works.

FWIW, I will say that in Guile, the size of syntax objects in Psyntax is already quite significant in practice, and most of that information is
never used unless the syntax objects are passed as the first argument to 'datum->syntax'.=C2=A0 Many years ago, I reduced the size of = 9;psyntax-pp.scm'
by an order of magnitude by stripping out most of that information from
the syntax objects in the expanded code.

I should also add that my syntax-case expander is still work-in-progress = and I haven't profiled the code yet. In particular, I may have to impro= ve structure sharing between syntax objects to make them less fat. My event= ual goal is to write a Scheme compiler in Guile, but it will still take som= e time.
=C2=A0
Therefore, I would be reluctant to make the syntax objects any larger
than they already are in Psyntax.

That&= #39;s reasonable. I don't think that one actually needs to use `ellipsi= s-identifier?' in run-time code.
=C2=A0
>=C2=A0 > P.S.: By the way, the module (system syntax) and in particu= lar the
>=C2=A0 > procedure syntax-local-binding has already helped me a lot = because I
>=C2=A0 > needed to attach extra information to symbols and Guile doe= sn't (yet)
>=C2=A0 > support Chez's define-property (well, this would be ano= ther feature
>=C2=A0 > request).
>
>=C2=A0 Hmm.=C2=A0 Can you tell me more specifically how you are using >=C2=A0 'syntax-local-binding' to accomplish this?=C2=A0 As the = Guile manual warns,
>=C2=A0 those interfaces are subject to change in future versions of Gui= le, and
>=C2=A0 therefore it is best to avoid them where possible.
>
> What I have been implementing is a pattern matcher and rewriter as a > macro in Guile that works much like syntax-case/syntax. Let's call= it
> my-syntax-case/my-syntax. When `my-syntax' is given a template, it= has
> to check whether an identifier appearing in the template is a
> "my-"pattern variable or not. For that, `my-syntax-case'= introduces
> (via `let-syntax') lexical bindings of the identifiers that are us= ed
> as pattern variables. The associated syntax transformer just outputs > an error (namely that the pattern variable is used outside of
> `my-syntax'). However, I also attach a custom property (with
> `make-object-property`) to this syntax transformer that holds
> information about the match and the nesting depth of the pattern
> variable. In order to retrieve this information in `my-syntax', I = use
> `syntax-local-binding' to get hold of the associated syntax
> transformer.

Okay.=C2=A0 I would suggest another approach that is more portable: instead=
of having the associated syntax transformers always return an error, add a clause so that when they are applied to a special keyword, they expand into something that includes the information about the match.

For example, you might take a look at 'define-tagged-inlinable' in<= br> Guile's implementation of srfi-9.scm, where I did something like this.<= br>

The problem is that I have to be able t= o test arbitrary identifiers for whether they are pattern variables. Now if= `x' is any identifier and I expand something like `(x :keyword k ...)`= and `x' is not a pattern variable that I control, anything may happen.= Another (minor) problem would be that the macro expansion (x :keyword k ..= .) would create new marks so I would have to pass many identifiers in `k ..= .' to the continuation. Contrary to that, calling a procedure during tr= ansformation does not introduce fresh marks.
=C2=A0
=
> In Chez Scheme, I would have used `define-property' to define my > custom property directly on the identifier standing for the pattern > variable. I haven't found an equivalent feature in Guile. I don= 9;t know
> how to nicely code my-syntax-case/my-syntax in standard R6RS.

Sure, that sounds like a nice feature.=C2=A0 I'll add it to my TODO lis= t :)

That would be great! :-)
=

All the best,

Marc
=C2=A0
--0000000000009a0ecb057ac84567--