Enhancement to the syntax system?

Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1925 bytes --]

Hi,

I want to discuss some of my experiences using the current syntax system
when making the racket match
matcher utilizing syntax-parse.

When doing this I tried to use another style of macro parsing then seen in
guiles ice-9 match e.g. Alex Shinns matcher for example. What I mean here
is that I tried to use #' #, #,@ and calling out to functions in stead of
writing one more macro. This works but it has a caveat.
Hygiene is harder to maintain. e.g.  I kept on hitting this kind of code
snippets

#'(let ((x v))
    #,(f rest #'x))

The problem with this code is hygiene, I need to make a gensym and use
with-syntax to bound x to that gensym in order to be safe
at the macro expansion. Apart from this I liked the flow of the program and
how natural it was to comprehend the code.

The downside to this style is that we looses safety with respect to hygiene
bugs. So can we improve on this. I would say yes!
Consider introduceing #. and #.@ that works very much like #, and #,@ but
with the distinction that #. creates a syntactic lambda e.g.
The construction of the syntax object translates to
#'(let ((x v)) #.(f rest #'x))

=>

#'(let ((x v)) #,(lambda (env) (with-syntax-env env (f rest #'x)))

E.g. we inject a lambda inside the syntax structure to be parsed by the
syntax expander later on and the syntax expander
will recognize the lambda and call it with the current environment of the
expander. The lambda will in turn set the syntax env that
should be used by the #'x form in order match that x with the bounded x. I
think that you get my intention.

The question is why something like this does not exists. Is it because of
weaknesses making this procedure unsound or any other reasons. I do think
such a feature would be important but on the other hand, someone should
have thought about this before and rejected the idea so my question is
simply why is this rejected any clues or pointers?

/Regards
Stefan

[-- Attachment #2: Type: text/html, Size: 2091 bytes --]

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hi Stefan,

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> Hygiene is harder to maintain. e.g.  I kept on hitting this kind of code
> snippets
>
> #'(let ((x v))
>     #,(f rest #'x))
>
> The problem with this code is hygiene, I need to make a gensym and use
> with-syntax to bound x to that gensym in order to be safe
> at the macro expansion.

What do you mean?  Here ‘x’ leads to a newly introduced binding.

Likewise, the code below doesn’t introduce bindings non-hygienic
bindings, and is non-ambiguous:

  (macroexpand '(let-syntax ((f (lambda (s)
                                  (syntax-case s ()
                                    ((_ x y)
                                     #`(let ((p x))
                                         (+ p #,(identity #'y))))))))
                  (f 1 2)))

  => #<tree-il (let (p) (p-28239) ((const 1)) (apply (toplevel +) (lexical p p-28239) (const 2)))>

Thanks,
Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

Maybe this help to see what I'm after,

#'(let ((x v)) #.(f #'x))

<=>

(let-syntax ((g (lambda (stx) (syntax-case  stx ((_ x) (f #'x)))))
   #'(let ((x v)) (g x))


Now I would like to have a corresponding #.@ notation as well but can't
find an analog
for that :-(

/Stefan

On Mon, Jul 2, 2012 at 9:28 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hi Stefan,
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> > Hygiene is harder to maintain. e.g.  I kept on hitting this kind of code
> > snippets
> >
> > #'(let ((x v))
> >     #,(f rest #'x))
> >
> > The problem with this code is hygiene, I need to make a gensym and use
> > with-syntax to bound x to that gensym in order to be safe
> > at the macro expansion.
>
> What do you mean?  Here ‘x’ leads to a newly introduced binding.
>
> Likewise, the code below doesn’t introduce bindings non-hygienic
> bindings, and is non-ambiguous:
>
>   (macroexpand '(let-syntax ((f (lambda (s)
>                                   (syntax-case s ()
>                                     ((_ x y)
>                                      #`(let ((p x))
>                                          (+ p #,(identity #'y))))))))
>                   (f 1 2)))
>
>   => #<tree-il (let (p) (p-28239) ((const 1)) (apply (toplevel +) (lexical
> p p-28239) (const 2)))>
>
> Thanks,
> Ludo’.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1878 bytes --]

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hey!

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> Maybe this help to see what I'm after,
>
> #'(let ((x v)) #.(f #'x))
>
> <=>
>
> (let-syntax ((g (lambda (stx) (syntax-case  stx ((_ x) (f #'x)))))
>    #'(let ((x v)) (g x))

Sorry, I fail to understand the problem you’re trying to solve.

The above idiom is not frequent to the point that special syntax is
needed, is it?

Thanks,
Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1402 bytes --]

On Tue, Jul 3, 2012 at 12:41 AM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hey!
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> > Maybe this help to see what I'm after,
> >
> > #'(let ((x v)) #.(f #'x))
> >
> > <=>
> >
> > (let-syntax ((g (lambda (stx) (syntax-case  stx ((_ x) (f #'x)))))
> >    #'(let ((x v)) (g x))
>
> Sorry, I fail to understand the problem you’re trying to solve.
>
> Hmm The #' probably need to be moved to the left of the expression and
there might be problems with phases in this analogy but I guess you saw
that I wanted to spin the function f on a syntax argument that contained
the binding of x done in the let which is not the case If you do a simple #,


The above idiom is not frequent to the point that special syntax is
> needed, is it?
>
>
If you want to code your macros like Alex does in ice-9/match.scm then it's
not a problem.
If you want to try another path using functions in stead of macros and
working hard with #, and #,@
you will for complex macros like a matcher need to gensym by hand or
destroy the readability of the
code. As illustrated by the simple example above. It's not that a
devastating issue because I have coded quite a lot of CL macros before and
can cope with gensymming, but I find it frustrating to know that there
probably exists a better way.


> Thanks,
> Ludo’.
>

/Stefan

[-- Attachment #2: Type: text/html, Size: 2168 bytes --]

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hey!

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

>> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>>
>> > Maybe this help to see what I'm after,
>> >
>> > #'(let ((x v)) #.(f #'x))
>> >
>> > <=>
>> >
>> > (let-syntax ((g (lambda (stx) (syntax-case  stx ((_ x) (f #'x)))))
>> >    #'(let ((x v)) (g x))

[...]

> If you want to try another path using functions in stead of macros and
> working hard with #, and #,@
> you will for complex macros like a matcher need to gensym by hand or
> destroy the readability of the
> code. As illustrated by the simple example above.

Hmm, the example above does not use gensym.  What am I missing?

Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

You do not need gensyms if you try to mimic or implement my suggested #. .
On the
other hand when if you do this

(define (f stx) #`(let ((x 1)) #,stx))

and use this with

#`(let ((x 2)) #,(f #'x))

the resulting expanded code would produce 1 which is not what you want.
So in the racket matcher I wrote I needed to do

(with-syntax ((x (datum->syntax stx (gensym "x")))) #`(let ((x 2)) #,(f
#'x))))))

Hope that this makes things clear!

/Stefan

On Tue, Jul 3, 2012 at 11:33 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hey!
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> >> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
> >>
> >> > Maybe this help to see what I'm after,
> >> >
> >> > #'(let ((x v)) #.(f #'x))
> >> >
> >> > <=>
> >> >
> >> > (let-syntax ((g (lambda (stx) (syntax-case  stx ((_ x) (f #'x)))))
> >> >    #'(let ((x v)) (g x))
>
> [...]
>
> > If you want to try another path using functions in stead of macros and
> > working hard with #, and #,@
> > you will for complex macros like a matcher need to gensym by hand or
> > destroy the readability of the
> > code. As illustrated by the simple example above.
>
> Hmm, the example above does not use gensym.  What am I missing?
>
> Ludo’.
>

[-- Attachment #2: Type: text/html, Size: 1910 bytes --]

Re: Enhancement to the syntax system?

[Marijn]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Stefan,

On 03-07-12 23:52, Stefan Israelsson Tampe wrote:
> You do not need gensyms if you try to mimic or implement my
> suggested #. . On the other hand when if you do this
> 
> (define (f stx) #`(let ((x 1)) #,stx))
> 
> and use this with
> 
> #`(let ((x 2)) #,(f #'x))
> 
> the resulting expanded code would produce 1 which is not what you
> want.

I tried it in Racket:


(define-for-syntax (f stx) #`(let ((x 1)) #,stx))

(define-syntax (m stx)
  (syntax-case stx ()
    ((_) #`(let ((x 2)) #,(f #'x))) ))

(m)


but I get error messages which I don't know what to do about. Do you
have running examples of code not producing the result you want?

Marijn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/z9PsACgkQp/VmCx0OL2xMNACbBJKq9nZJKTzCJbdnq0iPgJoE
rSwAn2yJ1JTLMUn6v5ZR/PatxcGxgVrU
=HFFX
-----END PGP SIGNATURE-----

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1986 bytes --]

Funny,  for guile I get,
scheme@(guile-user)> (define (f x) #`(let ((x 1)) #,x))
scheme@(guile-user)> (define-syntax m (lambda (x) (syntax-case x () ((_)
#`(let ((x 2)) #,(f #'x))))))
scheme@(guile-user)> (m)
$1 = 1

I ported the racket matcher to guile and worked in that environment, so
this illustrate the problem.
In racket the same idea will error out as you say (I did not find anything
wrong with your example).
I do prefer the racket behavior of bailing out because it can help
detecting the bug in stead of silently
behave against the intention for newbies. My original code where this issue
comes from is located in
compat/racket/match/match.scm in the syntax-parse repo at

https://gitorious.org/guile-syntax-parse/guile-syntax-parse

This is the racket matcher implemented using syntax-parse!

These bugs should have been corrected though in that code!

/Stefan

On Wed, Jul 4, 2012 at 9:47 AM, Marijn <hkBst@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Stefan,
>
> On 03-07-12 23:52, Stefan Israelsson Tampe wrote:
> > You do not need gensyms if you try to mimic or implement my
> > suggested #. . On the other hand when if you do this
> >
> > (define (f stx) #`(let ((x 1)) #,stx))
> >
> > and use this with
> >
> > #`(let ((x 2)) #,(f #'x))
> >
> > the resulting expanded code would produce 1 which is not what you
> > want.
>
> I tried it in Racket:
>
>
> (define-for-syntax (f stx) #`(let ((x 1)) #,stx))
>
> (define-syntax (m stx)
>   (syntax-case stx ()
>     ((_) #`(let ((x 2)) #,(f #'x))) ))
>
> (m)
>
>
> but I get error messages which I don't know what to do about. Do you
> have running examples of code not producing the result you want?
>
> Marijn
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk/z9PsACgkQp/VmCx0OL2xMNACbBJKq9nZJKTzCJbdnq0iPgJoE
> rSwAn2yJ1JTLMUn6v5ZR/PatxcGxgVrU
> =HFFX
> -----END PGP SIGNATURE-----
>

[-- Attachment #2: Type: text/html, Size: 2642 bytes --]

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hi!

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> You do not need gensyms if you try to mimic or implement my suggested #. .
> On the
> other hand when if you do this
>
> (define (f stx) #`(let ((x 1)) #,stx))
>
> and use this with
>
> #`(let ((x 2)) #,(f #'x))

OK, got it, thanks!

(In case others wonder, the complete example is:

  (define (f stx)
    #`(let ((x 1)) #,stx))
  (define-syntax foo
    (lambda (s)
      (syntax-case s ()
        ((_)
         #`(let ((x 2)) #,(f #'x))))))

  (foo)
  => 1
)

The situation can be diagnosed with:

  (define (f stx)
    #`(let ((x 1))
      #,(if (bound-identifier=? stx #'x) (error) stx)))

But it doesn’t help.

> (with-syntax ((x (datum->syntax stx (gensym "x")))) #`(let ((x 2)) #,(f
> #'x))))))

Often, you could use ‘generate-temporaries’, which is a bit nicer.

> Hope that this makes things clear!

It does, thanks!

It’s true that it’s annoying that the wrong binding is silently used.
Do you think it’s common enough to justify new syntax?

Thanks,
Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 2169 bytes --]

| It’s true that it’s annoying that the wrong binding is silently used.
| Do you think it’s common enough to justify new syntax?

Yes this highlights a comon problem when implementing racket match with #`.
I do think
that the best solution is to somehow extend the syntax expander to mimic my
sugested
#. and #.@. The simple solution is to rewrite according to

#`(... #.((x y) (f #'x #'y)))

->

#`(let-syntax ((g (lambda (x) (syntax-case x () ((_ x y) (f #'x #'y))))))
      (... (g x y))

But before going this path it would be interesting to hear what the true
wizards of
scheme have to say about this. It might have been considered already and
good
advice could be taken from them. I also feel that the issue needs to be
liffted up to the
community of at least syntax-case user crowd before doing anything Ill try
to spur
some discussions on it and come back later!

/Stefan

On Mon, Jul 9, 2012 at 5:52 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hi!
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> > You do not need gensyms if you try to mimic or implement my suggested #.
> .
> > On the
> > other hand when if you do this
> >
> > (define (f stx) #`(let ((x 1)) #,stx))
> >
> > and use this with
> >
> > #`(let ((x 2)) #,(f #'x))
>
> OK, got it, thanks!
>
> (In case others wonder, the complete example is:
>
>   (define (f stx)
>     #`(let ((x 1)) #,stx))
>   (define-syntax foo
>     (lambda (s)
>       (syntax-case s ()
>         ((_)
>          #`(let ((x 2)) #,(f #'x))))))
>
>   (foo)
>   => 1
> )
>
> The situation can be diagnosed with:
>
>   (define (f stx)
>     #`(let ((x 1))
>       #,(if (bound-identifier=? stx #'x) (error) stx)))
>
> But it doesn’t help.
>
> > (with-syntax ((x (datum->syntax stx (gensym "x")))) #`(let ((x 2)) #,(f
> > #'x))))))
>
> Often, you could use ‘generate-temporaries’, which is a bit nicer.
>
> > Hope that this makes things clear!
>
> It does, thanks!
>
> It’s true that it’s annoying that the wrong binding is silently used.
> Do you think it’s common enough to justify new syntax?
>
> Thanks,
> Ludo’.
>
>
>

[-- Attachment #2: Type: text/html, Size: 2892 bytes --]

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hi!

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> | It’s true that it’s annoying that the wrong binding is silently used.
> | Do you think it’s common enough to justify new syntax?
>
> Yes this highlights a comon problem when implementing racket match with #`.

Sure, but it’s not good-style.  ;-)  In general, ‘syntax-case’ is great,
but it’s low-level plumbing to be used with care, compared to
‘syntax-rules’.

> I do think
> that the best solution is to somehow extend the syntax expander to mimic my
> sugested
> #. and #.@. The simple solution is to rewrite according to
>
> #`(... #.((x y) (f #'x #'y)))
>
> ->
>
> #`(let-syntax ((g (lambda (x) (syntax-case x () ((_ x y) (f #'x #'y))))))
>       (... (g x y))

Unlike syntax-{quote,unquote,...}, #. has no non-syntax-prefixed
equivalent.  And what it does is non-trivial.  So I don’t feel
comfortable with this extension, FWIW.

[...]

> I also feel that the issue needs to be
> liffted up to the
> community of at least syntax-case user crowd before doing anything Ill try
> to spur
> some discussions on it and come back later!

Yes, this should be discussed on c.l.s or scheme-reports.

Thanks,
Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1952 bytes --]

I did miss something when trying in racket, it's a psyntax bug!

Look!

racket:
(define-for-syntax (f x) #`(let ((x 1)) #,x))
> (define-syntax (g x) (syntax-case x ()((_ y) #`(let ((x y)) #,(f #'x))))
)
> (g 4)
4

In guile,
scheme@(guile-user)> (define (f x) #`(let ((x 1)) #,x))
scheme@(guile-user)> (define-syntax g (lambda (x) (syntax-case x () ((_ y)
#`(let ((x y)) #,(f #'x))))))
scheme@(guile-user)> (g 4)
$1 = 1
!

I much prefer rackets version here. I will file a bug report and also try
to understand what spec that racket is following!


On Tue, Jul 10, 2012 at 10:24 AM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hi!
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> > | It’s true that it’s annoying that the wrong binding is silently used.
> > | Do you think it’s common enough to justify new syntax?
> >
> > Yes this highlights a comon problem when implementing racket match with
> #`.
>
> Sure, but it’s not good-style.  ;-)  In general, ‘syntax-case’ is great,
> but it’s low-level plumbing to be used with care, compared to
> ‘syntax-rules’.
>
> > I do think
> > that the best solution is to somehow extend the syntax expander to mimic
> my
> > sugested
> > #. and #.@. The simple solution is to rewrite according to
> >
> > #`(... #.((x y) (f #'x #'y)))
> >
> > ->
> >
> > #`(let-syntax ((g (lambda (x) (syntax-case x () ((_ x y) (f #'x #'y))))))
> >       (... (g x y))
>
> Unlike syntax-{quote,unquote,...}, #. has no non-syntax-prefixed
> equivalent.  And what it does is non-trivial.  So I don’t feel
> comfortable with this extension, FWIW.
>
> [...]
>
> > I also feel that the issue needs to be
> > liffted up to the
> > community of at least syntax-case user crowd before doing anything Ill
> try
> > to spur
> > some discussions on it and come back later!
>
> Yes, this should be discussed on c.l.s or scheme-reports.
>
> Thanks,
> Ludo’.
>

[-- Attachment #2: Type: text/html, Size: 2531 bytes --]

Re: Enhancement to the syntax system?

[Marijn]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10-07-12 15:35, Stefan Israelsson Tampe wrote:
> I did miss something when trying in racket, it's a psyntax bug!

I'm glad you're finally looking harder at the Racket behavior.

> racket: (define-for-syntax (f x) #`(let ((x 1)) #,x))
>> (define-syntax (g x) (syntax-case x ()((_ y) #`(let ((x y)) #,(f
>> #'x))))
> )
>> (g 4)
> 4

This code produces the same error for me with Racket which I reported
earlier, but I seem to have been using a non-release version. So,
switching to 5.2.1, I can report that indeed the code

#lang racket

(define-for-syntax (f x) #`(let ((x 1)) #,x))

(define-syntax (g x)
  (syntax-case x ()
    ((_ y) #`(let ((x y)) #,(f #'x)))))

(g 4)

produces 4, which makes sense to me. However when I retry my earlier
example

#lang racket

(define-for-syntax (f stx) #`(let ((x 1)) #,stx))

(define-syntax (m stx)
  (syntax-case stx ()
    ((_) #`(let ((x 2)) #,(f #'x))) ))

(m)

it now produces 1 instead of the expected 2.

It seems to me that both these results cannot be correct
simultaneously, but I'll await the experts' opinion on that.

Marijn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/8PXIACgkQp/VmCx0OL2zdfQCeOVJz/ECKExThGPJoqvRY3E41
gr0An0vSsuXK2B7xi0WT0LuzvVvhUbcy
=04xS
-----END PGP SIGNATURE-----

Re: [racket-dev] Enhancement to the syntax system?

[Eli Barzilay]

20 minutes ago, Marijn wrote:
> 
> It seems to me that both these results cannot be correct
> simultaneously, but I'll await the experts' opinion on that.

This does look weird:

  #lang racket
  (define-for-syntax (f stx) #`(let ([x 1]) #,stx))
  (define-syntax (m stx)
    (with-syntax ([zz (f #'x)]) #`(let ([x 2]) zz)))
  (m)

evaluates to 1, but if I change the first two "stx" names into "x"
*or* if I change the argument name for the macro to "x", then it
returns 2.

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: Enhancement to the syntax system?

[Matthew Flatt]

At Tue, 10 Jul 2012 10:51:57 -0400, Eli Barzilay wrote:
> 20 minutes ago, Marijn wrote:
> > 
> > It seems to me that both these results cannot be correct
> > simultaneously, but I'll await the experts' opinion on that.
> 
> This does look weird:
> 
>   #lang racket
>   (define-for-syntax (f stx) #`(let ([x 1]) #,stx))
>   (define-syntax (m stx)
>     (with-syntax ([zz (f #'x)]) #`(let ([x 2]) zz)))
>   (m)
> 
> evaluates to 1, but if I change the first two "stx" names into "x"
> *or* if I change the argument name for the macro to "x", then it
> returns 2.

It's natural --- but not correct --- to think that #` is responsible
for hygiene, in which case `(f #'x)' should keep the given `x' separate
from the `let'-bound `x' in the result.

Instead, hygiene is the responsibility of macro invocation, and

 #`(let ([x 1]) #,#'x)

is simply the same as

 #`(let ([x 1]) x)

and so

 (f #'x)

above is equivalent to

 #`(let ([x 1]) x)


If you change the example to

 #lang racket
 (begin-for-syntax 
  (define-syntax-rule (f body)
    #`(let ([x 1]) body)))
 (define-syntax (m stx)
   (with-syntax ([zz (f x)]) #`(let ([x 2]) zz)))
 (m)

so that `f' is used as a macro instead of a function, then you get 2,
since the macro-expansion of `(f x)' keeps the `x's separate.

_________________________
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Enhancement to the syntax system?

[Ludovic Courtès]

Hi,

Matthew Flatt <mflatt@cs.utah.edu> skribis:

> It's natural --- but not correct --- to think that #` is responsible
> for hygiene, in which case `(f #'x)' should keep the given `x' separate
> from the `let'-bound `x' in the result.

[...]

> If you change the example to
>
>  #lang racket
>  (begin-for-syntax 
>   (define-syntax-rule (f body)
>     #`(let ([x 1]) body)))
>  (define-syntax (m stx)
>    (with-syntax ([zz (f x)]) #`(let ([x 2]) zz)))
>  (m)
>
> so that `f' is used as a macro instead of a function, then you get 2,
> since the macro-expansion of `(f x)' keeps the `x's separate.

Interesting.  Thanks for the clarification and examples.

Ludo’.

Re: [racket-dev] Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

samth made a pointer to

http://srfi.schemers.org/srfi-72/srfi-72.html

It does not look like guile racket etc. have implemented this yet.

Am I wrong?

This is precisely what I'm after!

On Tue, Jul 10, 2012 at 5:26 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hi,
>
> Matthew Flatt <mflatt@cs.utah.edu> skribis:
>
> > It's natural --- but not correct --- to think that #` is responsible
> > for hygiene, in which case `(f #'x)' should keep the given `x' separate
> > from the `let'-bound `x' in the result.
>
> [...]
>
> > If you change the example to
> >
> >  #lang racket
> >  (begin-for-syntax
> >   (define-syntax-rule (f body)
> >     #`(let ([x 1]) body)))
> >  (define-syntax (m stx)
> >    (with-syntax ([zz (f x)]) #`(let ([x 2]) zz)))
> >  (m)
> >
> > so that `f' is used as a macro instead of a function, then you get 2,
> > since the macro-expansion of `(f x)' keeps the `x's separate.
>
> Interesting.  Thanks for the clarification and examples.
>
> Ludo’.
>
>

[-- Attachment #2: Type: text/html, Size: 1565 bytes --]

Re: [racket-dev] Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 3505 bytes --]

The question I posed was If it's possible to use srfi-72 in guile or
racket. It is indeed a wish
of mine that it's implemented because it will most certainly help me write
beutiful code because
that srfi cater to the coding style Ichoose to have. Without it one most
certainly need to use with-syntax to introduce the bindings in order to be
safe and avoid a multiple of possible syntactic loopholes as can see if you
read André van Tonder's

http://srfi.schemers.org/srfi-72/srfi-72.html

Anyway one does not need to change psyntax in order to come close to that
system. A more hacky
way is to use code like the one at the end of this document. It's a
reiteration and improvement on a
previous version. So with this hack I can write,

-------------
(use-modules (syntax unquote++))
(define (f x y) #`(let ((x 1)) (+ #,x #,y)))
(define (h y) ##`(let ((x 2)) #.((x) (f x y))))
(define-syntax g (lambda (x) (syntax-case x () ((_ y) ##`(let ((x y))
#.((x) (h x)))))))
scheme@(guile-user)> (g 3)
$2 = 5

--------------
In racket,
(define-for-syntax (f x y) #`(let ((x 1)) (+ #,x #,y)))
(define-for-syntax (h y) #`(let ((x 2)) #,(f #'x y))
(define-syntax (g stx) (syntax-case stx () ((_ y) #`(let ((x y)) #,(h
#'x)))))
> (g 3)
4
--------------

This shows that it was just luck previously when racket produced the
correct (for my intention) answer.
with srfi-72 a correct answer would have been produced. Without srfi-72 I
will then move over to use
##` and #. in my code because it will be easy to transfer later on if when
srfi-72 is available in some form.

/Stefan

(define-module (syntax unquote++)
  #:export (quasisyntax++ insyntax))

(define *s* (make-fluid '()))
(define *t* (make-fluid '()))

(define table (make-weak-key-hash-table))
(define (add-lambda lam)
  (let* ((id (gensym "id"))
         (x  (datum->syntax #'table id)))
    (hashq-set! table id lam)
    x))
(define (plexer x . l)
  (let ((lam (hashq-ref table x)))
    (apply lam l)))

(define (parse stx x)
  (syntax-case x (unsyntax insyntax unsyntax-splicing)
    ((unsyntax          . _) x)
    ((unsyntax-splicing . _) x)
    ((insyntax ((p ...) c ...))
     (with-syntax ((g (datum->syntax stx (gensym "g")))
                   (i (datum->syntax stx (gensym "i"))))
       (fluid-set! *s* (cons #'(g (lambda (x)
                                    (syntax-case x ()
                                      ((_ p ...) (plexer 'i #'p ...)))))
                             (fluid-ref *s*)))
       (fluid-set! *t* (cons #'(i (add-lambda
                                   (lambda (p ...) (begin c ...))))
                             (fluid-ref *t*)))
       #'(g p ...)))
    ((x . l)
     (cons (parse stx #'x) (parse stx #'l)))
    (x #'x)))

(define-syntax quasisyntax++
  (lambda (x)
    (syntax-case x ()
      ((_ y)
       (begin
         (fluid-set! *s* '())
         (fluid-set! *t* '())
         (with-syntax ((statement (parse x #'y))
                       (lets      (fluid-ref *s*))
                       (withs     (fluid-ref *t*)))
           #'(with-syntax withs
                   #`(let-syntax lets statement))))))))

(define (rg ch stream)
  (let ((x (read-char stream)))
    (cond
     ((eqv? x #\`)
      `(quasisyntax++ ,(read stream)))
     (#t
      (error "Wrong format of # reader extension")))))

(define (rg. ch stream) `(insyntax ,(read stream)))

(read-hash-extend #\# rg)
(read-hash-extend #\. rg.)

[-- Attachment #2: Type: text/html, Size: 3933 bytes --]

Re: [racket-dev] Enhancement to the syntax system?

[Eli Barzilay]

An hour and a half ago, Matthew Flatt wrote:
> 
> It's natural --- but not correct --- to think that #` is responsible
> for hygiene, in which case `(f #'x)' should keep the given `x'
> separate from the `let'-bound `x' in the result.
> 
> Instead, hygiene is the responsibility of macro invocation, and
> 
>  #`(let ([x 1]) #,#'x)
> 
> is simply the same as
> 
>  #`(let ([x 1]) x)
> 
> and so
> 
>  (f #'x)
> 
> above is equivalent to
> 
>  #`(let ([x 1]) x)
> 
> 
> If you change the example to
> 
>  #lang racket
>  (begin-for-syntax 
>   (define-syntax-rule (f body)
>     #`(let ([x 1]) body)))
>  (define-syntax (m stx)
>    (with-syntax ([zz (f x)]) #`(let ([x 2]) zz)))
>  (m)
> 
> so that `f' is used as a macro instead of a function, then you get 2,
> since the macro-expansion of `(f x)' keeps the `x's separate.

Yeah, I think that this kind of confusion is there, but it's easy (at
least "relatively easy") to build a mental model of how things work
and avoid such problems -- but the tricky bit here is that things
break when the `stx' is renamed to `x'.


50 minutes ago, Ryan Culpepper wrote:
> 
> The difference between
> 
> (define-for-syntax (f1 stx) #`(let ([x 1]) #,stx)
> 
> and
> 
> (define-for-syntax (f2 x) #`(let ([x 1]) #,x)
> 
> is that the 'x' in the template in the 'f2' is not bound-identifier=? to 
> the 'x' that appears in the template of 'm', because it has a rename 
> wrap due to the 'x' formal parameter of 'f2'. That is, 'f2' behaves 
> essentially the same as
> 
> (define-for-syntax (f2* x*) #`(let ([x* 1]) #,x*)

Yeah, I think that this is the more confusing bit.  (I suspected
something along this line, but didn't have time to figure out a good
explanation...)  So I think that the real confusion is actually a
combination of what Matthew pointed at in the above and this one,
making the result trickier than both.  In other words, I *think* that
the effect of the transformer's argument name makes it looks like the
#` *was* responsible for hygiene when it's actually just this point
that makes it happen...  (This is all "IIUC".)


> A likely mistake is to think that the wrap generated by the 'x' in
> 'f2' doesn't count because it happens before we get around to the
> "real" macro expansion that you care about. But that's not the case
> (at least in Racket).
> 
> A good rule of thumb is to never use local variable names in your
> macro implementation (including compile-time auxiliary functions)
> that also appear in the macro's template (including etc).

Yeah, and a subtle lesson here is that `stx' is a useful convention.
(I think that `x' is common in some guile code, so this would be a
point for them.)


> A related error is the "identifier used out of context" error that
> you get from, eg,
> 
> (define-syntax (m stx)
>    (let ([+ *])
>      #'(+ 1 2)))
> (m)  ;; => identifier used out of context in: +
> 
> The binding of '+' in the macro changes the meaning of the '+' in
> the template, even though the bindings exist at different
> phases. You could perhaps avoid this issue by changing the hygiene
> algorithm by adding a phase level to rename wraps and skipping
> different-phase rename wraps during resolution. I'm not sure if this
> is a good idea or if anyone has tried it.

(And this is what Matthew's last example gets by changing `f' to a
macro, right?  Also, Stefan posted a related message to the
scheme-reports list where he imlpemented some new #. thing which is
(roughly speaking) something that expands to a `let-syntax' and
therefore tries to do the same.)

In any case, it would be nice if the original example I posted (which
is a variant of what was discussed originally) would throw an error
instead of looking right in a way that is wrong...

-- 
          ((lambda (x) (x x)) (lambda (x) (x x)))          Eli Barzilay:
                    http://barzilay.org/                   Maze is Life!

Re: [racket-dev] Enhancement to the syntax system?

[Marijn Schouten (hkBst)]

On 07/10/2012 05:03 PM, Matthew Flatt wrote:
> At Tue, 10 Jul 2012 10:51:57 -0400, Eli Barzilay wrote:
>> 20 minutes ago, Marijn wrote:
>>>
>>> It seems to me that both these results cannot be correct
>>> simultaneously, but I'll await the experts' opinion on that.
>>
>> This does look weird:
>>
>>   #lang racket
>>   (define-for-syntax (f stx) #`(let ([x 1]) #,stx))
>>   (define-syntax (m stx)
>>     (with-syntax ([zz (f #'x)]) #`(let ([x 2]) zz)))
>>   (m)
>>
>> evaluates to 1, but if I change the first two "stx" names into "x"
>> *or* if I change the argument name for the macro to "x", then it
>> returns 2.
> 
> It's natural --- but not correct --- to think that #` is responsible
> for hygiene, in which case `(f #'x)' should keep the given `x' separate
> from the `let'-bound `x' in the result.
> 
> Instead, hygiene is the responsibility of macro invocation, and
> 
>  #`(let ([x 1]) #,#'x)
> 
> is simply the same as
> 
>  #`(let ([x 1]) x)
> 
> and so
> 
>  (f #'x)
> 
> above is equivalent to
> 
>  #`(let ([x 1]) x)

IIUC then you're saying that also all the following (fx #'x) and (fy
#'x) are equivalent to #`(let ((x 0)) x), but if you run this code then
you will get a 0 result only for (myy), contrary to what I would expect
based on the above explanation.

#lang racket

(define-for-syntax (fx x) #`(let ((x 0)) #,x))
(define-for-syntax (fy y) #`(let ((x 0)) #,y))

(define-syntax (mxx x)
  (syntax-case x () ((_) #`(let ((x 99)) #,(fx #'x)))))

(define-syntax (mxy x)
  (syntax-case x () ((_) #`(let ((x 99)) #,(fy #'x)))))

(define-syntax (myx y)
  (syntax-case y () ((_) #`(let ((x 99)) #,(fx #'x)))))

(define-syntax (myy y)
  (syntax-case y () ((_) #`(let ((x 99)) #,(fy #'x)))))

(mxx)
(mxy)
(myx)
(myy)

==>

99
99
99
0

Marijn

Re: Enhancement to the syntax system?

[Ryan Culpepper]

On 07/10/2012 10:51 AM, Eli Barzilay wrote:
> 20 minutes ago, Marijn wrote:
>>
>> It seems to me that both these results cannot be correct
>> simultaneously, but I'll await the experts' opinion on that.
>
> This does look weird:
>
>    #lang racket
>    (define-for-syntax (f stx) #`(let ([x 1]) #,stx))
>    (define-syntax (m stx)
>      (with-syntax ([zz (f #'x)]) #`(let ([x 2]) zz)))
>    (m)
>
> evaluates to 1, but if I change the first two "stx" names into "x"
> *or* if I change the argument name for the macro to "x", then it
> returns 2.

The difference between

(define-for-syntax (f1 stx) #`(let ([x 1]) #,stx)

and

(define-for-syntax (f2 x) #`(let ([x 1]) #,x)

is that the 'x' in the template in the 'f2' is not bound-identifier=? to 
the 'x' that appears in the template of 'm', because it has a rename 
wrap due to the 'x' formal parameter of 'f2'. That is, 'f2' behaves 
essentially the same as

(define-for-syntax (f2* x*) #`(let ([x* 1]) #,x*)

A likely mistake is to think that the wrap generated by the 'x' in 'f2' 
doesn't count because it happens before we get around to the "real" 
macro expansion that you care about. But that's not the case (at least 
in Racket).

A good rule of thumb is to never use local variable names in your macro 
implementation (including compile-time auxiliary functions) that also 
appear in the macro's template (including etc).

A related error is the "identifier used out of context" error that you 
get from, eg,

(define-syntax (m stx)
   (let ([+ *])
     #'(+ 1 2)))
(m)  ;; => identifier used out of context in: +

The binding of '+' in the macro changes the meaning of the '+' in the 
template, even though the bindings exist at different phases. You could 
perhaps avoid this issue by changing the hygiene algorithm by adding a 
phase level to rename wraps and skipping different-phase rename wraps 
during resolution. I'm not sure if this is a good idea or if anyone has 
tried it.

Ryan
_________________________
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: Enhancement to the syntax system?

[Ludovic Courtès]

Hi,

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> racket:
> (define-for-syntax (f x) #`(let ((x 1)) #,x))
>> (define-syntax (g x) (syntax-case x ()((_ y) #`(let ((x y)) #,(f #'x))))
> )
>> (g 4)
> 4
>
> In guile,
> scheme@(guile-user)> (define (f x) #`(let ((x 1)) #,x))
> scheme@(guile-user)> (define-syntax g (lambda (x) (syntax-case x () ((_ y)
> #`(let ((x y)) #,(f #'x))))))
> scheme@(guile-user)> (g 4)
> $1 = 1
> !
>
> I much prefer rackets version here. I will file a bug report and also try
> to understand what spec that racket is following!

Aaah, indeed, that’s a “better” outcome!  :-)

Ludo’.

Re: Enhancement to the syntax system?

[Stefan Israelsson Tampe]

[-- Attachment #1.1: Type: text/plain, Size: 2725 bytes --]

Ok, I coded a suggested syntax that is a quick solution but maybe not ideal
because really this should be hanlded on the psyntax level. We probably
cannot use the old syntax because it can break old code in other words one
has to introduce new language elements. Also I asked on #scheme and tried
to look into reference documentation for quasisyntax semantics. The answer
I got on #scheme was that psyntax is buggy although I do not agree on that.
I also felt that people is suspicious of my approach which is to write one
macro and then dispatch to functions in stead of writing macro after macro.
In all I don't think I am able to lift this idea to a larger community.

So I sat down and implemented a simple algorithm see attachement. with this
I can write,

scheme@(guile-user)> (use-modules (syntax unquote++))
scheme@(guile-user)>
(define (f x)
  #`(let ((x 1)) #,x))

scheme@(guile-user)>
(define-syntax g
  (lambda (x)
    (syntax-case x ()
      ((_ y)
        ##`(let ((x y))
            #.((x) (f #'x)))))))

scheme@(guile-user)> (g 4)
$1 = 4

This will explicitly state what variables to transfer to the inserted
expression. It's not ideal because I
feel that the good way to handle this is to implicitly, which demands
hacking psyntax, do this and simplify the syntax even more. also I cannot
see how to implement #.@ but to hack psyntax.

##` also allow #, and #,@ which has the old meaning.

Have fun
/Stefan

On Mon, Jul 9, 2012 at 5:52 PM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hi!
>
> Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:
>
> > You do not need gensyms if you try to mimic or implement my suggested #.
> .
> > On the
> > other hand when if you do this
> >
> > (define (f stx) #`(let ((x 1)) #,stx))
> >
> > and use this with
> >
> > #`(let ((x 2)) #,(f #'x))
>
> OK, got it, thanks!
>
> (In case others wonder, the complete example is:
>
>   (define (f stx)
>     #`(let ((x 1)) #,stx))
>   (define-syntax foo
>     (lambda (s)
>       (syntax-case s ()
>         ((_)
>          #`(let ((x 2)) #,(f #'x))))))
>
>   (foo)
>   => 1
> )
>
> The situation can be diagnosed with:
>
>   (define (f stx)
>     #`(let ((x 1))
>       #,(if (bound-identifier=? stx #'x) (error) stx)))
>
> But it doesn’t help.
>
> > (with-syntax ((x (datum->syntax stx (gensym "x")))) #`(let ((x 2)) #,(f
> > #'x))))))
>
> Often, you could use ‘generate-temporaries’, which is a bit nicer.
>
> > Hope that this makes things clear!
>
> It does, thanks!
>
> It’s true that it’s annoying that the wrong binding is silently used.
> Do you think it’s common enough to justify new syntax?
>
> Thanks,
> Ludo’.
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 3457 bytes --]

[-- Attachment #2: unquote++.scm --]
[-- Type: application/octet-stream, Size: 1237 bytes --]

(define-module (syntax unquote++)
  #:export (quasisyntax++ insyntax))

(define *s* (make-fluid '()))

(define (parse stx x)
  (syntax-case x (unsyntax insyntax unsyntax-splicing)
    ((unsyntax          . _) x)
    ((unsyntax-splicing . _) x)
    ((insyntax ((p ...) c ...))
     (with-syntax ((g (datum->syntax stx (gensym "g"))))
       (fluid-set! *s* (cons #'(g (lambda (x) 
                                    (syntax-case x () 
                                      ((_ p ...) (begin c ...)))))
                             (fluid-ref *s*)))
       #'(g p ...)))
    ((x . l)
     (cons (parse stx #'x) (parse stx #'l)))
    (x #'x)))

(define-syntax quasisyntax++
  (lambda (x)
    (syntax-case x ()
      ((_ y)
       (begin
         (fluid-set! *s* '())
         (with-syntax ((statement (parse x #'y))
                       (lets      (fluid-ref *s*)))
           #'(quasisyntax (let-syntax lets statement))))))))
               
(define (rg ch stream)
  (let ((x (read-char stream)))
    (cond
     ((eqv? x #\`)
      `(quasisyntax++ ,(read stream)))
     (#t
      (error "Wrong format of # reader extension")))))

(define (rg. ch stream) `(insyntax ,(read stream)))

(read-hash-extend #\# rg)
(read-hash-extend #\. rg.)