problems with syntax-case and with-syntax

problems with syntax-case and with-syntax

[Matt Wette]

Q1) The code below creates two macros.  One called `define-foo' which generates a new identifier and 
then defines that to #t.  The other, `define-foo/p', generates the same identifier (lexical issue?) 
and another identifier, then "calls" define-foo and then uses both identifiers in a `define'.  When
executed I get this error:

scheme@(guile-user)> (define-foo/p abc)
;;; <stdin>:2:0: warning: possibly unbound variable `wrap-abc'
<unnamed port>:2:0: <unnamed port>:2:0: In procedure module-lookup: Unbound variable: wrap-abc

What am I doing wrong here?


Q2) Also with respect to the code below.  Is there any way to pull the definitions for stx->str
and gen-id out of the define-syntax body to make them general purpose?


This code being developed under guile-2.2.2.

The code:

(define-syntax define-foo
  (lambda (x)
    (define (stx->str stx)
      (symbol->string (syntax->datum stx)))
    (define (gen-id tmpl-id . args)
      (datum->syntax
       tmpl-id
       (string->symbol
	(apply string-append
	       (map (lambda (ss) (if (string? ss) ss (stx->str ss))) args)))))
    (syntax-case x ()
      ((_ name)
       (with-syntax ((wrap (gen-id x "wrap-" #'name)))
	 #'(begin
	     (define wrap #t)))))))

(define-syntax define-foo/p
  (lambda (x)
    (define (stx->str stx)
      (symbol->string (syntax->datum stx)))
    (define (gen-id tmpl-id . args)
      (datum->syntax
       tmpl-id
       (string->symbol
	(apply string-append
	       (map (lambda (ss) (if (string? ss) ss (stx->str ss))) args)))))
    (syntax-case x ()
      ((_ name)
       (with-syntax ((wrap (gen-id x "wrap-" #'name))
		     (rewrap (gen-id x "rewrap-" #'name)))
	 #'(begin
	     (define-foo name)
	     (define rewrap wrap)))))))

;;(define-foo/p abc)
;; expect:
;; (define wrap-abc #t)
;; (define rewrap-abc wrap-abc)

;;(define-foo abc) ;; works (but does not define rewrap-abc
;; expect:
;; (define wrap-abc #t)


Thanks for the help,
Matt

Re: problems with syntax-case and with-syntax

[Mark H Weaver]

Matt Wette <matt.wette@gmail.com> writes:

> Q1) The code below creates two macros.  One called `define-foo' which generates a new identifier and 
> then defines that to #t.  The other, `define-foo/p', generates the same identifier (lexical issue?) 
> and another identifier, then "calls" define-foo and then uses both identifiers in a `define'.  When
> executed I get this error:
>
> scheme@(guile-user)> (define-foo/p abc)
> ;;; <stdin>:2:0: warning: possibly unbound variable `wrap-abc'
> <unnamed port>:2:0: <unnamed port>:2:0: In procedure module-lookup: Unbound variable: wrap-abc
>
> What am I doing wrong here?

The problem is that in Guile 2.2, whenever (define <id> ...) is found in
the expanded code, where <id> was introduced by a macro (i.e. not passed
as an explicit argument to the macro), Guile will rewrite the <id> into
a new name based on the hash of the entire definition form.

I don't know of any way to make this work without passing 'wrap-abc'
explicitly as an argument to the 'define-foo' macro.

FWIW, I've always been opposed to these non-standard semantics, but they
were included in Guile 2.2 over my strenuous objections:

  https://lists.gnu.org/archive/html/guile-devel/2014-01/msg00061.html
  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00021.html
  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00042.html

> Q2) Also with respect to the code below.  Is there any way to pull the
> definitions for stx->str and gen-id out of the define-syntax body to
> make them general purpose?

Yes, you can wrap the definitions of 'stx->str' within 'eval-when', like
this:

  (eval-when (expand load eval)
    (define (stx->str stx)
      ...)
    (define (gen-id tmpl-id . args)
      ...))

above the macro definitions that use them.

One more thing:

> (define-syntax define-foo
>   (lambda (x)
>     (define (stx->str stx)
>       (symbol->string (syntax->datum stx)))
>     (define (gen-id tmpl-id . args)
>       (datum->syntax
>        tmpl-id
>        (string->symbol
> 	(apply string-append
> 	       (map (lambda (ss) (if (string? ss) ss (stx->str ss))) args)))))
>     (syntax-case x ()
>       ((_ name)
>        (with-syntax ((wrap (gen-id x "wrap-" #'name)))
> 	 #'(begin
> 	     (define wrap #t)))))))

Here, the 'tmpl-id' that is being passed to 'datum-syntax' is not
actually an identifier, but rather a compound syntax object.  Although
we do not currently raise an error in this case (we probably should), it
is against the spec and likely to cause problems in the future, if not
today.  You should pass an actual identifier as 'tmpl-id'.

     Regards,
       Mark

Re: problems with syntax-case and with-syntax

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> The problem is that in Guile 2.2, whenever (define <id> ...) is found in
> the expanded code, where <id> was introduced by a macro (i.e. not passed
> as an explicit argument to the macro), Guile will rewrite the <id> into
> a new name based on the hash of the entire definition form.

I forgot to mention that only top-level definitions are munged in this
way.

Also, my parenthetical definition of what it means to be "introduced by
a macro" lacked precision.  To avoid <id> being "introduced by a macro",
it's not enough for <id> to have been passed an argument to the macro
that generated the definition.  If that were the case, you could work
around this by adding an additional layer of macros, where the upper
layer generated <id> and passed it down to the lower layer which would
generate the definition.

To avoid <id> being considered "introduced by a macro", <id> must
ultimately occur verbatim in the source code outside of any macro
template.

     Mark

Re: problems with syntax-case and with-syntax

[Matt Wette]

> On Aug 27, 2017, at 5:36 PM, Mark H Weaver <mhw@netris.org> wrote:
> 
> Matt Wette <matt.wette@gmail.com> writes:
> 
>> Q1) The code below creates two macros.  One called `define-foo' which generates a new identifier and 
>> then defines that to #t.  The other, `define-foo/p', generates the same identifier (lexical issue?) 
>> and another identifier, then "calls" define-foo and then uses both identifiers in a `define'.  When
>> executed I get this error:
>> 
>> scheme@(guile-user)> (define-foo/p abc)
>> ;;; <stdin>:2:0: warning: possibly unbound variable `wrap-abc'
>> <unnamed port>:2:0: <unnamed port>:2:0: In procedure module-lookup: Unbound variable: wrap-abc
>> 
>> What am I doing wrong here?
> 
> The problem is that in Guile 2.2, whenever (define <id> ...) is found in
> the expanded code, where <id> was introduced by a macro (i.e. not passed
> as an explicit argument to the macro), Guile will rewrite the <id> into
> a new name based on the hash of the entire definition form.
> 
> I don't know of any way to make this work without passing 'wrap-abc'
> explicitly as an argument to the 'define-foo' macro.
> 
> FWIW, I've always been opposed to these non-standard semantics, but they
> were included in Guile 2.2 over my strenuous objections:
> 
>  https://lists.gnu.org/archive/html/guile-devel/2014-01/msg00061.html
>  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00021.html
>  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00042.html
> 
>> Q2) Also with respect to the code below.  Is there any way to pull the
>> definitions for stx->str and gen-id out of the define-syntax body to
>> make them general purpose?
> 
> Yes, you can wrap the definitions of 'stx->str' within 'eval-when', like
> this:
> 
>  (eval-when (expand load eval)
>    (define (stx->str stx)
>      ...)
>    (define (gen-id tmpl-id . args)
>      ...))
> 
> above the macro definitions that use them.
> 
> One more thing:
> 
>> (define-syntax define-foo
>>  (lambda (x)
>>    (define (stx->str stx)
>>      (symbol->string (syntax->datum stx)))
>>    (define (gen-id tmpl-id . args)
>>      (datum->syntax
>>       tmpl-id
>>       (string->symbol
>> 	(apply string-append
>> 	       (map (lambda (ss) (if (string? ss) ss (stx->str ss))) args)))))
>>    (syntax-case x ()
>>      ((_ name)
>>       (with-syntax ((wrap (gen-id x "wrap-" #'name)))
>> 	 #'(begin
>> 	     (define wrap #t)))))))
> 
> Here, the 'tmpl-id' that is being passed to 'datum-syntax' is not
> actually an identifier, but rather a compound syntax object.  Although
> we do not currently raise an error in this case (we probably should), it
> is against the spec and likely to cause problems in the future, if not
> today.  You should pass an actual identifier as 'tmpl-id'.
> 
>     Regards,
>       Mark
> 

Thanks.  Things look much better now.

I just added a case for define-foo syntax with an extra arg for wrap, so I can pass it.
I wrapped gen-id in the eval-when.  (Yay!)
I replaced `(gen-id x "wrap-" #'name)' with `(gen-id #'name "wrap-" #'name)'.
(I used to write the last step that way but picked up a bad habit at some time.)

Matt

Re: problems with syntax-case and with-syntax

[Matt Wette]

> On Aug 27, 2017, at 5:36 PM, Mark H Weaver <mhw@netris.org> wrote:
> 
> FWIW, I've always been opposed to these non-standard semantics, but they
> were included in Guile 2.2 over my strenuous objections:
> 
>  https://lists.gnu.org/archive/html/guile-devel/2014-01/msg00061.html
>  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00021.html
>  https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00042.html <https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00042.html>

I look forward to reading through this thread.

Matt

Re: problems with syntax-case and with-syntax

[Chris Vine]

On Sun, 27 Aug 2017 20:36:53 -0400
Mark H Weaver <mhw@netris.org> wrote:
> Matt Wette <matt.wette@gmail.com> writes:
> 
> > Q1) The code below creates two macros.  One called `define-foo'
> > which generates a new identifier and then defines that to #t.  The
> > other, `define-foo/p', generates the same identifier (lexical
> > issue?) and another identifier, then "calls" define-foo and then
> > uses both identifiers in a `define'.  When executed I get this
> > error:
> >
> > scheme@(guile-user)> (define-foo/p abc)
> > ;;; <stdin>:2:0: warning: possibly unbound variable `wrap-abc'
> > <unnamed port>:2:0: <unnamed port>:2:0: In procedure module-lookup:
> > Unbound variable: wrap-abc
> >
> > What am I doing wrong here?  
> 
> The problem is that in Guile 2.2, whenever (define <id> ...) is found
> in the expanded code, where <id> was introduced by a macro (i.e. not
> passed as an explicit argument to the macro), Guile will rewrite the
> <id> into a new name based on the hash of the entire definition form.
> 
> I don't know of any way to make this work without passing 'wrap-abc'
> explicitly as an argument to the 'define-foo' macro.
> 
> FWIW, I've always been opposed to these non-standard semantics, but
> they were included in Guile 2.2 over my strenuous objections:
> 
>   https://lists.gnu.org/archive/html/guile-devel/2014-01/msg00061.html
>   https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00021.html
>   https://lists.gnu.org/archive/html/guile-devel/2011-11/msg00042.html

How interesting.

There seems very little uniformity in how the different schemes which I
happen to have available deal with the issue of top level hygiene with
nested macros.  Given the example in paragraph 6.10.10 of the guile-2.2
manual, after the definitions of both 'foo' and 'bar' have both been
made the following schemes evaluate '(foo)' as 37:

  guile-2.0, chicken;

and the following evaluate it as 42:

  guile-2.2, chez, kawa.

Chris

Re: problems with syntax-case and with-syntax

[Matt Wette]

> On Aug 27, 2017, at 6:35 PM, Mark H Weaver <mhw@netris.org> wrote:
> 
> Mark H Weaver <mhw@netris.org> writes:
> 
>> The problem is that in Guile 2.2, whenever (define <id> ...) is found in
>> the expanded code, where <id> was introduced by a macro (i.e. not passed
>> as an explicit argument to the macro), Guile will rewrite the <id> into
>> a new name based on the hash of the entire definition form.
> 
> I forgot to mention that only top-level definitions are munged in this
> way.
> 
> Also, my parenthetical definition of what it means to be "introduced by
> a macro" lacked precision.  To avoid <id> being "introduced by a macro",
> it's not enough for <id> to have been passed an argument to the macro
> that generated the definition.  If that were the case, you could work
> around this by adding an additional layer of macros, where the upper
> layer generated <id> and passed it down to the lower layer which would
> generate the definition.
> 
> To avoid <id> being considered "introduced by a macro", <id> must
> ultimately occur verbatim in the source code outside of any macro
> template.

I have read through the posts, and the Guile 2.2 ref manual.  The explanations
are not quite complete in my mind.  If all top-level id's introduced by macros
were munged, then it would break a lot of existing code.  See, for example,
the `define-structure' example in "The Scheme Programming Language", 4th ed.
It seems identifiers introduced by datum->syntax are preserved, as long 
as they are not redefined.  Is that correct?

In my case, I was redefining by architecture (or convention). I was generating 
"wrap-" + <identifier> in a macro that called a another macro that made the same 
definition.  Is it bad form to assume an convention like this?

Off to do more reading on this: Dybvig's paper on syntax-case and I have the 
book too.  and R6RS ...

Matt

Re: problems with syntax-case and with-syntax

[Matt Wette]

> On Sep 18, 2017, at 6:16 AM, Matt Wette <matt.wette@gmail.com> wrote:
> 
>> 
>> On Aug 27, 2017, at 6:35 PM, Mark H Weaver <mhw@netris.org> wrote:
>> 
>> Mark H Weaver <mhw@netris.org> writes:
>> 
>>> The problem is that in Guile 2.2, whenever (define <id> ...) is found in
>>> the expanded code, where <id> was introduced by a macro (i.e. not passed
>>> as an explicit argument to the macro), Guile will rewrite the <id> into
>>> a new name based on the hash of the entire definition form.
>> 
>> I forgot to mention that only top-level definitions are munged in this
>> way.
>> 
>> Also, my parenthetical definition of what it means to be "introduced by
>> a macro" lacked precision.  To avoid <id> being "introduced by a macro",
>> it's not enough for <id> to have been passed an argument to the macro
>> that generated the definition.  If that were the case, you could work
>> around this by adding an additional layer of macros, where the upper
>> layer generated <id> and passed it down to the lower layer which would
>> generate the definition.
>> 
>> To avoid <id> being considered "introduced by a macro", <id> must
>> ultimately occur verbatim in the source code outside of any macro
>> template.
> 
> I have read through the posts, and the Guile 2.2 ref manual.  The explanations
> are not quite complete in my mind.  If all top-level id's introduced by macros
> were munged, then it would break a lot of existing code.  See, for example,
> the `define-structure' example in "The Scheme Programming Language", 4th ed.
> It seems identifiers introduced by datum->syntax are preserved, as long 
> as they are not redefined.  Is that correct?
> 
> In my case, I was redefining by architecture (or convention). I was generating 
> "wrap-" + <identifier> in a macro that called a another macro that made the same 
> definition.  Is it bad form to assume an convention like this?
> 
> Off to do more reading on this: Dybvig's paper on syntax-case and I have the 
> book too.  and R6RS ...

I have been convinced that introducing top-level definitions is bad form, so I will 
be removing datum->syntax calls but stuffing some procedures into the associated 
struct, I think.  So instead of

  (define-fh-type foo_t)
  ...
  (unwrap-foo_t obj)

I will use

  (define-fh-type foo_t
     foo_t? make-foo_t)
  ...
  (fh-unwrap foo_t obj)

Matt