Need help to understand a macro

Need help to understand a macro

[Josef Wolf]

Hello,

I am trying to understand the defstruct macro from the "teach yourself
scheme in fixnum days" tutorial, which can be found in chapter 9 at
http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-11.html#node_chap_9

My first question is of a more generic type. For a better understanding
how macros work, I'd like to have a way to show what expansion a macro
would generate when it would be used. E.g., I'd like to do something
like

 (show-expansion (defstruct tree height girth age leaf-shape leaf-color))

and get what the expansion of this macro would produce:

 (begin
   (define make-tree
     (lambda fvfv
       [ ... and so on ... ]

Of course this is possible. After all, code is data and data is code.

My next question is more related to the defstruct macro.
In line 11, defstruct stores the default initializers into the vv vector:

   (if (pair? f) (cadr f) '(if #f #f)))

So if the field is a pair, the initializer is stored in vv. That's easy.
But if it is not a pair, '(if #f #f) is stored. What is this good for?
This 'if' would evaluate the 'else' part, which does not exist. So we
would get #f as a result. So why not storing #f in the first place? Why
is not

   (if (pair? f) (cadr f) #f))

used here?

Re: Need help to understand a macro

[Ken Raeburn]

On Mar 19, 2010, at 04:57, Josef Wolf wrote:
> My next question is more related to the defstruct macro.
> In line 11, defstruct stores the default initializers into the vv vector:
> 
>   (if (pair? f) (cadr f) '(if #f #f)))
> 
> So if the field is a pair, the initializer is stored in vv. That's easy.
> But if it is not a pair, '(if #f #f) is stored. What is this good for?
> This 'if' would evaluate the 'else' part, which does not exist. So we
> would get #f as a result. So why not storing #f in the first place? Why
> is not
> 
>   (if (pair? f) (cadr f) #f))
> 
> used here?

The result of (if #f #f) is unspecified, not #f, according to r5rs.  That means an implementation can produce whatever value it wants.

Guile has a special "unspecified" value which is distinct from #f and other values, for use when a function's return value is unspecified; in some ways this is probably better than picking something like #f, as it doesn't cause people unfamiliar with the distinction between a language specification and a language implementation to start assuming that #f (or whatever) is the value that Scheme always requires in that case.  Among other things, the unspecified value causes the REPL to not print a result:

guile> (if #f #f #f)
#f
guile> (if #f #f)
guile>

Using (if #f #f) here is basically saying, "the initial value is unspecified", instead of defaulting to #f or #t or 0 or '() or any other particular normal value.

Ken

Re: Need help to understand a macro

[Josef Wolf]

On Fri, Mar 19, 2010 at 08:54:02AM -0400, Ken Raeburn wrote:
> On Mar 19, 2010, at 04:57, Josef Wolf wrote:
> > My next question is more related to the defstruct macro.
> > In line 11, defstruct stores the default initializers into the vv vector:
> > 
> >   (if (pair? f) (cadr f) '(if #f #f)))
> > 
> > So if the field is a pair, the initializer is stored in vv. That's easy.
> > But if it is not a pair, '(if #f #f) is stored. What is this good for?
> > This 'if' would evaluate the 'else' part, which does not exist. So we
> > would get #f as a result. So why not storing #f in the first place? Why
> > is not
> > 
> >   (if (pair? f) (cadr f) #f))
> > 
> > used here?
> 
> The result of (if #f #f) is unspecified, not #f, according to r5rs.  That means an implementation can produce whatever value it wants.
> 
> Guile has a special "unspecified" value which is distinct from #f and other values, for use when a function's return value is unspecified; in some ways this is probably better than picking something like #f, as it doesn't cause people unfamiliar with the distinction between a language specification and a language implementation to start assuming that #f (or whatever) is the value that Scheme always requires in that case.  Among other things, the unspecified value causes the REPL to not print a result:
> 
> guile> (if #f #f #f)
> #f
> guile> (if #f #f)
> guile>
> 
> Using (if #f #f) here is basically saying, "the initial value is unspecified", instead of defaulting to #f or #t or 0 or '() or any other particular normal value.

I think I like this type of "unspecified". Much better than the "undefined
behavior" definition in C.

Thanks for the explanation, Ken!

Re: Need help to understand a macro

[Andy Wingo]

Hi Josef,

I seem to be the negative guy in replies to you. Apologies for that!

On Mon 22 Mar 2010 20:25, Josef Wolf <jw@raven.inka.de> writes:

> On Fri, Mar 19, 2010 at 08:54:02AM -0400, Ken Raeburn wrote:
>> 
>> The result of (if #f #f) is unspecified, not #f, according to r5rs.
>> That means an implementation can produce whatever value it wants.

In the R6RS, evaluating `(if #f #f)' returns "unspecified values" --
that is, even the number of values is unspecified. And in fact it would
make sense for `(if #f #f)' to be the same as `(values)' -- an
expression returning zero values.

> I think I like this type of "unspecified". Much better than the
> "undefined behavior" definition in C.

Unfortunately it really is unspecified :) OK it's better than C, in the
sense that it won't launch the missiles, but it would be better if
evaluating:

   (+ 2 (if #f #f))

yielded an error of "too few values to continuation" rather than "don't
know how to add #<unspecified>".

Cheers,

Andy
-- 
http://wingolog.org/

Re: Need help to understand a macro

[Josef Wolf]

On Mon, Mar 22, 2010 at 09:16:34PM +0100, Andy Wingo wrote:
> Hi Josef,
> 
> I seem to be the negative guy in replies to you.

Uh? I did not notice anything negative?

> Apologies for that!

I don't see any reason for you to apologize...

> On Mon 22 Mar 2010 20:25, Josef Wolf <jw@raven.inka.de> writes:
> 
> > On Fri, Mar 19, 2010 at 08:54:02AM -0400, Ken Raeburn wrote:
> >> 
> >> The result of (if #f #f) is unspecified, not #f, according to r5rs.
> >> That means an implementation can produce whatever value it wants.
> 
> In the R6RS, evaluating `(if #f #f)' returns "unspecified values" --
> that is, even the number of values is unspecified. And in fact it would
> make sense for `(if #f #f)' to be the same as `(values)' -- an
> expression returning zero values.

As I understood this, the "no values" variant is a guile extension and not
guaranteed by the standard...

> > I think I like this type of "unspecified". Much better than the
> > "undefined behavior" definition in C.
> 
> Unfortunately it really is unspecified :) OK it's better than C, in the
> sense that it won't launch the missiles, but it would be better if
> evaluating:
> 
>    (+ 2 (if #f #f))
> 
> yielded an error of "too few values to continuation" rather than "don't
> know how to add #<unspecified>".

Even better would be "too few values for '+'" or something. Oh, and a line
number.

Re: Need help to understand a macro

[Andy Wingo]

Hi,

On Fri 19 Mar 2010 09:57, Josef Wolf <jw@raven.inka.de> writes:

>  (show-expansion (defstruct tree height girth age leaf-shape leaf-color))

(macroexpand '(defstruct ...))

You might want to surround that with (pretty-print ...) from (ice-9
pretty-print).

> Of course this is possible. After all, code is data and data is code.

Sorta. Syntax-rules/syntax-case macros operate on syntax objects, not on
raw s-expressions. This allows them to preserve referential
transparency. So the result of expansion might not look like what you
would expect.

Read up on syntax-rules and syntax-case macros, when you get some time.
I used to love defmacros but I don't use them any more.

http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=doc/ref/api-macros.texi;h=51f54ed070af453a138282f61e7cd8dbbddc53a3;hb=HEAD

Happy hacking,

Andy
-- 
http://wingolog.org/

Re: Need help to understand a macro

[Andreas Rottmann]

Andy Wingo <wingo@pobox.com> writes:

> Hi,
>
> On Fri 19 Mar 2010 09:57, Josef Wolf <jw@raven.inka.de> writes:
>
>>  (show-expansion (defstruct tree height girth age leaf-shape leaf-color))
>
> (macroexpand '(defstruct ...))
>
> You might want to surround that with (pretty-print ...) from (ice-9
> pretty-print).
>
>> Of course this is possible. After all, code is data and data is code.
>
> Sorta. Syntax-rules/syntax-case macros operate on syntax objects, not on
> raw s-expressions. This allows them to preserve referential
> transparency. So the result of expansion might not look like what you
> would expect.
>
> Read up on syntax-rules and syntax-case macros, when you get some time.
> I used to love defmacros but I don't use them any more.
>
defmacros are bound to bite you in the proverbial ass real hard when you
combine them with modules, so it's best to avoid them like the
plague. At least that's my experience, FWIW.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Need help to understand a macro

[Josef Wolf]

On Fri, Mar 19, 2010 at 05:15:57PM +0100, Andreas Rottmann wrote:
> defmacros are bound to bite you in the proverbial ass real hard when you
> combine them with modules, so it's best to avoid them like the
> plague. At least that's my experience, FWIW.

Thanks for the warning. I'd like to understand why this is dangerous, though.
Do you have an example for me?

Do common-lisp macros suffer from the same problems or is this a scheme
specific problem?

Please note that I am very new to lisp, so please don't beat me if I ask
dumb questions :-)

Re: Need help to understand a macro

[Andreas Rottmann]

Josef Wolf <jw@raven.inka.de> writes:

> On Fri, Mar 19, 2010 at 05:15:57PM +0100, Andreas Rottmann wrote:
>> defmacros are bound to bite you in the proverbial ass real hard when you
>> combine them with modules, so it's best to avoid them like the
>> plague. At least that's my experience, FWIW.
>
> Thanks for the warning. I'd like to understand why this is dangerous, though.
> Do you have an example for me?
>
Yes, consider the following two modules:

;;---------------8<----------------
(define-module (test a)
  #:export-syntax (test-a)
  #:use-module (srfi srfi-34)
  #:use-module (srfi srfi-35))

(define (funny-condition? c)
  (and (message-condition? c)
       (string-prefix? "FUNNY: " (condition-message c))))

(define-syntax test-a
  (syntax-rules ()
    ((test-a expr)
     (guard (c ((funny-condition? c) 'funny))
       expr))))

(define-module (test b)
  #:use-module (srfi srfi-34)
  #:use-module (srfi srfi-35)
  #:use-module (test a))

(test-a
 (raise (condition (&message (message (string-append "FUNNY: "))))))
;;---------------8<----------------

Now running it causes this:

    WARNING: (test b): imported module (srfi srfi-34) overrides core binding `raise'
    Throw to key `unbound-variable':
    ERROR: In procedure module-lookup:
    ERROR: Unbound variable: funny-condition?
    Entering the debugger. Type `bt' for a backtrace or `c' to continue.

Now why doesn't this work, and return 'funny in module B's body? It's
because currently, the `guard' macro from SRFI-34 is implemented with
`define-macro', and it hence throws away (among other things) the
information what `funny-condition?' refers to when being expanded. 

The fully expanded code of the `test-a' form now contains a free
reference to `funny-condition?', while it should have a bound reference
referring to the `funny-condition?' defined in module (test a). 

As you can see, the use of a single defmacro can poison your otherwise
perfectly fine hygienic, referentially transparent, syntax-rules macros.

> Do common-lisp macros suffer from the same problems or is this a scheme
> specific problem?
>
I've just superficial knowledge of CL, but AFAIK, Common Lisp macros
still have similiar issues, but they are (largely) mitigated by the fact
that CL has a function namespace separate from the variable namespace.

> Please note that I am very new to lisp, so please don't beat me if I ask
> dumb questions :-)
>
They were actually very reasonable questions!

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Need help to understand a macro

[Josef Wolf]

On Fri, Mar 19, 2010 at 03:28:39PM +0100, Andy Wingo wrote:
> Hi,
> 
> On Fri 19 Mar 2010 09:57, Josef Wolf <jw@raven.inka.de> writes:
> 
> >  (show-expansion (defstruct tree height girth age leaf-shape leaf-color))
> 
> (macroexpand '(defstruct ...))
> 
> You might want to surround that with (pretty-print ...) from (ice-9
> pretty-print).
> 
> > Of course this is possible. After all, code is data and data is code.
> 
> Sorta. Syntax-rules/syntax-case macros operate on syntax objects, not on
> raw s-expressions. This allows them to preserve referential
> transparency. So the result of expansion might not look like what you
> would expect.

Ummm, I just started to hammer this macro thing into my head, so I don't
understand (yet) what you are talking about. I thought the power of lisp
macros comes from the fact that they operate on s-expressions? What are
syntax objects? What is referential transparency?

> Read up on syntax-rules and syntax-case macros, when you get some time.
> I used to love defmacros but I don't use them any more.
> 
> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=doc/ref/api-macros.texi;h=51f54ed070af453a138282f61e7cd8dbbddc53a3;hb=HEAD

Where can I find a version translated in pdf or html or something?

Thanks Andy!

Re: Need help to understand a macro

[Andy Wingo]

Hi Josef,

On Mon 22 Mar 2010 20:50, Josef Wolf <jw@raven.inka.de> writes:

>> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=doc/ref/api-macros.texi;h=51f54ed070af453a138282f61e7cd8dbbddc53a3;hb=HEAD
>
> Where can I find a version translated in pdf or html or something?

http://hydra.nixos.org/build/329394/download/3/guile.pdf I think. I just
wrote those docs last week :) Or

http://hydra.nixos.org/build/329394/download/2/guile.html/Macros.html#Macros

Cheers,

Andy
-- 
http://wingolog.org/

Re: Need help to understand a macro

[Josef Wolf]

On Mon, Mar 22, 2010 at 09:36:16PM +0100, Andy Wingo wrote:
> Hi Josef,
> 
> On Mon 22 Mar 2010 20:50, Josef Wolf <jw@raven.inka.de> writes:
> 
> >> http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=doc/ref/api-macros.texi;h=51f54ed070af453a138282f61e7cd8dbbddc53a3;hb=HEAD
> >
> > Where can I find a version translated in pdf or html or something?
> 
> http://hydra.nixos.org/build/329394/download/3/guile.pdf I think. I just
> wrote those docs last week :) Or

Ah, no wonder I can't find it when I type "info guile" ;-)

> http://hydra.nixos.org/build/329394/download/2/guile.html/Macros.html#Macros

Thanks, I'll check it out.

BTW: While we're at the docs, what I find confusing about guile documentation
is that it talks extensively about the C bindings, but has not much to say
about the language itself and how guile differs from the standard (extensions,
limitations). But maybe I've just not got the big picture yet...

Re: Need help to understand a macro

[Neil Jerram]

Josef Wolf <jw@raven.inka.de> writes:

> BTW: While we're at the docs, what I find confusing about guile documentation
> is that it talks extensively about the C bindings, but has not much to say
> about the language itself and how guile differs from the standard (extensions,
> limitations). But maybe I've just not got the big picture yet...

I have some changes pending that may help a bit with that.

For the standards that Guile follows, I think it's pretty faithful, so
there isn't much to say about limitations.  So that just leaves saying
what those standards are, and extensions.

For the former, one of my pending changes for the 2.0 manual has this:

===================
Guile implements Scheme as described in the
Report on the Algorithmic Language Scheme (usually known as
@acronym{R5RS}), providing clean and general data and control
structures.  Guile goes beyond the rather austere language presented
in @acronym{R5RS}, extending it with a module system, full access to
@acronym{POSIX} system calls, networking support, multiple threads,
dynamic linking, a foreign function call interface, powerful string
processing, and many other features needed for programming in the real
world.

The Scheme community has recently agreed and published R6RS, the
latest installment in the RnRS series.  R6RS significantly expands the
core Scheme language, and standardises many non-core functions that
implementations -- including Guile -- have previously done in
different ways.  Guile has been updated to incorporate some of the
features of R6RS, and to adjust some existing features to conform to
the R6RS specification, but it is by no means a complete R6RS
implementation.

Between R5RS and R6RS, the SRFI process
(@url{http://srfi.schemers.org/}) standardised interfaces for many
practical needs, such as multithreading programming and
multidimensional arrays.  Guile supports many SRFIs, as documented in
detail in @ref{SRFI Support}.

In summary, so far as relationship to the Scheme standards is
concerned, Guile is an R5RS implementation with many extensions, some
of which conform to SRFIs or to the relevant parts of R6RS.
===================

As far as extensions are concerned, I think it is very likely that the
manual isn't always clear about what things are standardized and what
are extensions.  But I'm not sure what to do about it - do you have a
suggestion?  Also, how much does it matter in practice?

Regards,
       Neil

Re: Need help to understand a macro

[szgyg]

Josef Wolf wrote:
> I am trying to understand the defstruct macro from the "teach yourself
> scheme in fixnum days" tutorial, which can be found in chapter 9 at
> http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-11.html#node_chap_9
>
> My first question is of a more generic type. For a better understanding
> how macros work, I'd like to have a way to show what expansion a macro
> would generate when it would be used. E.g., I'd like to do something
> like
>
>   (show-expansion (defstruct tree height girth age leaf-shape leaf-color))
>
> and get what the expansion of this macro would produce:
>
>   (begin
>     (define make-tree
>       (lambda fvfv
>         [ ... and so on ... ]

Use the trick from the inexplicable JRM's Syntax-rules Primer for the 
Merely Eccentric[1], wrap the output with quote:

...
(let ((ff (map (lambda (f) (if (pair? f) (car f) f))
                ff)))
   `'(begin
;  ^
       (define ,(string->symbol
                 (string-append "make-" s-s))
...


[1]
http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt

Re: Need help to understand a macro

[Josef Wolf]

On Sat, Mar 20, 2010 at 03:13:45PM +0100, szgyg wrote:
> Josef Wolf wrote:
> >I am trying to understand the defstruct macro from the "teach yourself
> >scheme in fixnum days" tutorial, which can be found in chapter 9 at
> >http://www.ccs.neu.edu/home/dorai/t-y-scheme/t-y-scheme-Z-H-11.html#node_chap_9
> >
> >My first question is of a more generic type. For a better understanding
> >how macros work, I'd like to have a way to show what expansion a macro
> >would generate when it would be used. E.g., I'd like to do something
> >like
> >
> >  (show-expansion (defstruct tree height girth age leaf-shape leaf-color))
> >
> >and get what the expansion of this macro would produce:
> >
> >  (begin
> >    (define make-tree
> >      (lambda fvfv
> >        [ ... and so on ... ]
> 
> Use the trick from the inexplicable JRM's Syntax-rules Primer for
> the Merely Eccentric[1], wrap the output with quote:
> 
> ...
> (let ((ff (map (lambda (f) (if (pair? f) (car f) f))
>                ff)))
>   `'(begin
> ;  ^
>       (define ,(string->symbol
>                 (string-append "make-" s-s))
> ...

Ah, sounds reasonable. Thanks!

> [1]
> http://www.xs4all.nl/~hipster/lib/scheme/gauche/define-syntax-primer.txt

I will save this link for later, since this stuff is still too advanced for
me. In the meantime, I'll be looking for something that's more entry-level...