Write a macro which defines a procedure

Write a macro which defines a procedure

[zelphirkaltstahl]

Hi Guile Users,

I want to write a macro, which creates a procedure of given name, where 
the given name is a not yet defined identifier.
Here is what I have so far:

--------8<--------begin-------->8--------
(use-modules (web uri)
              (web client)
              (json)
              (ice-9 iconv))

(define-syntax define-api-route
   (syntax-rules ()
     [(define-api-route route http-method my-content-type)
      (module-define! (current-module)
                      ;; `route` should be `/container/json` for example.
                      route
                      (lambda* (docker-socket #:key (data #f))
                        (call-with-values
                            (lambda ()
                              ;; Here the route needs to be given as a 
string to `http-get`.
                              (http-get (variable-name->string route)
                                        #:port docker-socket
                                        #:version '(1 . 1)
                                        #:keep-alive? #f
                                        ;; Why is my quasiquote not 
working as expected?
                                        #:headers `((host . ("localhost" 
. #f))
                                                    (content-type . 
(,my-content-type (charset . "utf-8"))))
                                        #:body (scm->json-string data)
                                        #:decode-body? #t
                                        #:streaming? #f))
                          (lambda (response response-text)
                            (let ([resp-text-as-string 
(bytevector->string response-text "utf-8")])
                              (cons response resp-text-as-string))))))]))

;; from: https://stackoverflow.com/a/21177216/1829329
(define-syntax variable-name->string
   (lambda (stx)
     (syntax-case stx ()
       ((_ id)
        (identifier? #'id)
        (datum->syntax #'id (symbol->string (syntax->datum #'id)))))))
--------8<--------end-------->8--------

However, when I call this macro as follows:


--------8<--------begin-------->8--------
(define-api-route /container/json 'POST 
application/x-www-form-urlencoded)
--------8<--------end-------->8--------

I get the following error:

--------8<--------begin-------->8--------
;;; <stdin>:33:0: warning: possibly unbound variable `/container/json'
;;; <stdin>:33:0: warning: possibly unbound variable 
`application/x-www-form-urlencoded'
<unnamed port>:33:0: In procedure module-lookup: Unbound variable: 
/container/json
--------8<--------end-------->8--------

So I see that there are at least 2 problems with my macro:

1. `/container/json` is not defined. I thought it would be defined, as I 
tried to make the macro define it in the current module.
2. `application/x-www-form-urlencoded` is evaluated, although I have it 
in a quasiquoted expression.

There are other things, which I would like to implement once I get the 
above working:

* I would like the macro to complain, if there is other stuff with the 
same identifier already defined inside the module, as a little safety 
measure. However, this might be difficult, I guess, because the macro 
might be done before those other definitions of same identifier are even 
executed.

Anyway, can you help me out writing the macro, fixing at least the 2 
issues or direct me to a tutorial, which covers how to write a macro 
which gives basically a new define form?

Regards,
Zelphir

Re: Write a macro which defines a procedure

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]


zelphirkaltstahl <zelphirkaltstahl@posteo.de> writes:

> Hi Guile Users,
>
> I want to write a macro, which creates a procedure of given name,
> where the given name is a not yet defined identifier.
> Here is what I have so far:
… 

> Anyway, can you help me out writing the macro, fixing at least the 2
> issues or direct me to a tutorial, which covers how to write a macro
> which gives basically a new define form?

I can’t debug your form right now, but I can give you an example where I
created macros to write macros:

https://bitbucket.org/ArneBab/wisp/src/5e860c746ee02c764bf378aeb8f436a1a341bd5c/examples/enter-three-witches.scm#lines-209

The Enter-Macro defines a macro which outputs its arguments as text.

It took me some time to get it right, but it works pretty well.

Bits and pieces:
- check for the presence of a macro-binding:
(syntax-case x ()
   ((_ (name more ...) b ...)
     ; new binding: only create it if the binding is not already a macro
     (not (eq? 'macro (syntax-local-binding (syntax name))))
     
- stay on the same level:
  #'(begin …

(this is originally written in wisp and then transformed to scheme for
easier sharing with other schemers:
https://bitbucket.org/ArneBab/wisp/src/5e860c746ee02c764bf378aeb8f436a1a341bd5c/examples/enter-three-witches.w#lines-209 )

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]

Re: Write a macro which defines a procedure

[Ricardo Wurmus]

Hi,

> However, when I call this macro as follows:
>
>
> --------8<--------begin-------->8--------
> (define-api-route /container/json 'POST
> application/x-www-form-urlencoded)
> --------8<--------end-------->8--------
>
> I get the following error:
>
> --------8<--------begin-------->8--------
> ;;; <stdin>:33:0: warning: possibly unbound variable `/container/json'
> ;;; <stdin>:33:0: warning: possibly unbound variable
> `application/x-www-form-urlencoded'
> <unnamed port>:33:0: In procedure module-lookup: Unbound variable:
> /container/json
> --------8<--------end-------->8--------

Here it might help you to see the expansion of the macro.  In the REPL
you can use

    ,expand (define-api-route /container/json 'POST application/x-www-form-urlencoded)

to see what Scheme code the thing expands to.


--
Ricardo

Re: Write a macro which defines a procedure

[Matt Wette]

On 7/19/19 3:51 PM, zelphirkaltstahl wrote:
> (module-define! (current-module)
>                      ;; `route` should be `/container/json` for example.
>                      route

(quote route)

Re: Write a macro which defines a procedure

[Zelphir Kaltstahl]

Hey Ricardo,

Thanks for the advice! I did not remember, that I could use ,expand in 
the Guile REPL.
This was I found out about an issue, but not sure how to solve it. It 
seems like `module-define!` wants to have a symbol as input. But I could 
not figure out how to create a symbol from an identifier.

Here is my updated version of the macro:

--------8<--------begin-------->8--------
(use-modules (web uri)
              (web client)
              (json)
              (ice-9 iconv))

(define-syntax variable-name->string
   (lambda (stx)
     (syntax-case stx ()
       ((_ id)
        (identifier? #'id)
        (datum->syntax #'id (symbol->string (syntax->datum #'id)))))))

(define-syntax define-api-route
   (syntax-rules ()
     [(define-api-route route http-method my-content-type)
      (module-define! (current-module)
                      ;; `route` should be `/container/json` for example.
                      #{route}#
                      (lambda* (docker-socket #:key (data #f))
                        (call-with-values
                            (lambda ()
                              ;; Here the route needs to be given as a 
string to `http-get`.
                              (http-get (variable-name->string route)
                                        #:port docker-socket
                                        #:version '(1 . 1)
                                        #:keep-alive? #f
                                        ;; Why is my quasiquote not 
working as expected?
                                        #:headers `((host . ("localhost" 
. #f))
                                                    (content-type . 
(my-content-type (charset . "utf-8"))))
                                        #; (list (cons "localhost" #f)
                                        (cons 'content-type (cons 
my-content-type (cons charset "utf-8"))))
                                        #:body (scm->json-string data)
                                        #:decode-body? #t
                                        #:streaming? #f))
                          (lambda (response response-text)
                            (let ([resp-text-as-string 
(bytevector->string response-text "utf-8")])
                              (cons response resp-text-as-string))))))]))
--------8<--------end-------->8--------

And expanding it:

--------8<--------begin-------->8--------
,expand(define-api-route
   /container/json
   'GET
   application/x-www-form-urlencoded)
$3 = (module-define!
   (current-module)
   /container/json
   (lambda* (docker-socket #:key (data #f #:data))
     (call-with-values
       (lambda ()
         (http-get
           "/container/json"
           #:port
           docker-socket
           #:version
           '(1 . 1)
           #:keep-alive?
           #f
           #:headers
           '((host "localhost" . #f)
             (content-type
               application/x-www-form-urlencoded
               (charset . "utf-8")))
           #:body
           (scm->json-string data)
           #:decode-body?
           #t
           #:streaming?
           #f))
       (lambda (response response-text)
         (let ((resp-text-as-string
                 (bytevector->string response-text "utf-8")))
           (cons response resp-text-as-string))))))
--------8<--------end-------->8--------

So using the Guile specific #{}# 
(https://www.gnu.org/software/guile/manual/html_node/Symbol-Read-Syntax.html#Symbol-Read-Syntax) 
did not work as I thought it might.
How can I make a symbol from an identifier? In this case:

/container/json/ -> '/container/json

Are there other things I should improve in this macro?

Regards,
Zelphir


On 2019-07-20 10:46, Ricardo Wurmus wrote:
> Hi,
> 
>> However, when I call this macro as follows:
>> 
>> 
>> --------8<--------begin-------->8--------
>> (define-api-route /container/json 'POST
>> application/x-www-form-urlencoded)
>> --------8<--------end-------->8--------
>> 
>> I get the following error:
>> 
>> --------8<--------begin-------->8--------
>> ;;; <stdin>:33:0: warning: possibly unbound variable `/container/json'
>> ;;; <stdin>:33:0: warning: possibly unbound variable
>> `application/x-www-form-urlencoded'
>> <unnamed port>:33:0: In procedure module-lookup: Unbound variable:
>> /container/json
>> --------8<--------end-------->8--------
> 
> Here it might help you to see the expansion of the macro.  In the REPL
> you can use
> 
>     ,expand (define-api-route /container/json 'POST
> application/x-www-form-urlencoded)
> 
> to see what Scheme code the thing expands to.
> 
> 
> --
> Ricardo

Re: Write a macro which defines a procedure

[Ricardo Wurmus]

Hi Zelphir,

> This was I found out about an issue, but not sure how to solve it. It
> seems like `module-define!` wants to have a symbol as input. But I
> could not figure out how to create a symbol from an identifier.
>
> Here is my updated version of the macro:
[…]
> (define-syntax define-api-route
>   (syntax-rules ()
>     [(define-api-route route http-method my-content-type)
>      (module-define! (current-module)
>                      ;; `route` should be `/container/json` for example.
>                      #{route}#

Use (quote route) instead.

-- 
Ricardo

Re: Write a macro which defines a procedure

[Mark H Weaver]

Matt Wette <matt.wette@gmail.com> writes:

> On 7/19/19 3:51 PM, zelphirkaltstahl wrote:
>> (module-define! (current-module)
>>                      ;; `route` should be `/container/json` for example.
>>                      route
>
> (quote route)

Yes, or equivalently:  'route

As an aside, is there a reason to not define it more simply as follows?

     Mark

--8<---------------cut here---------------start------------->8---
(define-syntax define-api-route
  (syntax-rules ()
    [(define-api-route route http-method my-content-type)
     ;; `route` should be `/container/json` for example.
     (define* (route docker-socket #:key (data #f))
       (call-with-values
           (lambda ()
             ;; Here the route needs to be given as a string to `http-get`.
             (http-get (variable-name->string route)
                       #:port docker-socket
                       #:version '(1 . 1)
                       #:keep-alive? #f
                       ;; Why is my quasiquote not working as expected?
                       #:headers `((host . ("localhost" . #f))
                                   (content-type . (,my-content-type (charset . "utf-8"))))
                       #:body (scm->json-string data)
                       #:decode-body? #t
                       #:streaming? #f))
         (lambda (response response-text)
           (let ([resp-text-as-string (bytevector->string response-text "utf-8")])
             (cons response resp-text-as-string)))))]))
--8<---------------cut here---------------end--------------->8---

Re: Write a macro which defines a procedure

[Zelphir Kaltstahl]

Hi,

At first I had the macro use `define`, but then I thought: "What if I
want to conditionally define a route procedure?". My guess is, that then
the define form would be inside some `cond` or `if` form and then it
would not work (I did not check this assumption though, because my macro
did not work yet.). That is why I looked for something that does what
`module-define!` does.

On 7/21/19 3:26 AM, Mark H Weaver wrote:
> Matt Wette <matt.wette@gmail.com> writes:
>
>> On 7/19/19 3:51 PM, zelphirkaltstahl wrote:
>>> (module-define! (current-module)
>>>                      ;; `route` should be `/container/json` for example.
>>>                      route
>> (quote route)
> Yes, or equivalently:  'route
>
> As an aside, is there a reason to not define it more simply as follows?
>
>      Mark
>
> --8<---------------cut here---------------start------------->8---
> (define-syntax define-api-route
>   (syntax-rules ()
>     [(define-api-route route http-method my-content-type)
>      ;; `route` should be `/container/json` for example.
>      (define* (route docker-socket #:key (data #f))
>        (call-with-values
>            (lambda ()
>              ;; Here the route needs to be given as a string to `http-get`.
>              (http-get (variable-name->string route)
>                        #:port docker-socket
>                        #:version '(1 . 1)
>                        #:keep-alive? #f
>                        ;; Why is my quasiquote not working as expected?
>                        #:headers `((host . ("localhost" . #f))
>                                    (content-type . (,my-content-type (charset . "utf-8"))))
>                        #:body (scm->json-string data)
>                        #:decode-body? #t
>                        #:streaming? #f))
>          (lambda (response response-text)
>            (let ([resp-text-as-string (bytevector->string response-text "utf-8")])
>              (cons response resp-text-as-string)))))]))
> --8<---------------cut here---------------end--------------->8---

Re: Write a macro which defines a procedure

[Zelphir Kaltstahl]

Ah thanks! Sometimes one does not see the simple solution. I was 
thinking that it would have to be something complicated.

On 2019-07-20 22:37, Ricardo Wurmus wrote:
> Hi Zelphir,
> 
>> This was I found out about an issue, but not sure how to solve it. It
>> seems like `module-define!` wants to have a symbol as input. But I
>> could not figure out how to create a symbol from an identifier.
>> 
>> Here is my updated version of the macro:
> […]
>> (define-syntax define-api-route
>>   (syntax-rules ()
>>     [(define-api-route route http-method my-content-type)
>>      (module-define! (current-module)
>>                      ;; `route` should be `/container/json` for 
>> example.
>>                      #{route}#
> 
> Use (quote route) instead.

Re: Write a macro which defines a procedure

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> At first I had the macro use `define`, but then I thought: "What if I
> want to conditionally define a route procedure?". My guess is, that then
> the define form would be inside some `cond` or `if` form and then it
> would not work

You're right, it would not work.  Nonetheless, I would strongly
discourage you from doing this kind of thing unless there's a very
compelling reason for it.

Note that we have 'cond-expand' from SRFI-0 for conditional inclusion of
definitions, if you need it.  'cond-expand' is roughly analogous to #if
in C.  It's less general than plain 'if', but there's an important
reason for that, and it has to do with the distinction between compile
time and run time.

If you use 'cond-expand', the condition is checked at compile time, and
consequently it is known at compile time which branch was taken and
therefore which definitions were chosen.

If you use 'cond', then the condition will be checked at run time, which
will normally rule out the possibility of optimization opportunities
such as inlining and partial evaluation.  At present, Guile is not able
to perform these optimizations for toplevel level definitions in any
case, but it might be possible in the future, at least for modules that
are not explicitly mutated using low-level operations like this.

Also, it's possible that a future version of Guile may be more
disciplined in its handling of phases, i.e. expand-time
(i.e. compile-time) vs run-time phases.  See section 7.2 (Import and
export levels) of the R6RS for details on one approach, which would
bring many advantages.  It's been a while since I looked at this
closely, and at this point I would need some time to refresh my memory
of the details, but it seems to me that the code you've written would
likely cause problems with such an approach.

Both the R6RS and R7RS have more declarative module systems, which bring
many benefits.  Added freedom in one area often implies a loss of
freedom somewhere else.  In this case, the freedom to treat modules as
arbitrarily mutable objects implies a loss of freedom in the compiler to
make any assumptions about what's in the module or even its set of
bindings.  That freedom carries a very significant cost.

       Mark

Re: Write a macro which defines a procedure

[Zelphir Kaltstahl]

Hi Mark!

On 7/21/19 4:56 AM, Mark H Weaver wrote:
> Hi Zelphir,
>
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> At first I had the macro use `define`, but then I thought: "What if I
>> want to conditionally define a route procedure?". My guess is, that then
>> the define form would be inside some `cond` or `if` form and then it
>> would not work
> You're right, it would not work.  Nonetheless, I would strongly
> discourage you from doing this kind of thing unless there's a very
> compelling reason for it.

To be honest there are not really any compelling reasons and I did think
about such consequences that you mention below, only the thought of not
being able to use the macro conditionally and wanting it to be usable
everywhere and fear of possibly making a mistake later on in my code,
trying to use it somewhere, where it cannot be used. However, now that I
think about it, there might not be a real need inside that project after
all, as the API route are predefined in the docker docs.

> Note that we have 'cond-expand' from SRFI-0 for conditional inclusion of
> definitions, if you need it.  'cond-expand' is roughly analogous to #if
> in C.  It's less general than plain 'if', but there's an important
> reason for that, and it has to do with the distinction between compile
> time and run time.

Aha! I did not know that : )

I will read about the `cond-expand` in the SRFI 0. Thanks for mentioning
SRFI 0. I guess I am still a little uneducated with regard to SRFIs and
what they contain. Zero seems like an important number.

> If you use 'cond-expand', the condition is checked at compile time, and
> consequently it is known at compile time which branch was taken and
> therefore which definitions were chosen.
>
> If you use 'cond', then the condition will be checked at run time, which
> will normally rule out the possibility of optimization opportunities
> such as inlining and partial evaluation.  At present, Guile is not able
> to perform these optimizations for toplevel level definitions in any
> case, but it might be possible in the future, at least for modules that
> are not explicitly mutated using low-level operations like this.
>
> Also, it's possible that a future version of Guile may be more
> disciplined in its handling of phases, i.e. expand-time
> (i.e. compile-time) vs run-time phases.  See section 7.2 (Import and
> export levels) of the R6RS for details on one approach, which would
> bring many advantages.  It's been a while since I looked at this
> closely, and at this point I would need some time to refresh my memory
> of the details, but it seems to me that the code you've written would
> likely cause problems with such an approach.
>
> Both the R6RS and R7RS have more declarative module systems, which bring
> many benefits.  Added freedom in one area often implies a loss of
> freedom somewhere else.  In this case, the freedom to treat modules as
> arbitrarily mutable objects implies a loss of freedom in the compiler to
> make any assumptions about what's in the module or even its set of
> bindings.  That freedom carries a very significant cost.
>
>        Mark

Thank you for your explanations. That's great feedback! It seems asking
on the Guile user list saved me again from doing something not so good
in my code. I will improve my code accordingly then, either by making
use of SRFI 0 or reverting to simple `define`, knowing that I can only
use it at the top level of a module or top level of the scope where I
want things to be available, as with normal `define` forms.

… *goes and reads SRFI 0* …

OK, so the limitations are, that `cond-expand` is restricted to the top
level, unless Guile implements it differently. Not sure what kind of
useful condition I can come up with inside the `cond-expand` and simply
putting a `#t` seems silly. I guess it is really for checking things you
can know at expand time. I do not have any such conditions in mind to
check. Perhaps I will stick with normal `define` forms for now and just
need to remember, that it will be expanded to a `define` form and thus
needs to be used like one. Actually, that would still make it possible
to return the created procedures and then I could still conditionally
return another procedure instead, so it is still somewhat flexible.

Regards,

Zelphir

Re: Write a macro which defines a procedure

[Zelphir Kaltstahl]

Hello Arne!

Thanks for posting those links!

I think your "already defined macro check" is part of what I was looking
for. I say part, because I was looking for something, that probably does
not exist: Something that checks for existing macros and for existing
procedures or other after compile-time defined things. However, the
macro runs before those things are even evaluated, so I guess the only
option would be to inspect the code and then find out what is defined.
This is not really practical.

So now I think that your check is actually as far as one can reasonably go.

Regards,

Zelphir

On 7/20/19 6:00 PM, guile-user-request@gnu.org wrote:
> Message: 2
> Date: Sat, 20 Jul 2019 09:04:56 +0200
> From: Arne Babenhauserheide <arne_bab@web.de>
> To: guile-user@gnu.org
> Subject: Re: Write a macro which defines a procedure
> Message-ID: <87sgr1nq5d.fsf@web.de>
> Content-Type: text/plain; charset="utf-8"
>
>
> zelphirkaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> Hi Guile Users,
>>
>> I want to write a macro, which creates a procedure of given name,
>> where the given name is a not yet defined identifier.
>> Here is what I have so far:
> … 
>
>> Anyway, can you help me out writing the macro, fixing at least the 2
>> issues or direct me to a tutorial, which covers how to write a macro
>> which gives basically a new define form?
> I can’t debug your form right now, but I can give you an example where I
> created macros to write macros:
>
> https://bitbucket.org/ArneBab/wisp/src/5e860c746ee02c764bf378aeb8f436a1a341bd5c/examples/enter-three-witches.scm#lines-209
>
> The Enter-Macro defines a macro which outputs its arguments as text.
>
> It took me some time to get it right, but it works pretty well.
>
> Bits and pieces:
> - check for the presence of a macro-binding:
> (syntax-case x ()
>    ((_ (name more ...) b ...)
>      ; new binding: only create it if the binding is not already a macro
>      (not (eq? 'macro (syntax-local-binding (syntax name))))
>      
> - stay on the same level:
>   #'(begin …
>
> (this is originally written in wisp and then transformed to scheme for
> easier sharing with other schemers:
> https://bitbucket.org/ArneBab/wisp/src/5e860c746ee02c764bf378aeb8f436a1a341bd5c/examples/enter-three-witches.w#lines-209 )
>
> Best wishes,
> Arne