syntax-case identifier name predicate in guard expression

syntax-case identifier name predicate in guard expression

[Zelphir Kaltstahl]

Hi Guile Users!

In previous e-mails I inquired about macros defining procedures.

Afterwards I noticed, that I need my macro to check whether some regular
expression is inside the name of an identifier.

For example I have an two identifiers (not strings and no values bound
to them yet):

--------8<--------8<--------8<--------
/containers/json
/containers/<some string in here>/json
-------->8-------->8-------->8--------

I seem unable to find a way using syntax-rules to distinguish between
these two, when they are only given as identifiers. So I looked into
syntax-case, which has guards and thought, that I might be able to use
guard clauses to check for substrings in the name of the identifier
there. However, so far my attempts have failed to get it right and I am
having difficulties finding any example for such a thing. I believe in
principle it should be possible, however. It is just that I do not know how.

Here is my current code, simplified:

--------8<--------8<--------8<--------
(define-syntax identifier-name->string
  (lambda (stx)
    (syntax-case stx ()
      ((_ id)
       (identifier? #'id)
       (datum->syntax #'id (symbol->string (syntax->datum #'id)))))))

(define-syntax define-api-route
  (lambda (stx)
    (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
      [(_ route GET my-content-type)
       (string-match "<[^>]+>" (identifier-name->string (syntax route)))
       (syntax (quote aaa))])))
-------->8-------->8-------->8--------

When calling that I get an error:

--------8<--------8<--------8<--------
(define-api-route /containers/<id>/json GET
application/x-www-form-urlencoded)

While compiling expression:
In procedure symbol->string: Wrong type argument in position 1
(expecting symbol): #<syntax /containers/<id>/json>
-------->8-------->8-------->8--------

I have to wrap `route` into (syntax …) though, because otherwise it
complains about a reference to a pattern variable outside of a syntax
form. I do not really understand this, but in all examples I have seen
for guard clauses, pattern variables are wrapped in a call to syntax.

However, identifier-name->string does not work with syntax objects. I am
not sure how I can get the name of the identifier inside the guard
clause as string, so that I can check for occurences of anything in
angle braces <>.

I've also tried multiple other approaches over the last 2 days, but
always there was some error that made it seem like it was the wrong
approach and fixing the errors led to other errors. For example the
following also does not work (whole transcript):

--------8<--------8<--------8<--------
GNU Guile 2.2.4
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (use-modules (web uri)
             (web client)
             (json)
             (ice-9 iconv)
             (ice-9 regex))
scheme@(guile-user)> (define-syntax identifier-name->string
  (lambda (stx)
    (syntax-case stx ()
      ((_ id)
       (identifier? #'id)
       (datum->syntax #'id (symbol->string (syntax->datum #'id)))))))
scheme@(guile-user)> (define-syntax define-api-route-with-template-variables
  (syntax-rules (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
    [(_ route GET my-content-type)
     (syntax
      (define* (route url-temp-vars docker-socket #:key (data #f))
        (let* ([route-as-string (identifier-name->string route)]
               [match-count
                (fold-matches "<[^>]+>" route-as-string 0 (λ (curr-match
count) (+ count 1)))])
          (cond
           [(= match-count (length url-template-vars))
            ;; build the final request url
            ;; loop over available url template variables
            (let ([request-url
                   (let loop ([constructed-route-string route-as-string]
                              [remaining-url-template-vars url-temp-vars])
                     (cond
                      ;; return the whole constructed request url
                      [(null? remaining-url-template-vars)
constructed-route-string]
                      [else
                       ;; replace another url template var and recur
                       (loop (regexp-substitute #f
                                                (string-match "<[^>]+>"
constructed-route-string)
                                                'pre (car
remaining-url-template-vars) 'post)
                             (cdr remaining-url-template-vars))]))])
              ;; do the call with the built request url
              (send-request-to-docker-socket request-url
                                             docker-socket
                                             my-content-type
                                             #:data data))]
           [else
            (error "wrong number of URL template variables"
                   route
                   route-as-string
                   url-template-vars)]))))]))
scheme@(guile-user)> (define-syntax define-api-route
  (lambda (stx)
  ;; https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
  ;; All HTTP methods are literals.
    (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)

      [(_ route GET my-content-type)
       (let ([route-as-string (identifier-name->string route)])
         (string-match "<[^>]+>" route-as-string))
       (define-api-route-with-template-variables route GET my-content-type)]

      [(_ route GET my-content-type)
       ;; optional guard clause, already implied by the position of this
case of syntax
       (not (string-match "<[^>]+>" (identifier-name->string route)))
       (syntax
        (define* (route docker-socket #:key (data #f))
          (call-with-values
              (lambda ()
                (http-get (identifier-name->string route)
                          #:port docker-socket
                          #:version '(1 . 1)
                          #:keep-alive? #f
                          #:headers `((host . ("localhost" . #f))
                                      (content-type . (my-content-type
(charset . "utf-8"))))
                          #:body (scm->json-string data)
                          #:decode-body? #t
                          #:streaming? #f))
            (lambda (response response-text)
              (let ([resp-text-as-string (bytevector->string
response-text "utf-8")])
                (cons response resp-text-as-string))))))])))
scheme@(guile-user)> (define-api-route /containers/json GET
application/x-www-form-urlencoded)
scheme@(guile-user)> (define-api-route /containers/<id>/json GET
application/x-www-form-urlencoded)
scheme@(guile-user)> /containers/<id>/json
$2 = #<procedure /containers/<id>/json (docker-socket #:key data)>
scheme@(guile-user)>
-------->8-------->8-------->8--------

As you can see, the /containers/<id>/json does not take template
variable after it is defined, which means, that the syntax-case macro
always goes into the other case and the check for <id> does not work.
Apparently the regex is in this code checked for literally the string
"route", which is not what I want, as route is only supposed to stand
for whatever the user gives as identifier when calling the macro and I
want to check inside that.

My question now is:

How can I check for the regular expression inside a name of an
identifier inside a guard clause?

If that is impossible, what other ways are there to distinguish between
two macro expansions, where one has some <> part in the identifier and
the other does not and get 2 different expansions? How is this usually done?

I know I already took up some time of you before with my questions. I
just cannot find examples for such things on my searches and do not have
a lot experience writing macros. So thank you, if you can help me out
again with this problem.

I will also include a link to my repository, which should be publicly
available, where a few versions of my attempts are in the commits:
https://gitlab.com/ZelphirKaltstahl/guile-scheme-macros/tree/dev/procedure-defining

Regards,

Zelphir

Re: syntax-case identifier name predicate in guard expression

[Zelphir Kaltstahl]

Hi Guile Users!

I made some progress in writing a procedure defining macro for creating
procedures which talk to an API.

I now have working code, which checks the name of an identifier in a
guard expression:

----8<----8<----8<----
(use-modules (web uri)
             (web client)
             (json)
             (ice-9 iconv)
             (ice-9 regex))


(define* (send-request-to-docker-socket request-url docker-socket
my-content-type #:key (data #f))
  (call-with-values
      (lambda ()
        (http-get request-url
                  #:port docker-socket
                  #:version '(1 . 1)
                  #:keep-alive? #f
                  #:headers `((host . ("localhost" . #f))
                              (content-type . (my-content-type (charset
. "utf-8"))))
                  #:body (scm->json-string data)
                  #:decode-body? #t
                  #:streaming? #f))
    (lambda (response response-text)
      (let ([resp-text-as-string (bytevector->string response-text
"utf-8")])
        (cons response resp-text-as-string)))))


(define-syntax define-api-route
  (lambda (stx)
    (define (identifier-name->string id)
      (symbol->string (syntax->datum id)))

    ;; Not needed yet.
    ;; (define (identifier->symbol id)
    ;;   (syntax->datum id))

    (define (contains-url-template-variable? route-as-string)
      (string-match "<[^>]+>" route-as-string))

    ;; We do not need a macro to produce syntax. Instead we need to use
a procedure to produce the
    ;; syntax, because we want to use it while evaluating another macro.
    (define make-simple-api-route-definition-syntax
      (lambda (route http-method my-content-type route-as-string)
        (syntax
         (quote simple-route))))

    (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
      [(_ route GET my-content-type)
       (contains-url-template-variable? (identifier-name->string (syntax
route)))
       (syntax (quote aaa))]
      ;; an else branch basically
      [(_ route GET my-content-type) #t
       (syntax
        (define* (route docker-socket #:key (data #f))
          (call-with-values
              (lambda ()
                ;; GET request because syntax GET was specified
                (http-get request-url
                          #:port docker-socket
                          #:version '(1 . 1)
                          #:keep-alive? #f
                          #:headers `((host . ("localhost" . #f))
                                      (content-type . (my-content-type
(charset . "utf-8"))))
                          #:body (scm->json-string data)
                          #:decode-body? #t
                          #:streaming? #f))
            (lambda (response response-text)
              (let ([resp-text-as-string (bytevector->string
response-text "utf-8")])
                (cons response resp-text-as-string))))))])))


(define* (connect-to-docker-socket #:key (socket-path
"/var/run/docker.sock"))
  (let ([docker-sock-addr (make-socket-address AF_UNIX socket-path)]
        [docker-sock (socket PF_UNIX SOCK_STREAM 0)])
    (setsockopt docker-sock SOL_SOCKET SO_REUSEADDR 1)
    (connect docker-sock docker-sock-addr)
    docker-sock))


(display
 (define-api-route /containers/json GET
"application/x-www-form-urlencoded"))


;; (let ([dock-sock (connect-to-docker-socket)])
;;   (let ([resp-and-resp-text
;;          (/containers/json dock-sock
;;                            #:data '(("all" . "true")
;;                                     ("filters" . (("name" . #("db"))
;;                                                   ("status" .
#("running" "exited"))))))])
;;     (display resp-and-resp-text)
;;     (newline)))


(display
 (define-api-route /container/json GET "application/x-www-form-urlencoded"))
----8<----8<----8<----

However, it seems, that now I have a new problem. It seems I cannot use
a define form inside a syntax-case case!

Running the above code I get the following error:

----8<----8<----8<----
ice-9/boot-9.scm:222:17: In procedure map1:
Syntax error:
/home/user/development/Guile/macros/procedure-defining/runnable-example.scm:78:1:
definition in expression context, where definitions are not allowed, in
form (define /containers/json (lambda* (docker-socket #:key (data #f))
(call-with-values (lambda () (http-get request-url #:port docker-socket
#:version (quote (1 . 1)) #:keep-alive? #f #:headers (quasiquote ((host
"localhost" . #f) (content-type "application/x-www-form-urlencoded"
(charset . "utf-8")))) #:body (scm->json-string data) #:decode-body? #t
#:streaming? #f)) (lambda (response response-text) (let
((resp-text-as-string (bytevector->string response-text "utf-8"))) (cons
response resp-text-as-string))))))
----8<----8<----8<----

While it was possible to have a define form inside a syntax-rules,
suddenly it seems impossible inside a syntax-case?

Removing the (syntax ...) around the define form does not help either
and gives the following error:

----8<----8<----8<----
ice-9/psyntax.scm:2612:57: In procedure build-dispatch-call:
Syntax error:
/home/user/development/Guile/macros/procedure-defining/runnable-example.scm:50:7:
definition in expression context, where definitions are not allowed, in
form (define route (lambda* (docker-socket #:key (data #f))
(call-with-values (lambda () (http-get request-url #:port docker-socket
#:version (quote (1 . 1)) #:keep-alive? #f #:headers (quasiquote ((host
"localhost" . #f) (content-type my-content-type (charset . "utf-8"))))
#:body (scm->json-string data) #:decode-body? #t #:streaming? #f))
(lambda (response response-text) (let ((resp-text-as-string
(bytevector->string response-text "utf-8"))) (cons response
resp-text-as-string))))))
----8<----8<----8<----

Once I exchange the define form inside the syntax-case with something
like (syntax (quote abc)), the error disappears, but then I am not
reaching the goal of defining a procedure.

I don't know how to make progress and it is not like searching
"procedure defining macros" yields particularly useful results and
examples of syntax-case usage I could find simply only use one case and
are not helpful for my purpose. I could not even find information about
whether there is an "else" keyword for syntax-case and decided to simply
use #t for the guard expression. If anyone can point me to an easy to
understand resource, which collects all this information and what to do
how with macros, that would awesome. I started reading the syntax-case
paper by Dybvig, but really, it should not be necessary to read all of
it, in order to understand how to use macros. I still seem to have gaps
in my understanding, otherwise I would be able to simply write this
macro down and be done. Currently I am not sure where I can find useful
examples of comparable macros, that show me how these things are usually
done.

What can I do, how can I change my code to get my procedure defined
inside syntax-case, or better even call another procedure, which give me
back the syntax, which will be the result of that syntax-case case?

Regards, Zelphir

Re: syntax-case identifier name predicate in guard expression

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> Hi Guile Users!
>
> I made some progress in writing a procedure defining macro for creating
> procedures which talk to an API.
>
> I now have working code, which checks the name of an identifier in a
> guard expression:
>
> ----8<----8<----8<----
> (use-modules (web uri)
>              (web client)
>              (json)
>              (ice-9 iconv)
>              (ice-9 regex))
>
>
> (define* (send-request-to-docker-socket request-url docker-socket
> my-content-type #:key (data #f))
>   (call-with-values
>       (lambda ()
>         (http-get request-url
>                   #:port docker-socket
>                   #:version '(1 . 1)
>                   #:keep-alive? #f
>                   #:headers `((host . ("localhost" . #f))
>                               (content-type . (my-content-type (charset
> . "utf-8"))))
>                   #:body (scm->json-string data)
>                   #:decode-body? #t
>                   #:streaming? #f))
>     (lambda (response response-text)
>       (let ([resp-text-as-string (bytevector->string response-text
> "utf-8")])
>         (cons response resp-text-as-string)))))
>
>
> (define-syntax define-api-route
>   (lambda (stx)
>     (define (identifier-name->string id)
>       (symbol->string (syntax->datum id)))
>
>     ;; Not needed yet.
>     ;; (define (identifier->symbol id)
>     ;;   (syntax->datum id))
>
>     (define (contains-url-template-variable? route-as-string)
>       (string-match "<[^>]+>" route-as-string))
>
>     ;; We do not need a macro to produce syntax. Instead we need to use
> a procedure to produce the
>     ;; syntax, because we want to use it while evaluating another macro.

Right.  Before, you were trying to perform the check within a procedural
macro, whose code was run when the 'define-api-route' macro was being
_defined_, which is too early.  Here you have the right idea.

>     (define make-simple-api-route-definition-syntax
>       (lambda (route http-method my-content-type route-as-string)
>         (syntax
>          (quote simple-route))))
>
>     (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
>       [(_ route GET my-content-type)
>        (contains-url-template-variable? (identifier-name->string (syntax
> route)))
>        (syntax (quote aaa))]
>       ;; an else branch basically
>       [(_ route GET my-content-type) #t
>        (syntax
>         (define* (route docker-socket #:key (data #f))
>           (call-with-values
>               (lambda ()
>                 ;; GET request because syntax GET was specified
>                 (http-get request-url
>                           #:port docker-socket
>                           #:version '(1 . 1)
>                           #:keep-alive? #f
>                           #:headers `((host . ("localhost" . #f))
>                                       (content-type . (my-content-type
> (charset . "utf-8"))))
>                           #:body (scm->json-string data)
>                           #:decode-body? #t
>                           #:streaming? #f))
>             (lambda (response response-text)
>               (let ([resp-text-as-string (bytevector->string
> response-text "utf-8")])
>                 (cons response resp-text-as-string))))))])))
>
>
> (define* (connect-to-docker-socket #:key (socket-path
> "/var/run/docker.sock"))
>   (let ([docker-sock-addr (make-socket-address AF_UNIX socket-path)]
>         [docker-sock (socket PF_UNIX SOCK_STREAM 0)])
>     (setsockopt docker-sock SOL_SOCKET SO_REUSEADDR 1)
>     (connect docker-sock docker-sock-addr)
>     docker-sock))
>
>
> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))
>
>
> ;; (let ([dock-sock (connect-to-docker-socket)])
> ;;   (let ([resp-and-resp-text
> ;;          (/containers/json dock-sock
> ;;                            #:data '(("all" . "true")
> ;;                                     ("filters" . (("name" . #("db"))
> ;;                                                   ("status" .
> #("running" "exited"))))))])
> ;;     (display resp-and-resp-text)
> ;;     (newline)))
>
>
> (display
>  (define-api-route /container/json GET "application/x-www-form-urlencoded"))
> ----8<----8<----8<----
>
> However, it seems, that now I have a new problem. It seems I cannot use
> a define form inside a syntax-case case!

No, that's not it.  The problem is not with the macro itself, but with
how you're attempting to use it.

The problem is simply that the macro expands into a definition, and you
are trying to use it within an expression context, where definitions are
not allowed, here:

> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))

This fails for the same reason that the following fails:

  (display (define foo 1))

> While it was possible to have a define form inside a syntax-rules,
> suddenly it seems impossible inside a syntax-case?

For purposes of this issue, it is irrelevant whether the macro was
defined using syntax-rules or syntax-case.

If you have a macro that expands into a definition, then you can only
use that macro in a context where definitions are allowed.

      Best,
       Mark

Re: syntax-case identifier name predicate in guard expression

[Zelphir Kaltstahl]

Hi Mark!

Thank you for your help!

I must have been blind from moving code back and forth and the (display
...) was still from when the output of the macro was simply a symbol,
for checking whether the guard expression in the (syntax-case ...) cases
works. Also I was confused because the error message mentioned the whole
define expression, which is why I looked in the wrong place : )

I got it working now, including the more complex case of having
variables in request URLs.
(https://gitlab.com/ZelphirKaltstahl/guile-scheme-macros/blob/fccb603a92f2c2d2709e169bbdd516fcf05a6816/procedure-defining/runnable-example.scm)

Fortunately I re-read the Guile docs and read about with-syntax.
Suddenly with-syntax made a lof of sense :D

I think I am finally getting it more or less. There were quite a few
snares to overcome for me, but I think that with some trial and error I
can now get things done.

The complex case should also be moved out of the syntax-case directly
and then I can use it in the library and can finally define API routes
without much trouble (hopefully! unless I overlooked some possibility
for sending requests to the docker API), yay!

Regards,

Zelphir


On 8/5/19 8:51 PM, Mark H Weaver wrote:
> Hi Zelphir,
>
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> Hi Guile Users!
>>
>> I made some progress in writing a procedure defining macro for creating
>> procedures which talk to an API.
>>
>> I now have working code, which checks the name of an identifier in a
>> guard expression:
>>
>> ----8<----8<----8<----
>> (use-modules (web uri)
>>              (web client)
>>              (json)
>>              (ice-9 iconv)
>>              (ice-9 regex))
>>
>>
>> (define* (send-request-to-docker-socket request-url docker-socket
>> my-content-type #:key (data #f))
>>   (call-with-values
>>       (lambda ()
>>         (http-get request-url
>>                   #:port docker-socket
>>                   #:version '(1 . 1)
>>                   #:keep-alive? #f
>>                   #:headers `((host . ("localhost" . #f))
>>                               (content-type . (my-content-type (charset
>> . "utf-8"))))
>>                   #:body (scm->json-string data)
>>                   #:decode-body? #t
>>                   #:streaming? #f))
>>     (lambda (response response-text)
>>       (let ([resp-text-as-string (bytevector->string response-text
>> "utf-8")])
>>         (cons response resp-text-as-string)))))
>>
>>
>> (define-syntax define-api-route
>>   (lambda (stx)
>>     (define (identifier-name->string id)
>>       (symbol->string (syntax->datum id)))
>>
>>     ;; Not needed yet.
>>     ;; (define (identifier->symbol id)
>>     ;;   (syntax->datum id))
>>
>>     (define (contains-url-template-variable? route-as-string)
>>       (string-match "<[^>]+>" route-as-string))
>>
>>     ;; We do not need a macro to produce syntax. Instead we need to use
>> a procedure to produce the
>>     ;; syntax, because we want to use it while evaluating another macro.
> Right.  Before, you were trying to perform the check within a procedural
> macro, whose code was run when the 'define-api-route' macro was being
> _defined_, which is too early.  Here you have the right idea.
>
>>     (define make-simple-api-route-definition-syntax
>>       (lambda (route http-method my-content-type route-as-string)
>>         (syntax
>>          (quote simple-route))))
>>
>>     (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
>>       [(_ route GET my-content-type)
>>        (contains-url-template-variable? (identifier-name->string (syntax
>> route)))
>>        (syntax (quote aaa))]
>>       ;; an else branch basically
>>       [(_ route GET my-content-type) #t
>>        (syntax
>>         (define* (route docker-socket #:key (data #f))
>>           (call-with-values
>>               (lambda ()
>>                 ;; GET request because syntax GET was specified
>>                 (http-get request-url
>>                           #:port docker-socket
>>                           #:version '(1 . 1)
>>                           #:keep-alive? #f
>>                           #:headers `((host . ("localhost" . #f))
>>                                       (content-type . (my-content-type
>> (charset . "utf-8"))))
>>                           #:body (scm->json-string data)
>>                           #:decode-body? #t
>>                           #:streaming? #f))
>>             (lambda (response response-text)
>>               (let ([resp-text-as-string (bytevector->string
>> response-text "utf-8")])
>>                 (cons response resp-text-as-string))))))])))
>>
>>
>> (define* (connect-to-docker-socket #:key (socket-path
>> "/var/run/docker.sock"))
>>   (let ([docker-sock-addr (make-socket-address AF_UNIX socket-path)]
>>         [docker-sock (socket PF_UNIX SOCK_STREAM 0)])
>>     (setsockopt docker-sock SOL_SOCKET SO_REUSEADDR 1)
>>     (connect docker-sock docker-sock-addr)
>>     docker-sock))
>>
>>
>> (display
>>  (define-api-route /containers/json GET
>> "application/x-www-form-urlencoded"))
>>
>>
>> ;; (let ([dock-sock (connect-to-docker-socket)])
>> ;;   (let ([resp-and-resp-text
>> ;;          (/containers/json dock-sock
>> ;;                            #:data '(("all" . "true")
>> ;;                                     ("filters" . (("name" . #("db"))
>> ;;                                                   ("status" .
>> #("running" "exited"))))))])
>> ;;     (display resp-and-resp-text)
>> ;;     (newline)))
>>
>>
>> (display
>>  (define-api-route /container/json GET "application/x-www-form-urlencoded"))
>> ----8<----8<----8<----
>>
>> However, it seems, that now I have a new problem. It seems I cannot use
>> a define form inside a syntax-case case!
> No, that's not it.  The problem is not with the macro itself, but with
> how you're attempting to use it.
>
> The problem is simply that the macro expands into a definition, and you
> are trying to use it within an expression context, where definitions are
> not allowed, here:
>
>> (display
>>  (define-api-route /containers/json GET
>> "application/x-www-form-urlencoded"))
> This fails for the same reason that the following fails:
>
>   (display (define foo 1))
>
>> While it was possible to have a define form inside a syntax-rules,
>> suddenly it seems impossible inside a syntax-case?
> For purposes of this issue, it is irrelevant whether the macro was
> defined using syntax-rules or syntax-case.
>
> If you have a macro that expands into a definition, then you can only
> use that macro in a context where definitions are allowed.
>
>       Best,
>        Mark