Control the order of expansion of syntax-case macros

Control the order of expansion of syntax-case macros

[Vivien Kraus]

Dear Guile users,

syntax-case macros can have cool side effects at expansion time.
However, they are still draped in a veil of mystery to me.

Consider this code:

(define-syntax hello
  (lambda (stx)
    (syntax-case stx ()
      (_
       (begin
         (format (current-error-port) "Hello!\n")
         (datum->syntax #f "hello"))))))

(define-syntax world
  (lambda (stx)
    (syntax-case stx ()
      (_
       (begin
         (format (current-error-port) "World!\n")
         (datum->syntax #f "world"))))))

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       #`(string-append #,hello " " #,world)))))

,expand hello-world


Running it gives me:

Hello!
World!
$1 = (string-append "hello" " " "world")

Cool! Now, I suspect there are no clear rules to guess the order of
expansion (whether Hello! is printed before World! or after). I would
very much like to twist that order and reliably get World! and then
Hello!. How can I achieve that?

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       (let* ((w #'world)
              (h #'hello))
         #`(string-append #,h " " #,w))))))

With this modification, I get the same order:

Hello!
World!
$1 = (string-append "hello" " " "world")

So my guess is that syntax objects are expanded lazily. Is there
something I can do to get #'world expanded before #'hello?

Best regards,

Vivien

RE: Control the order of expansion of syntax-case macros

[Maxime Devos via General Guile related discussions]

>syntax-case macros can have cool side effects at expansion time.
>However, they are still draped in a veil of mystery to me.

> Consider this code:

(define-syntax hello
  (lambda (stx)
    (syntax-case stx ()
      (_
       (begin
         (format (current-error-port) "Hello!\n")
         (datum->syntax #f "hello"))))))

You can simplify this to:

(define-syntax hello
  (lambda (stx)
     (format (current-error-port) "Hello!\n")
     (datum->syntax #f "hello")))

I think this simplified version makes it a little easier to think about how Scheme macros work.

(define-syntax world
  (lambda (stx)
    (syntax-case stx ()
      (_
       (begin
         (format (current-error-port) "World!\n")
         (datum->syntax #f "world"))))))

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       #`(string-append #,hello " " #,world)))))

(Likewise here)

,expand hello-world


>Running it gives me:
>
>Hello!
>World!
>$1 = (string-append "hello" " " "world")

I wonder, did you copy-paste all four blocks (both the define-syntax forms and the ,expand) at the same time, or one-by-one? I don’t have a Guile at hand to test, but IIUC if you paste the following in the REPL (without any ,expand!)

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       #`(string-append #,hello " " #,world)))))

you’ll get “Hello!<newline>World!” in the REPL, and  ‘,expand hello-world’ will only give you the ‘$1 = […]’ (_without_ the current-error-port output).

What happens here, is that Guile sees your ‘hello-world’ definition and expands it (not the macro that it defines, but rather the ‘lambda’ and its interior) (macros are usually defined with other macros – syntax-case is a macro!). But, it’s not just ‘hello-world’ that’s a macro, hello and world are macros too. Since they aren’t quoted, they are expanded.  So, during the expansion of the ‘define-syntax’ form above, you will see “Hello!” and “World!”. Can we choose which order ‘hello’ and ‘world’ are expanded, while keeping everything else the same? I don’t know.

(‘hello’ and ‘world’ expand to strings, which can be spliced into syntax. If they expanded to, say, a procedure (not just a lambda form, but rather the evaluated lambda), it is invalid (not just a type error for string-append, but invalid).)

>Cool! Now, I suspect there are no clear rules to guess the order of
expansion (whether Hello! is printed before World! or after). I would
very much like to twist that order and reliably get World! and then
Hello!. How can I achieve that?

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       (let* ((w #'world)
              (h #'hello))
         #`(string-append #,h " " #,w))))))

Wait, you are modifying more here!!! In this particular (artificial) example, it probably doesn’t matter, but in whatever the ‘real’ code will eventually be, it might. When expanding the above ‘define-syntax’ form, Guile doesn’t expand ‘world’ and ‘hello’, since they are quoted. If you paste the above definition in the REPL, you won’t get any “Hello! World!” or “World! Hello” messages. No side-effects happen from the expansion of the above define-syntax form, only once a hello-world form is expanded you’ll get the side-effects.

(Depending on what you’re doing, this change might be better or worse.)

>With this modification, I get the same order:

>Hello!
>World!
>$1 = (string-append "hello" " " "world")

>So my guess is that syntax objects are expanded lazily.

Not lazy. If you splice the same syntax into multiple locations, it will be expanded per-location. (I would call it ‘delayed’ if it weren’t for the ‘delay’ macro and ’force’ procedure that implement laziness.)

;; This one is a workaholic! Lots of World! World! World! …
(define-syntax a
  (lambda (stx)
    (let ((a #'world)) #`(list #,a #,a #,a #,a))))

For testing, consider replacing it by

(define-syntax hello-world
  (lambda (stx)
    (syntax-case stx ()
      (_
       (let* ((w #'world)
              (h #'hello))
         (pk 'Post-Quote w h)
         #`(string-append #,h " " #,w))))))
;; no information on stderr yet on stderr
,expand hello-world
;; now we see (post-quote #<syntax world …> #<syntax hello …>) Hello! World!

If you simply write #'world, then no expansion happens yet (except maybe in the sense that `(proc) ‘expands too’ (list 'proc) instead of running the procedure ‘proc’).

During #`(string-append #,h “ “ #,w), no expansion happens yet either. You are simply making a larger piece of code from some smaller pieces. Rather, it’s when you write ‘hello-world’ in some piece of code that Guile applies the lambda (lambda (stx) stuff) to #’hello-world (in realistic settings it would be #'(hello-world arg arg2)), and replaces the code fragment ‘hello-world’ by the output of the lambda. E.g., if the lambda returns #'(not-implemented), then ‘(stuff hello-world more-stuff)’ becomes ‘(stuff (not-implemented) more-stuff).

I think the best way to think of syntax, is as code with some lexical context and line numbers, and ‘quote’ (') is the equivalent of ‘syntax-quote’ (#') (or was it quote-syntax?).

>Is there something I can do to get #'world expanded before #'hello?

Rewritten question:  “Is there something I can do to get ‘world’ expanded before ‘hello’?”

I imagine the following would do it:

(define-syntax hello-world
  (lambda (stx)
    ;; let / let* wouldn’t make a difference, since it’s expansion order we’re concerned
    ;; with, not evaluation order, and the former is equally as unspecified(?) with ‘let’ as
    ;; with ‘let*’
    (let ((w world) (h hello))
      #`(string-append #,hello " " #,world)))))

(but keep in mind the possibly unintended time of expansion).

Perhaps another option would be to use the module reflection API and macro-binding or macro-transformer to extract the (lambda (stx) …) and now use that lambda, or syntax-local-bindings and do the same.

For (well-defined) methods of controlling the evaluation order,  I imagine let-syntax might be able to accomplish something, but I’m not sure.

Another option is to just let ‘hello’ and ‘world’ be procedure (that happen to accept syntax and produce syntax), and simply invoke them in hello-world (and use #, to insert them in the produced code). In this particular case, they could just be thunks. (If you do this, and also use the hello-world form in the same module as where it is defined (not just mentioned somewhere in another macro, but rather when the non-define-syntax forms are expanded, then need the transformer of the hello-world form to run), then make sure to add some eval-when where appropriate.)

Best regards,
Maxime Devos

Re: Control the order of expansion of syntax-case macros

[Nala Ginrut]

>
>
> So my guess is that syntax objects are expanded lazily. Is there
> something I can do to get #'world expanded before #'hello?
>

More accurately, it's Normal Order for macro and special form (actually
it's another fancy name of internal implemented macro), you may want to
take a look through the "application order vs normal order" part inside
SICP.

Here's a simple formula to understand practical Lazy:
Lazy = Normal + Updating

"Updating" is another fancy academic term for caching the evaluated value
to prevent redundant computation. Say, memorization.

I could be blamed for the over simplified explanation to say Normal Order
rather than more general term non-strict-evaluation, Normal Order is just
one of them, theoretically, but most useful.

In your case, there's no updating, so you shouldn't claim it "Lazy" in
computation. :-)




Best regards.

Re: Control the order of expansion of syntax-case macros

[Vivien Kraus]

Hello!

Le vendredi 20 décembre 2024 à 10:15 +0100, Maxime Devos a écrit :
> (define-syntax hello
>   (lambda (stx)
>     (syntax-case stx ()
>       (_
>        (begin
>          (format (current-error-port) "Hello!\n")
>          (datum->syntax #f "hello"))))))
>  
> You can simplify this to:
>  
> (define-syntax hello
>   (lambda (stx)
>      (format (current-error-port) "Hello!\n")
>      (datum->syntax #f "hello")))

This was an eye-opening moment for me: syntax-case is its own thing and
I don’t need the full define-syntax / lambda / syntax-case combo.

> I wonder, did you copy-paste all four blocks (both the define-syntax
> forms and the ,expand) at the same time, or one-by-one?

Oops, I was evaluating everything at once, so I did not notice the
format took place at definition time.

> Another option is to just let ‘hello’ and ‘world’ be procedure (that
> happen to accept syntax and produce syntax)

I’ll have to examine my new options now that I can think outside the
define-syntax / lambda / syntax-case box.

Anyway, thank you again for your help.

Best regards,

Vivien

Re: Control the order of expansion of syntax-case macros

[Zelphir Kaltstahl]

Hello Vivien

On 17.12.24 17:33, Vivien Kraus wrote:
> Dear Guile users,
>
> syntax-case macros can have cool side effects at expansion time.
> However, they are still draped in a veil of mystery to me.
>
> Consider this code:
>
> (define-syntax hello
>    (lambda (stx)
>      (syntax-case stx ()
>        (_
>         (begin
>           (format (current-error-port) "Hello!\n")
>           (datum->syntax #f "hello"))))))
>
> (define-syntax world
>    (lambda (stx)
>      (syntax-case stx ()
>        (_
>         (begin
>           (format (current-error-port) "World!\n")
>           (datum->syntax #f "world"))))))
>
> (define-syntax hello-world
>    (lambda (stx)
>      (syntax-case stx ()
>        (_
>         #`(string-append #,hello " " #,world)))))
>
> ,expand hello-world
>
>
> Running it gives me:
>
> Hello!
> World!
> $1 = (string-append "hello" " " "world")
>
> Cool! Now, I suspect there are no clear rules to guess the order of
> expansion (whether Hello! is printed before World! or after). I would
> very much like to twist that order and reliably get World! and then
> Hello!. How can I achieve that?
>
> (define-syntax hello-world
>    (lambda (stx)
>      (syntax-case stx ()
>        (_
>         (let* ((w #'world)
>                (h #'hello))
>           #`(string-append #,h " " #,w))))))
>
> With this modification, I get the same order:
>
> Hello!
> World!
> $1 = (string-append "hello" " " "world")
>
> So my guess is that syntax objects are expanded lazily. Is there
> something I can do to get #'world expanded before #'hello?
>
> Best regards,
>
> Vivien

I think CK macro style might make it easier/possible, since it is internally 
managing a stack of expressions and they compose like usual functions.

It might also be overkill for this specific macro. But if this is not just an 
experiment, maybe the actual use case would warrant CK macro style.

Check out the amazing https://okmij.org/ftp/Scheme/macros.html#ck-macros and not 
amazing, but perhaps helpful 
https://codeberg.org/ZelphirKaltstahl/guile-examples/src/commit/ece4060df673b1a3173856555aeca6d8d8c7fd25/macros/CK-macros/ck-macro.scm 
explanations as far as I have once understood what goes on. For a usage example 
you could look at 
https://codeberg.org/ZelphirKaltstahl/guile-examples/src/commit/ece4060df673b1a3173856555aeca6d8d8c7fd25/macros/contracts.

Once I got my usage of the CK macro right, I found such macro way easier to use 
than normal macro which expands from left to right basically (maybe this is not 
technically correct to say, I don't know).

Best regards, Zelphir

-- 
repositories:https://notabug.org/ZelphirKaltstahl,https://codeberg.org/ZelphirKaltstahl