define-module, #:export and export

define-module, #:export and export

[yarl baudig]

Hello guile.

I don't know if that's a bug. Anyway, I am confused about this so I ask. I came across this problem playing with guix source code. I will share different "tests" each test is a directory with nothing but the files I share.
each time the command to try the test (inside it's directory) is `guile --no-auto-compile -L . main.scm`

base/
main.scm:
-----
(define-module (main)
  #:export (valid-path?))

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (syntax-rules (name ...)
         ((_ name) id) ...)))))

(define-enumerate-type operation-id
  (valid-path? 1))

(define-syntax operation
  (syntax-rules ()
    ((operation name)
     (lambda ()
       (simple-format #t "~S\n" (operation-id name))
       (+ 3 (operation-id name))))))

(define-syntax define-operation
  (syntax-rules ()
    ((prout (name))
     (define name (operation name)))))

(define-operation (valid-path?))

(simple-format #t "~S\n" (operation-id valid-path?))
-----
RUNS


split-base/
lib.scm:
-----
(define-module (lib)
  #:export (operation-id))

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (syntax-rules (name ...)
         ((_ name) id) ...)))))

(define-enumerate-type operation-id
  (valid-path? 1))
-----
main.scm:
-----
(define-module (main)
  #:use-module (lib)
  ;; #:export (valid-path?))

(define-syntax operation
  (syntax-rules ()
    ((operation name)
     (lambda ()
       (simple-format #t "~S\n" (operation-id name))
       (+ 3 (operation-id name))))))

(define-syntax define-operation
  (syntax-rules ()
    ((prout (name))
     (define name (operation name)))))

(define-operation (valid-path?))
;; (export valid-path?)

;; (simple-format #t "~S\n" (operation-id valid-path?))
-----
Now. this RUNS. There is 3 tests from this "split-base"
split-define-module-export: from split-base, uncomment "#:export (valid-path?)" : FAILS
Note the difference with "base", which RUNS.
split-export : from split-base (so, comment again #:export), uncomment "(export valid-path?)" : RUNS
Here I note that there is a difference between #:export (...) and (export ...). Is this a bug? Or an undocumented feature? Or a misunderstanding from me?
split-simple-format : from split-base, uncomment "(simple-format ...)" : FAILS.
Here the "interesting" difference is between base and split-base. I am really confused there and I am asking for comments, please :).
Thank you!

Re: define-module, #:export and export

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 2051 bytes --]

On 04-01-2023 16:11, yarl baudig wrote:
> Hello guile.
> 
> I don't know if that's a bug. Anyway, I am confused about this so I ask. I came across this problem playing with guix source code. I will share different "tests" each test is a directory with nothing but the files I share.
> each time the command to try the test (inside it's directory) is `guile --no-auto-compile -L . main.scm`
> [...]
My (untested) hypothesis (*: things I'm unsure about)

If there is no #:export (...), then when
(define-operation (valid-path?)) is expanded, the identifier 
'valid-path?' is free (*).

Hence, the following ...

 >         (syntax-rules (name ...)
 >           ((_ name) id) ...)))))

refers to the free identifier 'valid-path?'.

Conversely, if there is #:export, then the (syntax-rules (name ...) ...) 
looks for the bound identifier 'valid-path?' (*).

Important: bound identifiers != free-identifier, even if they have the 
same symbol!  For (simple-format #t "~S\n" (operation-id valid-path?)) 
to work, the 'valid-path?' of that expression must be bound if the 
syntax-rules looks for a bound identifier, and free if it looks for a 
free identifier.

By adding a (export valid-path?) between the simple-format and the 
(define-operation ...), the 'valid-path?' in (simple-format ...) becomes 
a bound identifier (*).

Proposed solutions:

   (a) Move the (export ...) after the (simple-format ...) instead
       of in-between the define-operation and simple-format
   (b) Or put the uses of define-operation and define-enumeration-type
       in the same file.
   (c) Or use (eq? (syntax->datum ...) ...) instead of the
       'syntax-rules (...)', to ignore the lexical environment
       and hence the bound/free distinction.

       (This has consequences for users of 'main' that rename their
       import of 'valid-path?' -- by this change, operation-id will
       expect the unrenamed name, whereas in the original code (IIUC)
       it would expect the renamed name.)

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[Jean Abou Samra]

[-- Attachment #1.1: Type: text/plain, Size: 3103 bytes --]

Le 04/01/2023 à 18:28, Maxime Devos a écrit :
> On 04-01-2023 16:11, yarl baudig wrote:
>> Hello guile.
>>
>> I don't know if that's a bug. Anyway, I am confused about this so I 
>> ask. I came across this problem playing with guix source code. I will 
>> share different "tests" each test is a directory with nothing but the 
>> files I share.
>> each time the command to try the test (inside it's directory) is 
>> `guile --no-auto-compile -L . main.scm`
>> [...]
> My (untested) hypothesis (*: things I'm unsure about)
>
> If there is no #:export (...), then when
> (define-operation (valid-path?)) is expanded, the identifier 
> 'valid-path?' is free (*).
>
> Hence, the following ...
>
> >         (syntax-rules (name ...)
> >           ((_ name) id) ...)))))
>
> refers to the free identifier 'valid-path?'.
>
> Conversely, if there is #:export, then the (syntax-rules (name ...) 
> ...) looks for the bound identifier 'valid-path?' (*).
>
> Important: bound identifiers != free-identifier, even if they have the 
> same symbol!  For (simple-format #t "~S\n" (operation-id valid-path?)) 
> to work, the 'valid-path?' of that expression must be bound if the 
> syntax-rules looks for a bound identifier, and free if it looks for a 
> free identifier.


That is also my understanding, confirmed by

$ cat lib.scm
(define-module (lib)
   #:export (test))

(define-syntax test
   (lambda (sintax)
     (syntax-case sintax ()
       ((test id)
        (datum->syntax sintax (free-identifier=? #'id #'thing))))))

$ cat test.scm
(define-module (main)
   #:use-module (lib)
   #:export (thing)
   )

(display (test thing))
(newline)

(define thing 5)

$ guile3.0 -L . test.scm
#f


If you comment out #:export (thing), the result changes to #t.

To put it perhaps more simply, the use of #:export causes Guile to 
understand early that there will be a variable 'thing' in this module, 
and makes the identifier 'thing' refer to this variable that is not yet 
defined. However, hygiene implies that you want to be able to use 
keywords as if they were not keywords if they are rebound, e.g. the 
'else' here doesn't cause the cond clause to be taken:

(let ((else #f)) (cond (#f 'bla) (else 'foo) (#t 'bar)))
$1 = bar

The way this is done is by comparing with the original identifier given 
to syntax-rules. Technically, they are compared with free-identifier=? . 
This means that a use of the identifier matches the keyword iff both are 
unbound, or both are bound to the same lexical binding. However, this 
isn't the case here, as the keyword in the macro was unbound, but at the 
point of use, it has been bound by #:export.

Honestly, I think it is better to choose a different way of writing 
these macros that avoids this confusing issue. Try defining the 
operations at the same time as the enum so that the macro giving an enum 
member refers to the bindings of the operators. If you give more context 
on what you're trying to do, we could help more.

Best,
Jean


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[yarl baudig]

> 
> That is also my understanding, confirmed by
> 
> $ cat lib.scm
> (define-module (lib)
>    #:export (test))
> 
> (define-syntax test
>    (lambda (sintax)
>      (syntax-case sintax ()
>        ((test id)
>         (datum->syntax sintax (free-identifier=? #'id #'thing))))))
> 
> $ cat test.scm
> (define-module (main)
>    #:use-module (lib)
>    #:export (thing)
>    )
> 
> (display (test thing))
> (newline)
> 
> (define thing 5)
> 
> $ guile3.0 -L . test.scm
> #f
> 
> 
> If you comment out #:export (thing), the result changes to #t.
> 
> To put it perhaps more simply, the use of #:export causes Guile to 
> understand early that there will be a variable 'thing' in this module, 
> and makes the identifier 'thing' refer to this variable that is not yet 
> defined. However, hygiene implies that you want to be able to use 
> keywords as if they were not keywords if they are rebound, e.g. the 
> 'else' here doesn't cause the cond clause to be taken:
> 
> (let ((else #f)) (cond (#f 'bla) (else 'foo) (#t 'bar)))
> $1 = bar
> 
> The way this is done is by comparing with the original identifier given 
> to syntax-rules. Technically, they are compared with free-identifier=? . 
> This means that a use of the identifier matches the keyword iff both are 
> unbound, or both are bound to the same lexical binding. However, this 
> isn't the case here, as the keyword in the macro was unbound, but at the 
> point of use, it has been bound by #:export.
> 
> Honestly, I think it is better to choose a different way of writing 
> these macros that avoids this confusing issue. Try defining the 
> operations at the same time as the enum so that the macro giving an enum 
> member refers to the bindings of the operators. If you give more context 
> on what you're trying to do, we could help more.
> 
> Best,
> Jean
> 
> 

Thank you both. I am trying to process this. I (re-)read 6.8 Macros and read tspl4. I don't understand why you use "datum->syntax".
I heard your suggestion. I am only trying to grasp things for now. I realize I don't know enough (on guile and on guix) to go on a concrete goal yet.
I need to work more on this in order to understand but I wanted to thank you for answering that quickly!

Re: define-module, #:export and export

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1266 bytes --]



On 06-01-2023 14:31, yarl baudig wrote:
>>
>> That is also my understanding, confirmed by
>>
>> $ cat lib.scm
>> (define-module (lib)
>>     #:export (test))
>>
>> (define-syntax test
>>     (lambda (sintax)
>>       (syntax-case sintax ()
>>         ((test id)
>>          (datum->syntax sintax (free-identifier=? #'id #'thing))))))
>> [...]
>>
> 
> Thank you both. I am trying to process this. I (re-)read 6.8 Macros and read tspl4. I don't understand why you use "datum->syntax" > I heard your suggestion. I am only trying to grasp things for now. I 
realize I don't know enough (on guile and on guix) to go on a concrete 
goal yet.
> I need to work more on this in order to understand but I wanted to thank you for answering that quickly!
free-identifier=? returns a boolean.  However, the return value of the 
clauses of the syntax-case must be a syntax object.  Hence Jean 
converted the boolean into a syntax form with datum->syntax (IIUC).

Guile's implementation of macros is a little lax with typing,
in the sense that objects like #false and #true (but not symbols, 
because hygiene) can be returned too, but IIUC this is undocumented and 
not standard Scheme (*).

(*): unverified

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[Jean Abou Samra]

[-- Attachment #1.1: Type: text/plain, Size: 673 bytes --]

Le 06/01/2023 à 14:55, Maxime Devos a écrit :
> Guile's implementation of macros is a little lax with typing,
> in the sense that objects like #false and #true (but not symbols, 
> because hygiene) can be returned too, but IIUC this is undocumented 
> and not standard Scheme (*).



Well, that is what I thought too, but after checking, I realize that it 
actually is standard. See

http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-13.html

"""
More formally, a syntax object is:

- a pair of syntax objects,
-a vector of syntax objects,
- a nonpair, nonvector, nonsymbol value, or
- a wrapped syntax object.
"""

I learnt something today.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[yarl baudig]

What if I replace

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (syntax-rules (name ...)
         ((_ name) id) ...)))))

with

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (syntax-rules ()
         ((_ name) id) ...)))))

?

Re: define-module, #:export and export

[Jean Abou Samra]

[-- Attachment #1.1: Type: text/plain, Size: 1100 bytes --]

Le 08/01/2023 à 10:46, yarl baudig a écrit :
> What if I replace
>
> (define-syntax define-enumerate-type
>    (syntax-rules ()
>      ((_ name->int (name id) ...)
>       (define-syntax name->int
>         (syntax-rules (name ...)
>           ((_ name) id) ...)))))
>
> with
>
> (define-syntax define-enumerate-type
>    (syntax-rules ()
>      ((_ name->int (name id) ...)
>       (define-syntax name->int
>         (syntax-rules ()
>           ((_ name) id) ...)))))
>
> ?




Not going to work.

(define-syntax define-enumerate-type
   (syntax-rules ()
     ((_ name->int (name id) ...)
      (define-syntax name->int
        (syntax-rules ()
          ((_ name) id) ...)))))

(define-enumerate-type my-enum
   (member1 1)
   (member2 2))

(my-enum member2)
⇒ 1


The define-enumerate-type invocation expands to

(define-syntax my-enum
   (syntax-rules ()
     ((_ member1) 1)
     ((_ member2) 2)))

and since member1 is not declared as a literal in the first
argument to syntax-rules, it matches anything.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[yarl baudig]

> 
> Not going to work.
> , it matches anything.
> 
> 

Obviously...

Ok, I tried more.
Apparently,

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (syntax-rules (name ...)
	 ((_ name) id) ...)))))

works the same as

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (lambda (x)
	 (syntax-case x ()
	   ((_ n) (free-identifier=? #'name #'n) #'id) ...))))))

. Now what about

(define-syntax define-enumerate-type
  (syntax-rules ()
    ((_ name->int (name id) ...)
     (define-syntax name->int
       (lambda (x)
	 (syntax-case x ()
	   ((_ n) (equal?
		   (syntax->datum #'name)
		   (syntax->datum #'n))
	    #'id) ...))))))

?

Re: define-module, #:export and export

[Jean Abou Samra]

[-- Attachment #1.1: Type: text/plain, Size: 1249 bytes --]

Le 08/01/2023 à 16:18, yarl baudig a écrit :
> Obviously...
>
> Ok, I tried more.
> Apparently,
>
> (define-syntax define-enumerate-type
>    (syntax-rules ()
>      ((_ name->int (name id) ...)
>       (define-syntax name->int
>         (syntax-rules (name ...)
> 	 ((_ name) id) ...)))))
>
> works the same as
>
> (define-syntax define-enumerate-type
>    (syntax-rules ()
>      ((_ name->int (name id) ...)
>       (define-syntax name->int
>         (lambda (x)
> 	 (syntax-case x ()
> 	   ((_ n) (free-identifier=? #'name #'n) #'id) ...))))))
>
> . Now what about
>
> (define-syntax define-enumerate-type
>    (syntax-rules ()
>      ((_ name->int (name id) ...)
>       (define-syntax name->int
>         (lambda (x)
> 	 (syntax-case x ()
> 	   ((_ n) (equal?
> 		   (syntax->datum #'name)
> 		   (syntax->datum #'n))
> 	    #'id) ...))))))
>
> ?



Yes, that was Maxime's suggestion (c). It essentially means you take the 
symbol
value of the argument to the macro without caring about whether it is bound
or renamed.

If you're doing this, maybe it's simpler to just turn the macro into a 
function
that accepts symbol values and call it as (my-enum 'foo) instead of 
(my-enum foo)?




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: define-module, #:export and export

[yarl baudig]

For completeness, this works as well:

lib.scm:
(define-module (lib)
  #:use-module ((main) #:select (thing))
  #:export (test))

(define-syntax test
  (lambda (sintax)
    (syntax-case sintax ()
      ((test id)
       (free-identifier=? #'id #'thing)))))

main.scm:
(define-module (main)
  #:use-module (lib)
  #:export (thing))

(display (test thing))
(newline)
(define thing 5)