bug#62287: Ungexp inside vector problem

bug#62287: Ungexp inside vector problem

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 566 bytes --]

I would expect two last expressions return the same result, but the
former one doesn't do ungexp:

--8<---------------cut here---------------start------------->8---
(define a '(3 4))

(define b `#(1 2 ,a))

(eval-with-store
 #~(list '(1 2 #$a))) ;; => ((1 2 (3 4)))

(eval-with-store
 #~(list #(1 2 #$a))) ;; => (#(1 2 (ungexp a)))

(eval-with-store
 #~(list #$b)) ;; => (#(1 2 4))
--8<---------------cut here---------------end--------------->8---

Am I doing/expecting something wrong or there is a bug here?

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#62287: Ungexp inside vector problem

[Josselin Poiret via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]

Hi Andrew,

Andrew Tropin <andrew@trop.in> writes:

> I would expect two last expressions return the same result, but the
> former one doesn't do ungexp:
>
> --8<---------------cut here---------------start------------->8---
> (define a '(3 4))
>
> (define b `#(1 2 ,a))
>
> (eval-with-store
>  #~(list '(1 2 #$a))) ;; => ((1 2 (3 4)))
>
> (eval-with-store
>  #~(list #(1 2 #$a))) ;; => (#(1 2 (ungexp a)))
>
> (eval-with-store
>  #~(list #$b)) ;; => (#(1 2 4))
> --8<---------------cut here---------------end--------------->8---
>
> Am I doing/expecting something wrong or there is a bug here?

It's more related to how the guile reader works, and this is such a
corner case that I don't know whether we should fix.  Basically,
anything starting with # is a reader extension, and the next character
identifies which extension it is.  #( is the reader extension for
vectors, #~ for gexp and #$ for ungexp.

To simplify, whenever you use #~, guile will insert (gexp ...) instead,
which is a hygienic macro (not just a procedure!), that will look for
ungexps inside the expression.  That traversal is only made on cons
cells though, so it doesn't try to go through any piece of syntax that
is not a cons cell!  Since #( doesn't expand to a (vector ...)
cons-cell, the subexpression gets ignored for traversal.

This is in contrast to another reader extension, #' (for syntax), which
does expand to (syntax ...), and is thus further traversed!

You can find how both of these reader extensions operate in
<libguile/read.c>.

I guess the immediate fix is to use (vector ...) rather than #(...).  We
could also add code to the gexp traversal to also traverse vectors, but
I am not even sure if they go through the gexp->sexp dance unharmed, and
we also should in principle do that for everything that can get into a
gexp, not just vectors (eg. bytevectors).

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

bug#62287: Ungexp inside vector problem

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 2271 bytes --]

On 2023-03-20 11:45, Josselin Poiret wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> writes:
>
>> I would expect two last expressions return the same result, but the
>> former one doesn't do ungexp:
>>
>> --8<---------------cut here---------------start------------->8---
>> (define a '(3 4))
>>
>> (define b `#(1 2 ,a))
>>
>> (eval-with-store
>>  #~(list '(1 2 #$a))) ;; => ((1 2 (3 4)))
>>
>> (eval-with-store
>>  #~(list #(1 2 #$a))) ;; => (#(1 2 (ungexp a)))
>>
>> (eval-with-store
>>  #~(list #$b)) ;; => (#(1 2 4))
>> --8<---------------cut here---------------end--------------->8---
>>
>> Am I doing/expecting something wrong or there is a bug here?
>
> It's more related to how the guile reader works, and this is such a
> corner case that I don't know whether we should fix.  Basically,
> anything starting with # is a reader extension, and the next character
> identifies which extension it is.  #( is the reader extension for
> vectors, #~ for gexp and #$ for ungexp.
>
> To simplify, whenever you use #~, guile will insert (gexp ...) instead,
> which is a hygienic macro (not just a procedure!), that will look for
> ungexps inside the expression.  That traversal is only made on cons
> cells though, so it doesn't try to go through any piece of syntax that
> is not a cons cell!  Since #( doesn't expand to a (vector ...)
> cons-cell, the subexpression gets ignored for traversal.
>
> This is in contrast to another reader extension, #' (for syntax), which
> does expand to (syntax ...), and is thus further traversed!
>
> You can find how both of these reader extensions operate in
> <libguile/read.c>.
>
> I guess the immediate fix is to use (vector ...) rather than #(...).  We
> could also add code to the gexp traversal to also traverse vectors, but
> I am not even sure if they go through the gexp->sexp dance unharmed, and
> we also should in principle do that for everything that can get into a
> gexp, not just vectors (eg. bytevectors).
>

Thank you very much for extensive explanation!  I have a few tasks
related to the guile reader extensions, so when I get my hands dirty
with it, I'll probably share my new thoughts and opinions on this topic.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]