bug#54003: local-file after Guile update

bug#54003: local-file after Guile update

[Aleksandr Vityazev]

Hi,

I have the following dotfiles directory structure where I store my Guix home
config:

|-etc
|
|-subdir
|   |
|   |-home


I often use the 'local-file' procedure in modules in the home directory. For
example in /subdir/home/shells.scm:

#+begin_src scheme :scheme guile :season guile
(simple-service 'add-zsh-files
                home-files-service-type
                `(("config/zsh/zshrc"
                   ,(local-file "../../etc/zsh/zshrc"))))
#+end_src
And it works, as expected.

The following example also worked before the
076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
'guile-3.0-latest') commit.

in /subdir/home/wm.scm: 
#+begin_src scheme :scheme guile :season guile
(service
 home-rofi-service-type
 (home-rofi-configuration
  (config-rasi
   `(,#~(format
         #f "@theme \"~a\""
         #$(local-file "../../etc/rofi/nord.rasi"))))))
#+end_src

After 076e825dc5d585943ce820a279fffe4af09757fb
using (local-file "../../etc/rofi/nord.rasi") and not
(local-file "etc/rofi/nord.rasi") causes an error:

guix home: error: canonicalize-path: No such file or
directory: "../../etc/rofi/nord.rasi".

The example that causes the error uses the original Guix home implementation,
which uses gexps.  When I figure out how to make a minimal reproducer that
does not depend on it, I will update this report.

-- 
Best regards,
Aleksandr Vityazev

bug#54003: local-file after Guile update

[Liliana Marie Prikler]

Hi Aleksandr

Am Montag, dem 14.02.2022 um 21:50 +0000 schrieb Aleksandr Vityazev:
> Hi,
> 
> I have the following dotfiles directory structure where I store my
> Guix home config:
> 
> > -etc
> > 
> > -subdir
> >   |
> >   |-home
> 
> 
> I often use the 'local-file' procedure in modules in the home
> directory. For example in /subdir/home/shells.scm:
> 
> #+begin_src scheme :scheme guile :season guile
> (simple-service 'add-zsh-files
>                 home-files-service-type
>                 `(("config/zsh/zshrc"
>                    ,(local-file "../../etc/zsh/zshrc"))))
> #+end_src
> And it works, as expected.
Be careful when invoking Hyrum's Law.  The meaning of this local-file
expression changes with the current working directory, which might be
different across invocations.  To avoid such issues, you can (almost?)
always use (current-file-name), which you might want to canonicalize
for the operations you're about to do.
Taking (dirname) thrice then brings you to $GUIX_HOME_CONFIG_ROOT,
which contains both etc and subdir.

> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version
> 3.0.8 as
> 'guile-3.0-latest') commit.
> 
> in /subdir/home/wm.scm: 
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
> 
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
> 
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".
> 
> The example that causes the error uses the original Guix home
> implementation, which uses gexps.  When I figure out how to make a
> minimal reproducer that does not depend on it, I will update this
> report.
I don't think this is an issue particularly in Guix Home, but for a MWE
you could try only writing a bashrc, which ought to work with upstream
as well.  Use `guix home build` rather than `guix home reconfigure` so
as to not activate it.

Cheers

bug#54003: local-file after Guile update

[Ludovic Courtès]

Hi,

Aleksandr Vityazev <avityazev@posteo.org> skribis:

> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
> 'guile-3.0-latest') commit.
>
> in /subdir/home/wm.scm: 
>
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
>
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
>
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".

I’ve witnessed a similar problem that I worked around in
c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Needs some investigation…

Ludo’.

bug#54003: local-file after Guile update

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Aleksandr Vityazev <avityazev@posteo.org> skribis:
>
>> The following example also worked before the
>> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
>> 'guile-3.0-latest') commit.
>>
>> in /subdir/home/wm.scm: 
>>
>> #+begin_src scheme :scheme guile :season guile
>> (service
>>  home-rofi-service-type
>>  (home-rofi-configuration
>>   (config-rasi
>>    `(,#~(format
>>          #f "@theme \"~a\""
>>          #$(local-file "../../etc/rofi/nord.rasi"))))))
>> #+end_src
>>
>> After 076e825dc5d585943ce820a279fffe4af09757fb
>> using (local-file "../../etc/rofi/nord.rasi") and not
>> (local-file "etc/rofi/nord.rasi") causes an error:
>>
>> guix home: error: canonicalize-path: No such file or
>> directory: "../../etc/rofi/nord.rasi".
>
> I’ve witnessed a similar problem that I worked around in
> c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Another example is the berlin configuration; attempting to reconfigure
it currently yields:

--8<---------------cut here---------------start------------->8---
$ sudo guix system reconfigure -L modules berlin.scm
guix system: warning: the following accounts appear more than once: zabbix
guix system: warning: the following groups appear more than once: zabbix
guix system: error: canonicalize-path: No such file or directory: "../../sync-disarchive-db.scm"
--8<---------------cut here---------------end--------------->8---

I'm positive this is new behavior.

Thanks,

Maxim

bug#54003: local-file after Guile update

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

Hi!

For now, here’s a workaround: instead of writing

  #~(… #$(local-file …) …)

you can write:

  (gexp (… (ungexp (local-file …)) …))

or:

  (let ((file (local-file …)))
    #~(… #$file …))

The problem is that ‘read’ hash extensions (like those for #~ and #$)
would so far return sexps with associated source properties, but psyntax
in 3.0.8 ignores those source properties.

I believe this is due to Guile commit
54bbe0b2846c5b1aa366c91d679ba724869c8cda, specifically this bit:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 203 bytes --]

     (define source-annotation
       (lambda (x)
-        (if (syntax? x)
-            (syntax-sourcev x)
-            (datum-sourcev x))))
+        (and (syntax? x)
+             (syntax-sourcev x))))

[-- Attachment #3: Type: text/plain, Size: 73 bytes --]


I’m looking for a reasonable workaround we could apply.

Ludo’.

bug#54003: local-file after Guile update

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]

Ludovic Courtès schreef op di 15-02-2022 om 20:42 [+0100]:
> The problem is that ‘read’ hash extensions (like those for #~ and #$)
> would so far return sexps with associated source properties, but
> psyntax
> in 3.0.8 ignores those source properties.

Read hash extensions can return syntax objects.  In a Guile 3.0.8 REPL:

(define-module (hat))
(define (hat s-exp)
  `#(hat ,s-exp))

(read-hash-extend #\^
  (lambda (_ port)
    ;; Use 'read-syntax' instead of a combination of 'read' and
    ;; 'datum->syntax' for source properties.
    #`(hat '#,(read-syntax port))))

#^(foo bar)
$4 = #(hat (foo bar))

Doing something like this in the hash extension for G-exps might work.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#54003: local-file after Guile update

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Read hash extensions can return syntax objects.  In a Guile 3.0.8 REPL:
>
> (define-module (hat))
> (define (hat s-exp)
>   `#(hat ,s-exp))
>
> (read-hash-extend #\^
>   (lambda (_ port)
>     ;; Use 'read-syntax' instead of a combination of 'read' and
>     ;; 'datum->syntax' for source properties.
>     #`(hat '#,(read-syntax port))))
>
> #^(foo bar)
> $4 = #(hat (foo bar))
>
> Doing something like this in the hash extension for G-exps might work.

Yes, that’s one possible workaround, but then ‘read’ would find itself
returning syntax objects as well.

We have to see to what extent this is a problem in Guix proper; I think
it will at least break ‘guix style’ and related code.

Ludo’.

bug#54003: local-file after Guile update

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

Hi!

This *ahem* pretty hack works around the problem.  I have yet to see if
it works with earlier Guile versions.  Thoughts?

I’d have preferred to avoid monkey-patching but I couldn’t think of a
way to do that.

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 5989 bytes --]

From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Wed, 16 Feb 2022 11:27:43 +0100
Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.

Read hash extensions preserve source location info as source properties
on their result.  However, in Guile 3.0.8, that location would be
dismissed, leading 'local-file' to fail to resolve file names relative
to the source directory.

Fixes <https://issues.guix.gnu.org/54003>.
Reported by Aleksandr Vityazev <avityazev@posteo.org>.

* guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
[read-syntax*]: New variables.
[read-ungexp]: Adjust to expect either sexps or syntax objects.
[read-gexp]: Call 'read-procedure'.
* tests/gexp.scm ("local-file, relative file name, within gexp")
("local-file, relative file name, within gexp, compiled"): New tests.
---
 guix/gexp.scm  | 50 +++++++++++++++++++++++++++++++++++++++++++++-----
 tests/gexp.scm | 27 +++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 5 deletions(-)

diff --git a/guix/gexp.scm b/guix/gexp.scm
index d23683e2a6..e229c1fc8f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -2176,6 +2176,29 @@ (define log-port
 ;;;
 
 (eval-when (expand load eval)
+  (define-once read-syntax-redefined?
+    ;; Have we already redefined 'read-syntax'?  This needs to be done on
+    ;; 3.0.8 only to work around <https://issues.guix.gnu.org/54003>.
+    (or (not (module-variable the-scm-module 'read-syntax))
+        (not (guile-version>? "3.0.7"))))
+
+  (define read-procedure
+    ;; The current read procedure being called: either 'read' or
+    ;; 'read-syntax'.
+    (make-parameter read))
+
+  (define read-syntax*
+    ;; Replacement for 'read-syntax'.
+    (let ((read-syntax (and=> (module-variable the-scm-module 'read-syntax)
+                              variable-ref)))
+      (lambda (port . rest)
+        (parameterize ((read-procedure read-syntax))
+          (apply read-syntax port rest)))))
+
+  (unless read-syntax-redefined?
+    (set! (@ (guile) read-syntax) read-syntax*)
+    (set! read-syntax-redefined? #t))
+
   (define* (read-ungexp chr port #:optional native?)
     "Read an 'ungexp' or 'ungexp-splicing' form from PORT.  When NATIVE? is
 true, use 'ungexp-native' and 'ungexp-native-splicing' instead."
@@ -2191,22 +2214,39 @@ (define unquote-symbol
              'ungexp-native
              'ungexp))))
 
-    (match (read port)
-      ((? symbol? symbol)
-       (let ((str (symbol->string symbol)))
+    (define symbolic?
+      ;; Depending on whether (read-procedure) is 'read' or 'read-syntax', we
+      ;; might get either sexps or syntax objects.  Adjust accordingly.
+      (if (eq? (read-procedure) read)
+          symbol?
+          (compose symbol? syntax->datum)))
+
+    (define symbolic->string
+      (if (eq? (read-procedure) read)
+          symbol->string
+          (compose symbol->string syntax->datum)))
+
+    (define wrapped-symbol
+      (if (eq? (read-procedure) read)
+          (lambda (_ symbol) symbol)
+          datum->syntax))
+
+    (match ((read-procedure) port)
+      ((? symbolic? symbol)
+       (let ((str (symbolic->string symbol)))
          (match (string-index-right str #\:)
            (#f
             `(,unquote-symbol ,symbol))
            (colon
             (let ((name   (string->symbol (substring str 0 colon)))
                   (output (substring str (+ colon 1))))
-              `(,unquote-symbol ,name ,output))))))
+              `(,unquote-symbol ,(wrapped-symbol symbol name) ,output))))))
       (x
        `(,unquote-symbol ,x))))
 
   (define (read-gexp chr port)
     "Read a 'gexp' form from PORT."
-    `(gexp ,(read port)))
+    `(gexp ,((read-procedure) port)))
 
   ;; Extend the reader
   (read-hash-extend #\~ read-gexp)
diff --git a/tests/gexp.scm b/tests/gexp.scm
index bcda516623..33c0e4bf8c 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -28,6 +28,7 @@ (define-module (test-gexp)
   #:use-module (guix tests)
   #:use-module ((guix build utils) #:select (with-directory-excursion))
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
+  #:use-module ((guix ui) #:select (load*))
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bootstrap)
@@ -222,6 +223,32 @@ (define defmod 'define-module) ;fool Geiser
       (let ((file (local-file (string-copy "../base32.scm"))))
         (local-file-absolute-file-name file)))))
 
+(test-assert "local-file, relative file name, within gexp"
+  (let* ((file     (search-path %load-path "guix/base32.scm"))
+         (interned (add-to-store %store "base32.scm" #f "sha256" file)))
+    (equal? `(the file is ,interned)
+            (gexp->sexp*
+             #~(the file is #$(local-file "../guix/base32.scm"))))))
+
+(test-assert "local-file, relative file name, within gexp, compiled"
+  ;; In Guile 3.0.8, everything read by the #~ and #$ read hash extensions
+  ;; would lack source location info, which in turn would lead
+  ;; (current-source-directory), called by 'local-file', to return #f, thereby
+  ;; breaking 'local-file' resolution.  See
+  ;; <https://issues.guix.gnu.org/54003>.
+  (let ((file (tmpnam)))
+    (call-with-output-file file
+      (lambda (port)
+        (display (string-append "#~(this file is #$(local-file \""
+                                (basename file) "\" \"t.scm\"))")
+                 port)))
+
+    (let* ((interned (add-to-store %store "t.scm" #f "sha256" file))
+           (module   (make-fresh-user-module)))
+      (module-use! module (resolve-interface '(guix gexp)))
+      (equal? `(this file is ,interned)
+              (gexp->sexp* (load* file module))))))
+
 (test-assertm "local-file, #:select?"
   (mlet* %store-monad ((select? -> (lambda (file stat)
                                      (member (basename file)

base-commit: 791069737c8c51582cc021438dae32eb0fb7b8e0
-- 
2.34.0

bug#54003: local-file after Guile update

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
> Date: Wed, 16 Feb 2022 11:27:43 +0100
> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>
> Read hash extensions preserve source location info as source properties
> on their result.  However, in Guile 3.0.8, that location would be
> dismissed, leading 'local-file' to fail to resolve file names relative
> to the source directory.
>
> Fixes <https://issues.guix.gnu.org/54003>.
> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>
> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
> [read-syntax*]: New variables.
> [read-ungexp]: Adjust to expect either sexps or syntax objects.
> [read-gexp]: Call 'read-procedure'.
> * tests/gexp.scm ("local-file, relative file name, within gexp")
> ("local-file, relative file name, within gexp, compiled"): New tests.

Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
it works with 3.0.7.

I’m leaving the bug open so we can see what to do on the Guile side.

Ludo’.

bug#54003: 'local-file' fails to resolve relative file names with Guile 3.0.8

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>>>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Wed, 16 Feb 2022 11:27:43 +0100
>> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>>
>> Read hash extensions preserve source location info as source properties
>> on their result.  However, in Guile 3.0.8, that location would be
>> dismissed, leading 'local-file' to fail to resolve file names relative
>> to the source directory.
>>
>> Fixes <https://issues.guix.gnu.org/54003>.
>> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>>
>> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
>> [read-syntax*]: New variables.
>> [read-ungexp]: Adjust to expect either sexps or syntax objects.
>> [read-gexp]: Call 'read-procedure'.
>> * tests/gexp.scm ("local-file, relative file name, within gexp")
>> ("local-file, relative file name, within gexp, compiled"): New tests.
>
> Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
> it works with 3.0.7.
>
> I’m leaving the bug open so we can see what to do on the Guile side.

Fixed in Guile:

  https://git.savannah.gnu.org/cgit/guile.git/commit/?id=347321ece9fc85ddf74af3c798230b7b187fbce9

Closing!

Ludo’.