bug#73275: ‘with-imported-modules’ fails to cope with dangling symlinks

bug#73275: ‘with-imported-modules’ fails to cope with dangling symlinks

[Ludovic Courtès]

Consider this code, which uses ‘with-imported-modules’ with a computed
module ‘foo’ (thereby leading to the creation of a “module-import”
derivation, via ‘imported-files/derivation’):

--8<---------------cut here---------------start------------->8---
(use-modules (guix))

(pk (stat:type (lstat (search-path %load-path "guix/base16.scm"))))

(let* ((foo (scheme-file "foo.scm" #~(define-module (foo))))
       (bar (scheme-file "bar.scm"
                         (with-imported-modules `((guix base16)
                                                  ((foo) => ,foo))
                           #~(begin
                               (use-modules (guix base16) (foo))
                               (pk 'hi!))))))
  (with-store store
    (run-with-store store
      (mlet %store-monad ((drv (lower-object bar)))
        (built-derivations (list (pk 'drv drv)))))))
--8<---------------cut here---------------end--------------->8---

It works fine in “normal” conditions:

--8<---------------cut here---------------start------------->8---
$ guile imported-modules-dangling-symlink.scm

;;; (regular)

;;; (drv #<derivation /gnu/store/8lzl8p1rjwxnhdyvjiwa4w35pclm1a37-bar.scm.drv => /gnu/store/px8b5453gspxbjixi33x96hjm558s6gk-bar.scm 7f6f88c84f50>)
--8<---------------cut here---------------end--------------->8---

But it fails when running from a profile:

--8<---------------cut here---------------start------------->8---
$ guix shell -CWP guile guix -- guile imported-modules-dangling-symlink.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix-debugging/imported-modules-dangling-symlink.scm
;;; compiled /home/ludo/.cache/guile/ccache/3.0-LE-8-4.7/home/ludo/src/guix-debugging/imported-modules-dangling-symlink.scm.go

;;; (symlink)

;;; (drv #<derivation /gnu/store/8r2vh6x599i82gf9rphfxldaiyr67glq-bar.scm.drv => /gnu/store/a1sz94kb0h1n0k238w49bw4zdrrhml6h-bar.scm 7fce7f287640>)
building path(s) `/gnu/store/qmxv9mavj48jmqb3x4ayq2a2q948kd40-module-import-compiled'
[ 1/ 4] Loading './foo.scm'...
Backtrace:
           6 (primitive-load "/gnu/store/9c5vjhqsjxkg5fvaxhxz14w6ms3?")
In ice-9/eval.scm:
    619:8  5 (_ #f)
In srfi/srfi-1.scm:
   460:18  4 (fold #<procedure 7ffff5f2a9a0 at ice-9/eval.scm:336:1?> ?)
   460:18  3 (fold #<procedure 7ffff5f41c60 at ice-9/eval.scm:336:1?> ?)
In ice-9/eval.scm:
   245:16  2 (_ #(#(#<directory (guix build utils) 7ffff7746320>) # ?))
In ice-9/boot-9.scm:
  1982:24  1 (_ _)
In unknown file:
           0 (stat "./guix/base16.scm" #<undefined>)

ERROR: In procedure stat:
In procedure stat: No such file or directory: "./guix/base16.scm"
builder for `/gnu/store/f3w43lq1wfq35dym2r48sav22x4l88dh-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/8r2vh6x599i82gf9rphfxldaiyr67glq-bar.scm.drv': 1 dependencies couldn't be built
Backtrace:
In ice-9/boot-9.scm:
  1755:12 11 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
          10 (apply-smob/0 #<thunk 7fce95112300>)
In ice-9/boot-9.scm:
    724:2  9 (call-with-prompt _ _ #<procedure default-prompt-handle?>)
In ice-9/eval.scm:
    619:8  8 (_ #(#(#<directory (guile-user) 7fce95115c80>)))
In ice-9/boot-9.scm:
   2858:4  7 (save-module-excursion _)
  4408:12  6 (_)
  1755:12  5 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   689:37  4 (thunk)
  2210:25  3 (run-with-store #<store-connection 256.100 7fce90c59a50> ?)
  2082:38  2 (_ #<store-connection 256.100 7fce7f2875a0>)
  1449:15  1 (_ #<store-connection 256.100 7fce7f2875a0> _ _)
In ice-9/boot-9.scm:
  1676:22  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1676:22: In procedure raise-exception:
ERROR:
  1. &store-protocol-error:
      message: "build of `/gnu/store/8r2vh6x599i82gf9rphfxldaiyr67glq-bar.scm.drv' failed"
      status: 100
--8<---------------cut here---------------end--------------->8---

The key difference is that, in the profile, ‘guix/base16.scm’ is a
symlink and the import machinery ends up inserting a dangling symlink in
the store:

--8<---------------cut here---------------start------------->8---
$ guix gc --references /gnu/store/f3w43lq1wfq35dym2r48sav22x4l88dh-module-import-compiled.drv
/gnu/store/3w29f4qjbap3awsi7yngnp2l1zvb5gbc-module-import.drv
/gnu/store/4ah6s53jjfmcqnly3mdgh8zpkk0jyz28-guile-3.0.9.drv
/gnu/store/9c5vjhqsjxkg5fvaxhxz14w6ms317wsg-compile-modules
/gnu/store/c1qxdflcdb2hv4rjglap3qb5jfqhs72r-build-utils.scm
$ guix gc --references /gnu/store/3w29f4qjbap3awsi7yngnp2l1zvb5gbc-module-import.drv
/gnu/store/4ah6s53jjfmcqnly3mdgh8zpkk0jyz28-guile-3.0.9.drv
/gnu/store/c1qxdflcdb2hv4rjglap3qb5jfqhs72r-build-utils.scm
/gnu/store/g0660kdr7n20927cxx2pmmr9dyj4l4q4-foo.scm.drv
/gnu/store/mg756z8cykpz9szdk71c5zchyymyl2br-module-import-builder
/gnu/store/nkz7k36n9i7qwvwk7ayyd9sq4ssdghxh-base16.scm
$ ls -l /gnu/store/nkz7k36n9i7qwvwk7ayyd9sq4ssdghxh-base16.scm
lrwxrwxrwx 1 root root 102 Jan  1  1970 /gnu/store/nkz7k36n9i7qwvwk7ayyd9sq4ssdghxh-base16.scm -> /gnu/store/jk9pi1qk158zdmnh4q3nzp3l53cvzhnk-guix-1.4.0-24.9a2ddcc/share/guile/site/3.0/guix/base16.scm
$ guix gc --references /gnu/store/nkz7k36n9i7qwvwk7ayyd9sq4ssdghxh-base16.scm | grep base16
$ echo $?
1
--8<---------------cut here---------------end--------------->8---

Note that the symlink is valid when you have a full view of the store,
but it’s dangling inside the build environment because that
/gnu/store/nkz7k36n9i7qwvwk7ayyd9sq4ssdghxh-base16.scm item doesn’t have
a reference to its target file, as shown by ‘guix gc --references’.

There’s no such problem on regular imports as created by
‘imported-files’.

This is sorta similar to the issue discussed in
<https://issues.guix.gnu.org/72867>.

Ludo’.

bug#73275: ‘with-imported-modules’ fails to cope with dangling symlinks

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> $ guix shell -CWP guile guix -- guile imported-modules-dangling-symlink.scm
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/ludo/src/guix-debugging/imported-modules-dangling-symlink.scm
> ;;; compiled /home/ludo/.cache/guile/ccache/3.0-LE-8-4.7/home/ludo/src/guix-debugging/imported-modules-dangling-symlink.scm.go
>
> ;;; (symlink)
>
> ;;; (drv #<derivation /gnu/store/8r2vh6x599i82gf9rphfxldaiyr67glq-bar.scm.drv => /gnu/store/a1sz94kb0h1n0k238w49bw4zdrrhml6h-bar.scm 7fce7f287640>)
> building path(s) `/gnu/store/qmxv9mavj48jmqb3x4ayq2a2q948kd40-module-import-compiled'
> [ 1/ 4] Loading './foo.scm'...
> Backtrace:
>            6 (primitive-load "/gnu/store/9c5vjhqsjxkg5fvaxhxz14w6ms3?")
> In ice-9/eval.scm:
>     619:8  5 (_ #f)
> In srfi/srfi-1.scm:
>    460:18  4 (fold #<procedure 7ffff5f2a9a0 at ice-9/eval.scm:336:1?> ?)
>    460:18  3 (fold #<procedure 7ffff5f41c60 at ice-9/eval.scm:336:1?> ?)
> In ice-9/eval.scm:
>    245:16  2 (_ #(#(#<directory (guix build utils) 7ffff7746320>) # ?))
> In ice-9/boot-9.scm:
>   1982:24  1 (_ _)
> In unknown file:
>            0 (stat "./guix/base16.scm" #<undefined>)
>
> ERROR: In procedure stat:
> In procedure stat: No such file or directory: "./guix/base16.scm"
> builder for `/gnu/store/f3w43lq1wfq35dym2r48sav22x4l88dh-module-import-compiled.drv' failed with exit code 1

Fixed in a7bb45b39d7d698e0868c2b3ac9097b70ef9d401.

The trick is that ‘imported-files/derivation’ should copy files instead
of symlinking them.  As noted in the commit log, I believe this was the
original intent but due to a typo it did not work that way.

Ludo’.