Symlinks in "guix pack" and "guix shell"

Symlinks in "guix pack" and "guix shell"

[Konrad Hinsen]

Dear Guix experts,

I am seeing inconsistent behavior with the creation of symbolic links
in containers and exports, and I wonder whether this is a bug or missing
documentation.

According to the documentation, there are no constraints on what I can
link to. In practice, some symbolic link requests have no effect, but
there is no error message either.

Example: I am trying to tell programs in a container to use the
certificates from package nss-certs, instead of requiring the user
to share /etc/ssl from the host machine. So I add 

   -S /etc/ssl=etc/ssl

to my command lines for "guix shell -C" and "guix pack" (for creating
Docker or Singularity images.

Sometimes this works, sometimes it doesn't. With "guix shell", I never
got it to work. With "guix pack", the observed behavior is random.
Adding a completely unrelated package to my container can make the
difference.

Can someone shed some light on this?

Cheers,
  Konrad

Re: Symlinks in "guix pack" and "guix shell"

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 3707 bytes --]

On 2024-01-29 10:30:21 +0100, Konrad Hinsen wrote:
> Dear Guix experts,
>
> I am seeing inconsistent behavior with the creation of symbolic links
> in containers and exports, and I wonder whether this is a bug or missing
> documentation.
>
> According to the documentation, there are no constraints on what I can
> link to. In practice, some symbolic link requests have no effect, but
> there is no error message either.

That is interesting, at least in the guix shell case, I do get an error:

    $ guix shell -S /etc/ssl=etc/ssl -C bash coreutils -- ls -l /etc
    Backtrace:
    In guix/ui.scm:
      2287:10 19 (run-guix-command _ . _)
    In ice-9/boot-9.scm:
      1752:10 18 (with-exception-handler _ _ #:unwind? _ # _)
    In guix/store.scm:
        674:3 17 (guix-environment* _)
    In ice-9/boot-9.scm:
      1752:10 16 (with-exception-handler _ _ #:unwind? _ # _)
    In guix/store.scm:
       661:37 15 (thunk)
       1300:8 14 (call-with-build-handler _ _)
       1300:8 13 (call-with-build-handler #<procedure 7f3c6eb4e6f0 at g…> …)
    In guix/status.scm:
        839:4 12 (call-with-status-report _ _)
    In guix/scripts/environment.scm:
      1162:11 11 (_)
    In guix/store.scm:
      2180:25 10 (run-with-store #<store-connection 256.99 7f3c6dc21d70> …)
    In guix/scripts/environment.scm:
        873:8  9 (_ _)
    In gnu/build/linux-container.scm:
       397:16  8 (call-with-container (#<<file-system> device: "none…> …) …)
       265:16  7 (run-container _ _ (cgroup mnt pid ipc uts user net) _ # …)
    In guix/scripts/environment.scm:
       903:14  6 (_)
    In srfi/srfi-1.scm:
        634:9  5 (for-each #<procedure 7f3c7f64ecc0 at guix/scripts/env…> …)
    In ice-9/boot-9.scm:
      1747:15  4 (with-exception-handler #<procedure 7f3c6afeca80 at ic…> …)
      1747:15  3 (with-exception-handler #<procedure 7f3c6afeca20 at ic…> …)
    In gnu/build/install.scm:
       108:23  2 (_)
    In ice-9/boot-9.scm:
      1685:16  1 (raise-exception _ #:continuable? _)
      1685:16  0 (raise-exception _ #:continuable? _)

    ice-9/boot-9.scm:1685:16: In procedure raise-exception:
    symlink `.//etc/ssl' points to nonexistent file `/gnu/store/kxvplpa4c0xf5pap6br8abip2bq9f0p8-profile/etc/ssl'

>
> Example: I am trying to tell programs in a container to use the
> certificates from package nss-certs, instead of requiring the user
> to share /etc/ssl from the host machine. So I add
>
>    -S /etc/ssl=etc/ssl
>
> to my command lines for "guix shell -C" and "guix pack" (for creating
> Docker or Singularity images.
>
> Sometimes this works, sometimes it doesn't. With "guix shell", I never
> got it to work. With "guix pack", the observed behavior is random.
> Adding a completely unrelated package to my container can make the
> difference.

I think the relevant part is whether some certificate package is installed (via
propagated inputs) or not.  If I explicitly add nss-certs to the command above,
it starts to work:

    $ guix shell -S /etc/ssl=etc/ssl -C bash coreutils nss-certs -- ls -l /etc
    total 12
    -rw-r--r-- 1 wolf users 32 Jan 29 11:14 group
    -rw------- 1 wolf users 20 Jan 29 11:14 hosts
    -rw-r--r-- 1 wolf users 92 Jan 29 11:14 passwd
    lrwxrwxrwx 1 wolf users 59 Jan 29 11:14 ssl -> /gnu/store/s0mdc10pkb9kcmnzjfywax1d42rm4car-profile/etc/ssl

Could you please share some examples where you get no error, but it has no
effect?  Maybe they really are bugs (and should be reported).

Have a nice day,
Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Symlinks in "guix pack" and "guix shell"

[Konrad Hinsen]

Hi Tomas,

Thanks for your reply!

> I think the relevant part is whether some certificate package is installed (via
> propagated inputs) or not.  If I explicitly add nss-certs to the command above,
> it starts to work:

Sorry, I forgot to add that I always have nss-certs among my packages
when I try this symlink. Otherwise I get an error message as well.

> Could you please share some examples where you get no error, but it has no
> effect?  Maybe they really are bugs (and should be reported).

Here is my most recent experiment:

$ guix time-machine --url=https://codeberg.org/khinsen/guix.git --commit=615023ddc9407f27ec7c04060fb520a4cbb3b03f  --disable-authentication -- pack -S /etc/ssl=etc/ssl -S /etc/protocols=etc/protocols --format=squashfs --save-provenance snakemake bash git git-annex coreutils wget python nss-certs net-base
guix time-machine: warning: channel authentication disabled
guix pack: warning: ambiguous package specification `python'
guix pack: warning: choosing python@3.10.7 from gnu/packages/python.scm:610:2
/gnu/store/psq0bqcsnbk7p5w1m2771mlilyikicp6-snakemake-bash-git-git-annex-coreutils-squashfs-pack.gz.squashfs

$ singularity exec /gnu/store/psq0bqcsnbk7p5w1m2771mlilyikicp6-snakemake-bash-git-git-annex-coreutils-squashfs-pack.gz.squashfs bash
WARNING: passwd file doesn't exist in container, not updating
WARNING: group file doesn't exist in container, not updating
Singularity> ls /etc
hosts  localtime  resolv.conf
Singularity> exit

Neither /etc/ssl nor /etc/protocols are present in the container.

On the other hand, the same packages exported for Docker:

$ guix time-machine --url=https://codeberg.org/khinsen/guix.git --commit=615023ddc9407f27ec7c04060fb520a4cbb3b03f  --disable-authentication -- pack -S /etc/ssl=etc/ssl -S /etc/protocols=etc/protocols --format=docker --save-provenance snakemake bash git git-annex coreutils wget python nss-certs net-base
guix time-machine: warning: channel authentication disabled
guix pack: warning: ambiguous package specification `python'
guix pack: warning: choosing python@3.10.7 from gnu/packages/python.scm:432:2
/gnu/store/90ylyh60vvi2inqg9i8wlcfbv8m2px89-snakemake-bash-git-git-annex-coreutils-docker-pack.tar.gz

$ docker load -i /gnu/store/90ylyh60vvi2inqg9i8wlcfbv8m2px89-snakemake-bash-git-git-annex-coreutils-docker-pack.tar.gz
Loaded image: snakemake-bash-git-git-annex-coreutils:latest

$ docker run -it snakemake-bash-git-git-annex-coreutils:latest bash
bash-5.1# ls /etc
ls /etc
hostname  hosts  mtab  protocols  resolv.conf  ssl

Both links are present.

Finally, "guix shell" now works (I hadn't tried for a few months):

$ guix time-machine --url=https://codeberg.org/khinsen/guix.git --commit=615023ddc9407f27ec7c04060fb520a4cbb3b03f  --disable-authentication -- shell -C -N -S /etc/ssl=etc/ssl -S /etc/protocols=etc/protocols snakemake bash git git-annex coreutils wget python nss-certs net-base
guix time-machine: warning: channel authentication disabled
guix shell: warning: ambiguous package specification `python'
guix shell: warning: choosing python@3.10.7 from gnu/packages/python.scm:432:2
[env]$ ls /etc
group  hosts  nsswitch.conf  passwd  protocols	resolv.conf  services  ssl

For this example, the only failure is for "pack –format=squashfs".
But the very same command line applied to a different commit (a few
weeks earlier) worked fine.

In case you wonder why I use a personal fork of Guix: For Singularity
export to be usable, I need a patch that is on core-updates
(https://git.savannah.gnu.org/cgit/guix.git/commit/?h=core-updates&id=d8875d76555faf1a11f6838312bfe4b7f6121d2c). In
the meantime, I added a graft version of this patch to my personal fork.

Finally, I cleaned up the output of the above commands by removing the
usual messages from the build process, keeping only the messages that
are specific to my example.

Cheers,
  Konrad.

Re: Symlinks in "guix pack" and "guix shell"

[Konrad Hinsen]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> Here is my most recent experiment:

I found a simpler one, using a more recent Guix commit:

   guix time-machine --commit=7b0863f07a113caef26fea13909bd97d250b629e \
   -- pack -S /etc/ssl=etc/ssl --format=squashfs bash nss-certs

Unfortunately, I have no idea how to debug this, as the image is
constructed by the daemon.

Cheers,
  Konrad

Re: Symlinks in "guix pack" and "guix shell"

[Konrad Hinsen]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> I found a simpler one, using a more recent Guix commit:
>
>    guix time-machine --commit=7b0863f07a113caef26fea13909bd97d250b629e \
>    -- pack -S /etc/ssl=etc/ssl --format=squashfs bash nss-certs
>
> Unfortunately, I have no idea how to debug this, as the image is
> constructed by the daemon.

I think I found the bug, by playing with variations on my command line:

   https://issues.guix.gnu.org/68841

Cheers,
  Konrad.