Re: Introducing Guix to HPC at my institution

["Ludovic Courtès" <ludovic.courtes@inria.fr>]

Howdy!

Sébastien Lerique <sl@eauchat.org> skribis:

> Case 1: with substitutes, on a local (non-NFS) folder
> -----------------------------------------------------
>
>  $ guix archive --authorize <   bin/../share/guix/ci.guix.gnu.org.pub
>  $ guix-daemon --disable-chroot &
>  $ guix install hello
>  [... dowloading ...]
>  unexpected substituter message 'defining `GUIX_LOCPATH', along
>  these lines:
>
>       guix install glibc-utf8-locales
>       export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"
>
>  See the "Application Setup" section in the manual, for more   info.
>  '
>
> I think this is https://issues.guix.gnu.org/45166.
>
> Using instead `guix pack -R -S /bin=bin -S /etc=etc -S /lib=lib guix
> bash glibc-utf8-locales`, and:
>
>  $ export GUIX_LOCPATH=$(pwd)/lib/locale
>  $ ls $GUIX_LOCPATH/
>  2.31
>
> seems to start working, but then `guix install hello` fails with the
> same error, and:
>
>  $ ls $GUIX_LOCPATH/
>  ls: cannot access '/tmp/sebastien-lerique/guix/lib/locale/': No
>  such file or directory
>
> Is it possible that glibc-utf8-locales gets garbage-collected?

Can you run guix-daemon like so?

  LC_ALL=en_US.utf8 guix-daemon --disable-chroot &

The “unexpected substitute message” thing is a bug: it turns out that
stderr of ‘guix substitute’ is consumed directly by the daemon at this
point, which it shouldn’t (similar to the issue fixed by
ee3226e9d54891c7e696912245e4904435be191c).

> Case 2: from source (no substitutes), on a local (non-NFS) folder
> -----------------------------------------------------------------
>
> No need for the locale fix here (another bug appears before that):
>
>  $ guix-daemon --disable-chroot &
>  # --no-offload works around the missing nscd problem; up to now
>      this
>  # doesn't seem necessary when using substitutes
>  $ guix install hello --no-offload
>  [... builds for a while ...]
>  build of
>  /gnu/store/pkn1w1q3xkn273kpmggc4dnq6n6hr9jy-bzip2-mesboot-1.0.8.drv 
>  failed
>
> The build log for bzip2-mesboot ends with:
>
>  tcc -ar cq libbz2.a blocksort.o huffman.o crctable.o randtable.o
>  compress.o decompress.o bzlib.o
>  ranlib libbz2.a
>  /gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0/bin/sh:
>  ranlib: Command not found.
>  make: *** [libbz2.a] Error 127
>  command "make" "CC=tcc -I ." "AR=tcc -ar" "bzip2"
>  "PREFIX=/gnu/store/s94hyrv1vgllxir5niiyzfc9g80l5kcd-bzip2-mesboot-1.0.8" 
>  failed with status 2
>
> Is this new and should be reported as a bug? Should binutils-mesboot0
> be part of bzip2-mesboot's inputs? (I haven't learned about the
> boostrapping mechanism yet.)

The problem is that ‘--disable-chroot’ is a bit of the wild west: build
processes can access the whole file system and what you do as a user can
interfere with them.

It could be that the bzip2 build failure above is just that: the build
process picks something from /usr/lib or /usr/bin, and that breaks
everything.

I think ‘--disable-chroot’ is OK if you’re going to use substitutes for
almost everything.  Otherwise, it’s not good.  Your use case calls for
built-in support; that way, the daemon could take still advantage of
user namespaces to set up a chroot and all.

> Case 3: with substitutes, on an NFS share
> -----------------------------------------
>
> Again no need for the locale fix (another bug appears before that):
>
>  $ guix archive --authorize <   bin/../share/guix/ci.guix.gnu.org.pub
>  $ guix-daemon --disable-chroot &
>  $ guix install hello
>  [... downloading ...]
>  guix install: error: cannot unlink
>  `/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib': 
>  Directory not empty
>
> so checking:
>
>  $ ls -lA   /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib
>  total 3840
>  -r-xr-xr-x 1 sebastien-lerique froeseuni   64832 Jan  1  1970
>   .nfs000000000172caa5000054bd
>  [... more .nfs files ...]

Ah yes, that’s a known issue with NFS.  I’m not aware of any workaround.

Thanks,
Ludo’.