From: "Sébastien Lerique" <sl@eauchat.org>
To: "Ludovic Courtès" <ludovic.courtes@inria.fr>
Cc: guix-science@gnu.org, zimoun <zimon.toutoune@gmail.com>
Subject: Re: Introducing Guix to HPC at my institution
Date: Wed, 31 Mar 2021 14:23:10 +0900 [thread overview]
Message-ID: <87ft0bvqv5.fsf@eauchat.org> (raw)
In-Reply-To: <87o8f1jee2.fsf@inria.fr>
Hi Ludo, zimoun, all,
Thanks again for your answers, for the links to documentation, and
for the support on IRC!
> Hmm could it be that nscd is not running, and that
> /etc/nsswitch.conf
> specifies a “non-standard” NSS plugin, such as ‘sssd’?
>
> https://guix.gnu.org/manual/en/html_node/Application-Setup.html#Name-Service-Switch-1
>
> Does ‘guix install hello --no-offload’ work around the issue?
Yes, that solves the issue!
>> $ guix archive --authorize < \
>> /gnu/store/xb4szjambyi52bpnkjv080g2mlfqqpp0-profile/share/guix/ci.guix.gnu.org.pub
>> guix archive: error: mkdir: Permission denied
>
> I guess it’s trying to write to /etc/guix. You probably need to
> set
> GUIX_STATE_DIRECTORY=$HOME/.local/var/guix here.
Yes, that would be GUIX_CONFIGURATION_DIRECTORY. As discussed on
IRC though, /etc in the tarball is a symlink to the profile, so
neither `guix archive` nor I can create /etc/guix. So I resorted
to creating a `/etc-guix` folder which I use for
GUIX_CONFIGURATION_DIRECTORY.
>> 2. knowing that my environment has user namespaces (and knowing
>> that
>> that may not be enough), would it be possible to set up
>> guix-daemon as
>> non-root with the assistance of a sysadmin (so say with root
>> access
>> during setup), with build users and all, and have it provide
>> identical
>> guarantees to a guix-daemon running as root? If not, why not,
>> and
>> could we work around that?
>
> Good question. It should be possible to make the daemon run as
> non-root; that’s what the trick with the ‘guix pack -R’ wrapper
> should
> achieve, but it could also be a built-in capability.
Yes, I'd be very happy to try and build that once things work!
So, to summarize the current status, here is what I do:
On my Guix System (`deigo` is my ssh config for the cluster):
$ scp (guix pack -R -S /bin=bin -S /etc=etc guix bash)
deigo:guix-pack.tar.gz
Then on deigo:
$ mkdir -p /a-working-directory/guix
$ cd /a-working-directory/guix
$ tar xzf ~/guix-pack.tar.gz
$ mkdir etc-guix
$ bin/bash
$ . etc/profile
$ export GUIX_STATE_DIRECTORY=$(pwd)/var/guix
$ export GUIX_LOG_DIRECTORY=$(pwd)/var/log
$ export GUIX_CONFIGURATION_DIRECTORY=$(pwd)/etc-guix
Then the fun starts. There are 4 different cases and problems,
depending on whether I use substitutes or not, and on whether
`/a-working-directory` is an NFS share or a local mount.
Case 1: with substitutes, on a local (non-NFS) folder
-----------------------------------------------------
$ guix archive --authorize <
bin/../share/guix/ci.guix.gnu.org.pub
$ guix-daemon --disable-chroot &
$ guix install hello
[... dowloading ...]
unexpected substituter message 'defining `GUIX_LOCPATH', along
these lines:
guix install glibc-utf8-locales
export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"
See the "Application Setup" section in the manual, for more
info.
'
I think this is https://issues.guix.gnu.org/45166.
Using instead `guix pack -R -S /bin=bin -S /etc=etc -S /lib=lib
guix bash glibc-utf8-locales`, and:
$ export GUIX_LOCPATH=$(pwd)/lib/locale
$ ls $GUIX_LOCPATH/
2.31
seems to start working, but then `guix install hello` fails with
the same error, and:
$ ls $GUIX_LOCPATH/
ls: cannot access '/tmp/sebastien-lerique/guix/lib/locale/': No
such file or directory
Is it possible that glibc-utf8-locales gets garbage-collected?
Case 2: from source (no substitutes), on a local (non-NFS) folder
-----------------------------------------------------------------
No need for the locale fix here (another bug appears before that):
$ guix-daemon --disable-chroot &
# --no-offload works around the missing nscd problem; up to now
this
# doesn't seem necessary when using substitutes
$ guix install hello --no-offload
[... builds for a while ...]
build of
/gnu/store/pkn1w1q3xkn273kpmggc4dnq6n6hr9jy-bzip2-mesboot-1.0.8.drv
failed
The build log for bzip2-mesboot ends with:
tcc -ar cq libbz2.a blocksort.o huffman.o crctable.o randtable.o
compress.o decompress.o bzlib.o
ranlib libbz2.a
/gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0/bin/sh:
ranlib: Command not found.
make: *** [libbz2.a] Error 127
command "make" "CC=tcc -I ." "AR=tcc -ar" "bzip2"
"PREFIX=/gnu/store/s94hyrv1vgllxir5niiyzfc9g80l5kcd-bzip2-mesboot-1.0.8"
failed with status 2
Is this new and should be reported as a bug? Should
binutils-mesboot0 be part of bzip2-mesboot's inputs? (I haven't
learned about the boostrapping mechanism yet.)
Case 3: with substitutes, on an NFS share
-----------------------------------------
Again no need for the locale fix (another bug appears before
that):
$ guix archive --authorize <
bin/../share/guix/ci.guix.gnu.org.pub
$ guix-daemon --disable-chroot &
$ guix install hello
[... downloading ...]
guix install: error: cannot unlink
`/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib':
Directory not empty
so checking:
$ ls -lA
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib
total 3840
-r-xr-xr-x 1 sebastien-lerique froeseuni 64832 Jan 1 1970
.nfs000000000172caa5000054bd
[... more .nfs files ...]
$ lsof -f --
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs000000000172caa5000054bd`
reports:
[... some warnings ...]
COMMAND PID USER FD TYPE DEVICE SIZE/OFF
NODE NAME
bash 25616 sebastien-lerique mem REG 0,52 64832
24300197
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs000000000172caa5000054bd
guix-daem 25934 sebastien-lerique mem REG 0,52 64832
24300197
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs000000000172caa5000054bd
Not sure what's going wrong here. Maybe it's more
garbage-collecting (as in case 1), failing because of an nfs
nuance or bug?
Case 4: from source (no substitutes), on an NFS share
-----------------------------------------------------
Fails in the exact same way as Case 2.
I have no idea how to make progress in any of these cases (except
the bzip2-mesboot one maybe), so any input on these different
problems is very welcome :)
Thanks and happy hacking!
Sébastien
next prev parent reply other threads:[~2021-03-31 5:23 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-15 3:12 Introducing Guix to HPC at my institution Sébastien Lerique
2021-03-15 13:47 ` zimoun
2021-03-16 1:54 ` Sébastien Lerique
2021-03-16 8:06 ` zimoun
2021-03-16 9:05 ` Ludovic Courtès
2021-03-18 2:26 ` Sébastien Lerique
2021-03-26 8:22 ` Sébastien Lerique
2021-03-29 12:03 ` Ludovic Courtès
2021-03-30 1:54 ` Sébastien Lerique
2021-03-30 7:21 ` Ludovic Courtès
2021-03-31 5:23 ` Sébastien Lerique [this message]
2021-04-01 8:35 ` Ludovic Courtès
2021-04-01 14:34 ` Sébastien Lerique
2021-04-10 20:43 ` Ludovic Courtès
2021-04-12 1:21 ` Sébastien Lerique
2021-04-12 12:43 ` Ludovic Courtès
-- strict thread matches above, loose matches on Subject: below --
2021-03-29 0:45 Sébastien Lerique
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ft0bvqv5.fsf@eauchat.org \
--to=sl@eauchat.org \
--cc=guix-science@gnu.org \
--cc=ludovic.courtes@inria.fr \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.