Re: Introducing Guix to HPC at my institution

["Ludovic Courtès" <ludovic.courtes@inria.fr>]

Hi Sébastien,

Sébastien Lerique <sl@eauchat.org> skribis:

> On 29 Mar 2021 at 21:03, Ludovic Courtès <ludovic.courtes@inria.fr>
> wrote:
>
>> Instead of installing the “regular” binary tarball inside a
>> namespace,
>> it might be easier to create a tarball like so:
>>
>>   guix pack -RR -S /bin=bin -S /etc=etc guix bash
>>
>> … and to unpack the resulting tarball.

[...]

> Thanks! I `guix pack`'ed with a single -R since I shouldn't need
> PRoot, and had to add
>
>  export GUIX_LOG_DIRECTORY=$HOME/.local/var/log

OK.

> to the environment setup in order to start guix-daemon. Then `guix
> install hello` warned at the beginning with
>
>  user with UID 7352 not found
>
> and later failed with
>
>  Backtrace:
>  In ice-9/boot-9.scm:
>    1736:10  6 (with-exception-handler _ _ #:unwind? _ # _)
>  In unknown file:
>             5 (apply-smob/0 #<thunk 7f5a32f85520>)
>  In ice-9/boot-9.scm:
>      718:2  4 (call-with-prompt _ _ #<procedure
>      default-prompt-handle…>)
>  In ice-9/eval.scm:
>      619:8  3 (_ #(#(#<directory (guile-user) 7f5a32f88c80>)))
>  In guix/ui.scm:
>    2164:12  2 (run-guix-command _ . _)
>  In guix/scripts/offload.scm:
>     782:21  1 (guix-offload "x86_64-linux" "0" "1" "0")
>  In unknown file:
>             0 (getpw 7352)
>
>  ERROR: In procedure getpw:
>  In procedure getpw: entry not found

Hmm could it be that nscd is not running, and that /etc/nsswitch.conf
specifies a “non-standard” NSS plugin, such as ‘sssd’?

  https://guix.gnu.org/manual/en/html_node/Application-Setup.html#Name-Service-Switch-1

Does ‘guix install hello --no-offload’ work around the issue?

> Another problem appeared for substitutes (as 'guix install hello' was
> rebuilding the world):
>
>  $ guix archive --authorize < \
>    /gnu/store/xb4szjambyi52bpnkjv080g2mlfqqpp0-profile/share/guix/ci.guix.gnu.org.pub
>  guix archive: error: mkdir: Permission denied

I guess it’s trying to write to /etc/guix.  You probably need to set
GUIX_STATE_DIRECTORY=$HOME/.local/var/guix here.

> I quickly went through `guix/scripts/pack.scm` and
> `gnu/packages/aux-files/run-in-namespace.c` to try and understand 
> what `guix pack -R` does, but it looks like I need to learn more about
> namespaces before I can digest that. My high-level understanding from
> poking around is that the binaries from `guix pack -R` run inside a
> namespace where /gnu is bind-mounted (along with /proc, /dev, /sys,
> ...), but the rest of the host filesystem is still available, and
> other host processes can also be seen. Then, anything I run from
> inside the `guix pack`'ed bash inherits that namespace, which is what
> makes guix-daemon work.

Correct!

> If this is incorrect, could you maybe give a high-level overview of
> what it does? The documentation is a bit scarce on the topic (or I
> didn't find it).

The design and implementation are documented in blog posts:

  https://guix.gnu.org/en/blog/2018/tarballs-the-ultimate-container-image-format/
  https://hpc.guix.info/blog/2020/05/faster-relocatable-packs-with-fakechroot/

> I'm also wondering why `guix-daemon` must be invoked with
> `--disable-chroot`. Doesn't that make the reproducibility 
> guarantees more brittle (according to the docs)?

True, but I think (?) that’s currently unavoidable in this setup…
though I forget why.  You might want to try without ‘--disable-chroot’.

> Ok. My use cases here are:
> - trying to provide a setup that others can more-or-less easily   try
>  out to get a feel for Guix on the HPC cluster (if possible   without
> duplicating /gnu/store), thus creating interest and user   support for
> later presenting the case to sysadmins / university   administration;

Safe sharing among users is not possible without a daemon coordinating
accesses to the store and acting as a trusted proxy.

> - getting to know the exact requirements that I would have to
>   convince sysadmins to agree to, with rationales/explanations for 
>  the hardest-to-obtain. Getting them to run guix-daemon as root   is
> probably a lost case, so I'd like to explore everything else   that is
> possible.

Yeah.

> So let's transform my two questions above into the two following:
>
> 1. what would be the list of requirements that explain why guix-daemon
> must run as root? What does each of those requirements accomplish?

I think this article remains relevant:

  https://hpc.guix.info/blog/2017/09/reproducibility-and-root-privileges/

> 2. knowing that my environment has user namespaces (and knowing that
> that may not be enough), would it be possible to set up guix-daemon as
> non-root with the assistance of a sysadmin (so say with root access
> during setup), with build users and all, and have it provide identical
> guarantees to a guix-daemon running as root? If not, why not, and
> could we work around that?

Good question.  It should be possible to make the daemon run as
non-root; that’s what the trick with the ‘guix pack -R’ wrapper should
achieve, but it could also be a built-in capability.

Food for thought!

Thanks,
Ludo’.