From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id REYrHuGEYmBWbAAAgWs5BA (envelope-from ) for ; Tue, 30 Mar 2021 03:54:41 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id T+7aF+GEYmADQQAAbx9fmQ (envelope-from ) for ; Tue, 30 Mar 2021 01:54:41 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C47611086D for ; Tue, 30 Mar 2021 03:54:40 +0200 (CEST) Received: from localhost ([::1]:35456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lR3aY-0003GF-Bq for larch@yhetil.org; Mon, 29 Mar 2021 21:54:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57560) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lR3aS-0003G2-Mo for guix-science@gnu.org; Mon, 29 Mar 2021 21:54:32 -0400 Received: from elegua.eauchat.org ([91.224.149.118]:51250 helo=eauchat.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lR3aK-0007n8-JA for guix-science@gnu.org; Mon, 29 Mar 2021 21:54:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eauchat.org; s=mail; t=1617069258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qJIFiEZkJOkBEQoOafdq2qZMuXYgvrQcv5ge66v26Nc=; b=fMKpoJr3p0rEz47dJ7xFB5AZDL4K6lurQzbyKsYB5SFK8gyN6CSL/681RHjtQ8CXkbCYBQ 1RjzpmlDZqpjuBd35e2gZ1VuMl0kNlWO8IsYeYr4lHxJpxEFC9nZStNpTt3uobX3bQ3I+F u3zr+U2cFS9y2duMBuY846hY6C5R1VQ= References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> From: =?utf-8?Q?S=C3=A9bastien?= Lerique To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution In-reply-to: <87k0pqrwub.fsf@inria.fr> Date: Tue, 30 Mar 2021 10:54:01 +0900 Message-ID: <87blb1wgna.fsf@eauchat.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=91.224.149.118; envelope-from=sl@eauchat.org; helo=eauchat.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617069281; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=qJIFiEZkJOkBEQoOafdq2qZMuXYgvrQcv5ge66v26Nc=; b=fsmSGDcM3zhB3iPgkNpUMoHu+yvvksrYgD4PANGWYL9KR0wPv9+8AMccKdTLW1o1XDQKZL ZHs4SINMEXIpjkyc1SSqz4vbgIP6rxfxzlaoOk+gbBAOT2UWPNaRINdsZPAIhrO5mAX4+E ZuKH7TkTq9aMyYs1FSgkgq4d9mexrFmimC7YEjvlZtdDMWIbIwQQ3dkOxw5awqaWUZdcof /SIh3kXh7rrxUXO1UOE1M2RjgvMWX1CoyaBg97yknP6HbWE8VEhUugxFnqFn5DTDXzf5cy miU9U7v5/JW0JXWPEoHB+PGiSMN0Ul5MTkVD/pVaF1Oatb/nYhoh92WJ1jhtuQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617069281; a=rsa-sha256; cv=none; b=Ma8oQFXRVGMi699sR/qlE8248QVGp+35P6u0JXtMp/fCg4cO+CtQr2hD8FcMOb8883g2Y/ x+d3h22Pl0HTB/1/fvDYLKqBevscH/DHrY+KkaofvEHnTqFsqgzrLhjmgT5sMuLFTryjeM H8JgtwX0D3W7JfTwCFO5whrkTISurlKUpNEkxqTTdVpwLelUvhfjC5zTHjX39jmhiudHWM WaeoTATOb155jC2KYzmLEjT3rQFkKdDU5u0Ak6fJnVtIZZ5CwwB2OkudOtDWW5VSgpWqmV DeOwvxbZ9fRjrPaejd8xhhj4eGrfTUCNnc5vgU1ZP76AfWS8qm8+iqj6PtoZUg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b=fMKpoJr3; dmarc=fail reason="SPF not aligned (relaxed)" header.from=eauchat.org (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: 0.18 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b=fMKpoJr3; dmarc=fail reason="SPF not aligned (relaxed)" header.from=eauchat.org (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: C47611086D X-Spam-Score: 0.18 X-Migadu-Scanner: scn0.migadu.com X-TUID: mDhT7NSsAUx8 Hi Ludo, Thanks for the help! On 29 Mar 2021 at 21:03, Ludovic Court=C3=A8s=20 wrote: > Instead of installing the =E2=80=9Cregular=E2=80=9D binary tarball inside= a=20 > namespace, > it might be easier to create a tarball like so: > > guix pack -RR -S /bin=3Dbin -S /etc=3Detc guix bash > > =E2=80=A6 and to unpack the resulting tarball. > > From there, you can run ./bin/sh to get a shell that =E2=80=9Csees=E2=80= =9D=20 > /gnu/store. > You can then run: > > . ./etc/profile > > And then, you should be able to run the daemon, like so: > > export GUIX_STATE_DIRECTORY=3D$HOME/.local/var/guix > guix-daemon --disable-chroot & > > (Adapted from > .) > > Does that work for you? Thanks! I `guix pack`'ed with a single -R since I shouldn't need=20 PRoot, and had to add export GUIX_LOG_DIRECTORY=3D$HOME/.local/var/log to the environment setup in order to start guix-daemon. Then `guix=20 install hello` warned at the beginning with user with UID 7352 not found and later failed with Backtrace: In ice-9/boot-9.scm: 1736:10 6 (with-exception-handler _ _ #:unwind? _ # _) In unknown file: 5 (apply-smob/0 #) In ice-9/boot-9.scm: 718:2 4 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 3 (_ #(#(#))) In guix/ui.scm: 2164:12 2 (run-guix-command _ . _) In guix/scripts/offload.scm: 782:21 1 (guix-offload "x86_64-linux" "0" "1" "0") In unknown file: 0 (getpw 7352) ERROR: In procedure getpw: In procedure getpw: entry not found killing process 9106 guix install: error: unexpected EOF reading a line I'm guessing some part on the non-namespaced uids are leaking into=20 the user namespace? Another problem appeared for substitutes (as 'guix install hello'=20 was rebuilding the world): $ guix archive --authorize < \ /gnu/store/xb4szjambyi52bpnkjv080g2mlfqqpp0-profile/share/guix/ci.guix.= gnu.org.pub guix archive: error: mkdir: Permission denied I quickly went through `guix/scripts/pack.scm` and=20 `gnu/packages/aux-files/run-in-namespace.c` to try and understand=20 what `guix pack -R` does, but it looks like I need to learn more=20 about namespaces before I can digest that. My high-level=20 understanding from poking around is that the binaries from `guix=20 pack -R` run inside a namespace where /gnu is bind-mounted (along=20 with /proc, /dev, /sys, ...), but the rest of the host filesystem=20 is still available, and other host processes can also be seen.=20 Then, anything I run from inside the `guix pack`'ed bash inherits=20 that namespace, which is what makes guix-daemon work. If this is incorrect, could you maybe give a high-level overview=20 of what it does? The documentation is a bit scarce on the topic=20 (or I didn't find it). I'm also wondering why `guix-daemon` must be invoked with=20 `--disable-chroot`. Doesn't that make the reproducibility=20 guarantees more brittle (according to the docs)? >> - is it possible to create build users inside the=20 >> user-namespaced >> chroot? > > No: you still have a single UID at hand, so there=E2=80=99s no way to=20 > allocate > new ones. > >> - last but not least, how would I go about sharing this setup=20 >> with >> other users on the cluster? Ideally I would like to have a >> non-priviliged build daemon that other users can call on. (Is=20 >> there >> such a thing as kernel group namespaces?) > > It=E2=80=99s not really sharable. To share it, you would need some sort= =20 > of a > shared trusted =E2=80=9Cproxy=E2=80=9D; that=E2=80=99s precisely what gui= x-daemon is in=20 > normal > multi-user setups. Ok. My use cases here are: - trying to provide a setup that others can more-or-less easily=20 try out to get a feel for Guix on the HPC cluster (if possible=20 without duplicating /gnu/store), thus creating interest and user=20 support for later presenting the case to sysadmins / university=20 administration; - getting to know the exact requirements that I would have to=20 convince sysadmins to agree to, with rationales/explanations for=20 the hardest-to-obtain. Getting them to run guix-daemon as root=20 is probably a lost case, so I'd like to explore everything else=20 that is possible. So let's transform my two questions above into the two following: 1. what would be the list of requirements that explain why=20 guix-daemon must run as root? What does each of those requirements=20 accomplish? 2. knowing that my environment has user namespaces (and knowing=20 that that may not be enough), would it be possible to set up=20 guix-daemon as non-root with the assistance of a sysadmin (so say=20 with root access during setup), with build users and all, and have=20 it provide identical guarantees to a guix-daemon running as root?=20 If not, why not, and could we work around that? Thanks again for all the help! S=C3=A9bastien