From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wJ6vI2gHZGDwSAAAgWs5BA (envelope-from ) for ; Wed, 31 Mar 2021 07:23:52 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id qAFcHWgHZGCSUQAAbx9fmQ (envelope-from ) for ; Wed, 31 Mar 2021 05:23:52 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D0A5610E9D for ; Wed, 31 Mar 2021 07:23:51 +0200 (CEST) Received: from localhost ([::1]:43168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRTKY-00006c-Gy for larch@yhetil.org; Wed, 31 Mar 2021 01:23:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRTKS-0008WK-Oy for guix-science@gnu.org; Wed, 31 Mar 2021 01:23:44 -0400 Received: from elegua.eauchat.org ([91.224.149.118]:49802 helo=eauchat.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lRTKN-0005WC-98 for guix-science@gnu.org; Wed, 31 Mar 2021 01:23:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eauchat.org; s=mail; t=1617168210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oopdOyw5BWu2fL7+LMMGVgKS0zyE62ww7eYFbpKmgFw=; b=T8k4chvvsOFPo7bz/JLR6HGT7tpkswzB1z8qyYIc5JLftjDfy0Sz4QngFluLowSJ4nE6Bc PtbA5k2gag4Upu4PS5mSDBDC51lmKPjLxZdtOZX1/0Z91MXLo81xJ5FgbUg/4Nh5q7YIcX h4EErK1c3AZGH9hHovi8+I8A1YDidKk= References: <878s6pds9t.fsf@eauchat.org> <87zgz3c17o.fsf@eauchat.org> <8735wva2p9.fsf@gnu.org> <87r1kdci49.fsf@eauchat.org> <877dlucsur.fsf@eauchat.org> <87k0pqrwub.fsf@inria.fr> <87blb1wgna.fsf@eauchat.org> <87o8f1jee2.fsf@inria.fr> From: =?utf-8?Q?S=C3=A9bastien?= Lerique To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org, zimoun Subject: Re: Introducing Guix to HPC at my institution In-reply-to: <87o8f1jee2.fsf@inria.fr> Date: Wed, 31 Mar 2021 14:23:10 +0900 Message-ID: <87ft0bvqv5.fsf@eauchat.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=91.224.149.118; envelope-from=sl@eauchat.org; helo=eauchat.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617168231; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=oopdOyw5BWu2fL7+LMMGVgKS0zyE62ww7eYFbpKmgFw=; b=CQT8CvmJy7QTn2cmx4Q6R8NrD5+5FeV2qBW8js56LKaJy53NRslFE5U7YCrbJ2dD76Xe0w 1lpbqVhNsweWQGlc9PWGfSFlF8qIFcPhxvlHcJRJiah3rTUhHptpZDiO64mm5Qrz5VSVRF DRBlz5ax65+VyNKvkH4LeJHIP80e7QPMGmPh1B3LpAmj8kIegkrrP0TjgL1Q484dUwAaVL US4BMTrj23teHWyIR+K/ZwbuRiLNj7DkDggiqMKKPVcXCD3em+pZKiZkRMI3SejtIV2pd6 BReQLsLvxVzkJuuEVrMJFxFEQ8Mf9vceTk9j6kwUildyHU1yClWb5vAUjrO+9g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617168231; a=rsa-sha256; cv=none; b=feeFXMffCJ7/SE0F1jNHnKNfyG0MA3Vo7vBUg24ZMLJpUCqxuBNvp+hC50+lZuu0XaMV1e IUl0yCz0Jr55Jtea9ngL74VNxvxwD06acNtJ3T81E/DuyhI7Rz4e7Z2tKGC+f9LkY+XlCW vn4YOKoRJWMTjWHN6hmGu6brnOEY6r7zMp7JT5cjHp1zwvGcvEiDoHSFUxSlDD3DnFL8oq Ky4a4WleioaEKkanRXGd4M+yJFRtW1lwYpdoJfPo1hSrOs/z5WkzNeZEphP9mx8hyBo6Ao COvjwnrfCjS6RvpWmMb1jKfDIyhb9VnI8R9kxBQsEbopgiZCBasolQHDGjSDmQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b=T8k4chvv; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: -1.32 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=eauchat.org header.s=mail header.b=T8k4chvv; dmarc=fail reason="SPF not aligned (relaxed)" header.from=eauchat.org (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: D0A5610E9D X-Spam-Score: -1.32 X-Migadu-Scanner: scn0.migadu.com X-TUID: i1eLEF3RBo9t Hi Ludo, zimoun, all, Thanks again for your answers, for the links to documentation, and=20 for the support on IRC! > Hmm could it be that nscd is not running, and that=20 > /etc/nsswitch.conf > specifies a =E2=80=9Cnon-standard=E2=80=9D NSS plugin, such as =E2=80=98s= ssd=E2=80=99? > > https://guix.gnu.org/manual/en/html_node/Application-Setup.html#Name-Se= rvice-Switch-1 > > Does =E2=80=98guix install hello --no-offload=E2=80=99 work around the is= sue? Yes, that solves the issue! >> $ guix archive --authorize < \ >> /gnu/store/xb4szjambyi52bpnkjv080g2mlfqqpp0-profile/share/guix/ci.gui= x.gnu.org.pub >> guix archive: error: mkdir: Permission denied > > I guess it=E2=80=99s trying to write to /etc/guix. You probably need to= =20 > set > GUIX_STATE_DIRECTORY=3D$HOME/.local/var/guix here. Yes, that would be GUIX_CONFIGURATION_DIRECTORY. As discussed on=20 IRC though, /etc in the tarball is a symlink to the profile, so=20 neither `guix archive` nor I can create /etc/guix. So I resorted=20 to creating a `/etc-guix` folder which I use for=20 GUIX_CONFIGURATION_DIRECTORY. >> 2. knowing that my environment has user namespaces (and knowing=20 >> that >> that may not be enough), would it be possible to set up=20 >> guix-daemon as >> non-root with the assistance of a sysadmin (so say with root=20 >> access >> during setup), with build users and all, and have it provide=20 >> identical >> guarantees to a guix-daemon running as root? If not, why not,=20 >> and >> could we work around that? > > Good question. It should be possible to make the daemon run as > non-root; that=E2=80=99s what the trick with the =E2=80=98guix pack -R=E2= =80=99 wrapper=20 > should > achieve, but it could also be a built-in capability. Yes, I'd be very happy to try and build that once things work! So, to summarize the current status, here is what I do: On my Guix System (`deigo` is my ssh config for the cluster): $ scp (guix pack -R -S /bin=3Dbin -S /etc=3Detc guix bash)=20 deigo:guix-pack.tar.gz Then on deigo: $ mkdir -p /a-working-directory/guix $ cd /a-working-directory/guix $ tar xzf ~/guix-pack.tar.gz $ mkdir etc-guix $ bin/bash $ . etc/profile $ export GUIX_STATE_DIRECTORY=3D$(pwd)/var/guix $ export GUIX_LOG_DIRECTORY=3D$(pwd)/var/log $ export GUIX_CONFIGURATION_DIRECTORY=3D$(pwd)/etc-guix Then the fun starts. There are 4 different cases and problems,=20 depending on whether I use substitutes or not, and on whether=20 `/a-working-directory` is an NFS share or a local mount. Case 1: with substitutes, on a local (non-NFS) folder ----------------------------------------------------- $ guix archive --authorize <=20 bin/../share/guix/ci.guix.gnu.org.pub $ guix-daemon --disable-chroot & $ guix install hello [... dowloading ...] unexpected substituter message 'defining `GUIX_LOCPATH', along=20 these lines: guix install glibc-utf8-locales export GUIX_LOCPATH=3D"$HOME/.guix-profile/lib/locale" See the "Application Setup" section in the manual, for more=20 info. ' I think this is https://issues.guix.gnu.org/45166. Using instead `guix pack -R -S /bin=3Dbin -S /etc=3Detc -S /lib=3Dlib=20 guix bash glibc-utf8-locales`, and: $ export GUIX_LOCPATH=3D$(pwd)/lib/locale $ ls $GUIX_LOCPATH/ 2.31 seems to start working, but then `guix install hello` fails with=20 the same error, and: $ ls $GUIX_LOCPATH/ ls: cannot access '/tmp/sebastien-lerique/guix/lib/locale/': No=20 such file or directory Is it possible that glibc-utf8-locales gets garbage-collected? Case 2: from source (no substitutes), on a local (non-NFS) folder ----------------------------------------------------------------- No need for the locale fix here (another bug appears before that): $ guix-daemon --disable-chroot & # --no-offload works around the missing nscd problem; up to now=20 this # doesn't seem necessary when using substitutes $ guix install hello --no-offload [... builds for a while ...] build of=20 /gnu/store/pkn1w1q3xkn273kpmggc4dnq6n6hr9jy-bzip2-mesboot-1.0.8.drv=20 failed The build log for bzip2-mesboot ends with: tcc -ar cq libbz2.a blocksort.o huffman.o crctable.o randtable.o=20 compress.o decompress.o bzlib.o ranlib libbz2.a /gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0/bin/sh:=20 ranlib: Command not found. make: *** [libbz2.a] Error 127 command "make" "CC=3Dtcc -I ." "AR=3Dtcc -ar" "bzip2"=20 "PREFIX=3D/gnu/store/s94hyrv1vgllxir5niiyzfc9g80l5kcd-bzip2-mesboot-1.0.8= "=20 failed with status 2 Is this new and should be reported as a bug? Should=20 binutils-mesboot0 be part of bzip2-mesboot's inputs? (I haven't=20 learned about the boostrapping mechanism yet.) Case 3: with substitutes, on an NFS share ----------------------------------------- Again no need for the locale fix (another bug appears before=20 that): $ guix archive --authorize <=20 bin/../share/guix/ci.guix.gnu.org.pub $ guix-daemon --disable-chroot & $ guix install hello [... downloading ...] guix install: error: cannot unlink=20 `/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib':=20 Directory not empty so checking: $ ls -lA=20 /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib total 3840 -r-xr-xr-x 1 sebastien-lerique froeseuni 64832 Jan 1 1970=20 .nfs000000000172caa5000054bd [... more .nfs files ...] $ lsof -f --=20 /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs0000000001= 72caa5000054bd`=20 reports: [... some warnings ...] COMMAND PID USER FD TYPE DEVICE SIZE/OFF=20 NODE NAME bash 25616 sebastien-lerique mem REG 0,52 64832=20 24300197=20 /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs0000000001= 72caa5000054bd guix-daem 25934 sebastien-lerique mem REG 0,52 64832=20 24300197=20 /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/.nfs0000000001= 72caa5000054bd Not sure what's going wrong here. Maybe it's more=20 garbage-collecting (as in case 1), failing because of an nfs=20 nuance or bug? Case 4: from source (no substitutes), on an NFS share ----------------------------------------------------- Fails in the exact same way as Case 2. I have no idea how to make progress in any of these cases (except=20 the bzip2-mesboot one maybe), so any input on these different=20 problems is very welcome :) Thanks and happy hacking! S=C3=A9bastien