From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id yCkOMtN5OmbfSwEA62LTzQ:P1 (envelope-from ) for ; Tue, 07 May 2024 20:58:27 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id yCkOMtN5OmbfSwEA62LTzQ (envelope-from ) for ; Tue, 07 May 2024 20:58:27 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=PF9NzjKk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1715108307; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=VD3kmuxXzbmQFxh+R3iHMT92yYYceJZwrsu9ejGEDhI=; b=L9JD3AA2xt7ISN/3gfPA8DPH/9GsjhYFaRYO0UuaA8NHehwuFCLQdTWQyXuxbV/syW5z8v +71TvynBH9+rl0JzeWLvVwAL5dxgQuhTk/QcIyUQgRPCPKjNeLw2gJMjnWhti9uh2YvUSv KH8ILtQxO/W2nD8Dk0o+txNRctA++uOGgSKy8NRLv90kNCfhgRTzdZRfrex1yJ27eMIQRd xPug1ghUfHvlZz/aagM5A4JMSIF6QaRbjoCTJMDcST6N6+Ejzo2SvZcDwoy3qRcwTtkSxj Uqmu1wOlKiEjkxFK6aewPq+ZsU/6WtaDu01/M4gYJNNLqU11tUScssYpXfL10w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1715108307; a=rsa-sha256; cv=none; b=P5VLXlrxYeaPj/OXrWbt/wc5R2NsuIdmyfQRayw1/GK5bpunl2pz/thWaRJrl0TjjLhODk 4L5VHLbTRODXcLqbHBDIgbcTDMcdHxHAj7x9arM187ohKF83KByWqVkEDaAySOqN3ft0Eq 4NxrIr1jp2r740EZNJ4Nh3cgX03sA6sioOGhyEMUz33HnG0JqmibMBlN6DA4/ZTr3YGEvW 2kj0EKJ/KmTBIIySR8yg2vhKPQIBt9doVBRfedYslqco9Owqy7a9HDYUJgtnZS8dqdYCzT 9ccS9HObCqndLH1uqoNVTgj0mJsBOtMqmbGNdxAY4Ywm+dQlzIlsUNtx8895Pw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=PF9NzjKk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4544A67FBB for ; Tue, 07 May 2024 20:58:27 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s4Q0n-0006fl-09; Tue, 07 May 2024 14:58:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s4Q0Q-0006Zy-KS for guix-devel@gnu.org; Tue, 07 May 2024 14:57:41 -0400 Received: from mail-lj1-x22b.google.com ([2a00:1450:4864:20::22b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s4Q0O-0004VA-IA; Tue, 07 May 2024 14:57:38 -0400 Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-2e381f7c9c4so960451fa.3; Tue, 07 May 2024 11:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715108254; x=1715713054; darn=gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=VD3kmuxXzbmQFxh+R3iHMT92yYYceJZwrsu9ejGEDhI=; b=PF9NzjKkveR36vJ3XRZx8P+IwU0CU530SZQogpSMg5wBy+7tX1RX05Mrjo6Pakuebk oaBv9ad98mLlraSErcY4b1luVs2vmI26dpvq3KacldfFmzuVM53CqE34C4DYn2EoMKYp XsphJY6wkk14J3Q0gaknUERiN218ir+SkHENtJVxaok31/79HBDf2Ctsxc22dnkj5tK2 dH6H53VzQp3MXa0y7tvfepHJO8gbmj+OOfNUaiOTFRq6lQaFnBrqaifcgFT0qgNtzFW0 orvSW1f7nCiIj5KjxDVu+AS6umuiF0lo4NUzpqoTS7cqkzBamrbmbMi6CO/3UmmbrAmc xG2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715108254; x=1715713054; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VD3kmuxXzbmQFxh+R3iHMT92yYYceJZwrsu9ejGEDhI=; b=kIOTGv+hYJU9dQ+AO8bh+rop+eZbXLmUZLmfOtbWwdyyIVLGPcZQp316heayvSspuv FzKiM3HyG7U442SfDgn1RDdk1twqVJpTksmz5Ilmp9DUDbFRDuovpdSjy4rBqn572Gcx 43oC2Srl9I0m0GTiAh3qTwopV1ccoNNR0rzFduAQ566sggkgKwliBUJA0CYIg4naSMaQ oGB4parDkPkyunDYhUEMY7U9ZMXyLWHaiIy7n4ZFC4iv/JX/cVR7JEMRjBqfwTy1ESOy J1clEiHH7YfKeQ3rduMw2wrzpXTTCnoWdFqERNEE0siIzh8jDrFxq0v2QVbEfT2hllpk lacw== X-Forwarded-Encrypted: i=1; AJvYcCWe7zAY/sbyR7fz307OlSwz+u1LWcoJ9q+bQOFqHaQYbvEk1oL964Uk/FljlgdYFFHZAnTyCK+x+OkAJuZYi/tVZLU= X-Gm-Message-State: AOJu0YyMhAqqTdN9Z+4dvjay41CRgbBsczaPPwiT4qs9f7u8CpnMK8CR 4nfSx39nGADZIoZB4mb8Yyd+pUthmCyr1A1v7/rVJ1TucwiN7KWP95AgQw== X-Google-Smtp-Source: AGHT+IFaPQSCaFMyZq3UpSTXor9frnpohSFojKhm3gHXx8tSSrt27CmG41ZiJI88oImHamhdFIo8uw== X-Received: by 2002:a2e:a715:0:b0:2de:1457:9d22 with SMTP id 38308e7fff4ca-2e445f80bccmr2874541fa.0.1715108254166; Tue, 07 May 2024 11:57:34 -0700 (PDT) Received: from lili (roam-nat-fw-prg-194-254-61-40.net.univ-paris-diderot.fr. [194.254.61.40]) by smtp.gmail.com with ESMTPSA id iv16-20020a05600c549000b0041bff91ea43sm20475690wmb.37.2024.05.07.11.57.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 May 2024 11:57:33 -0700 (PDT) From: Simon Tournier To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Ekaitz Zarraga Cc: Attila Lendvai , Giovanni Biscuolo , Guix Devel Subject: 3 kinds of bootstrap (was Re: backdoor injection via release tarballs combined with binary artifacts) In-Reply-To: <87wmp5l3r3.fsf@gnu.org> References: <87ttkon4c4.fsf@protonmail.com> <8734s1mn5p.fsf@xelera.eu> <87zfu9ku4l.fsf@xelera.eu> <6e743725-26f0-669c-b088-e56c850110c8@elenq.tech> <87wmp5l3r3.fsf@gnu.org> Date: Tue, 07 May 2024 20:22:22 +0200 Message-ID: <87msp11nz5.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::22b; envelope-from=zimon.toutoune@gmail.com; helo=mail-lj1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -6.71 X-Spam-Score: -6.71 X-Migadu-Queue-Id: 4544A67FBB X-Migadu-Scanner: mx11.migadu.com X-TUID: DvfkuALvkyCB Hi, I am late to the party=E2=80=A6 On mer., 10 avril 2024 at 15:57, Ludovic Court=C3=A8s wrote: >> That has happened to me too. >> Why not use Git directly always? > > Because it create{s,d} a bootstrapping issue. The > =E2=80=9Cbuiltin:git-download=E2=80=9D method was added only recently to = guix-daemon and > cannot be assumed to be available yet: > > https://issues.guix.gnu.org/65866 [...] > I think we should gradually move to building everything from > source=E2=80=94i.e., fetching code from VCS and adding Autoconf & co. as = inputs. > > This has been suggested several times before. The difficulty, as you > point out, will lie in addressing bootstrapping issues with core > packages: glibc, GCC, Binutils, Coreutils, etc. I=E2=80=99m not sure how= to do > that but=E2=80=A6 [...] > =E2=80=A6 live-bootstrap can probably be a good source of inspiration to = find a > way to build those core packages (or some of them) straight from a VCS > checkout. IMHO, we need to distinguish because there is different types of issues and thus different potential workarounds. :-) 1. Bootstrap how to download source code. 2. Bootstrap how to build core packages. 3. Bootstrap the driver (say guix-daemon and helpers). Well, having solutions for #1 and #3 would naturally provide a solution for #2. Although the devil is about details. ;-) About #1 =3D=3D=3D=3D=3D=3D=3D=3D You cannot use the binary =E2=80=99git=E2=80=99 in order to download the so= urce code of Git to build the binary =E2=80=99git=E2=80=99. Yeah, circular dependency. = :-) Therefore, Git source code is pulled using another method, say from tarball, such method which also needs to be built from source, so it also needs yet another method. The usual chicken-or-the-egg problem. The current workaround is to =E2=80=9Chide=E2=80=9D the problem and introdu= ce a =E2=80=9Cbuiltin:download=E2=80=9D method: it=E2=80=99s an =E2=80=9Copaque= =E2=80=9D binary that is hard to inspect. Roughly, the workaround had been introduced by [1] on Oct. 2016. Almost 8 years ago, so it works! :-) The argument for accepting this =E2=80=9Copaque=E2=80=9D method is because = it is a fixed-output derivation. Other said, we know beforehand the SHA256 checksum. Thus the claim is: being =E2=80=9Copaque=E2=80=9D does not matte= r because the SH256 checksum can be computed independently and all the source code can be audited. For cutting another cycle, another =E2=80=9Copaque=E2=80=9D had be introduc= ed: =E2=80=9Cbuiltin:git-download=E2=80=9D. All applies similarly. Do not take me wrong with =E2=80=9Copaque=E2=80=9D. I mean that the method= depends on the couple user-revision and daemon-revision. Other said, it is not straightforward to know when Alice and Bob are using the exact same method for downloading source code. Since it is not fully transparent, it is =E2=80=9Copaque=E2=80=9D. :-) Somehow we are applying to all what we need for cutting a specific circular dependency. We have some packages named =E2=80=99foo-bootstrap=E2= =80=99 that are aimed to solve some dependency problem about packages, then we do not use them for all; we just use them for cutting a circular dependency. I think a similar strategy should be applied for the fetch methods. We could have =E2=80=9Cgit-fetch=E2=80=9D relying on the initial Git method= , i.e., a transparent derivation where it=E2=80=99s straightforward to audit all: the dependencies and the builder. And for some specific cases, we could have =E2=80=9Cgit-fetch/bootstrap=E2= =80=9D relying on =E2=80=9Cbuiltin:git-download=E2=80=9D. It eases to know which packages= are very important to care. I think that =E2=80=9Cbuiltin:download=E2=80=9D and =E2=80=9Cbuiltin:git-do= wnload=E2=80=9D applied to all =E2=80=9Curl-fetch=E2=80=9D and =E2=80=9Cgit-fetch=E2=80=9D both downgr= ade the complete transparency level for solving very specific bootstrapping problem. Last about #1, please note that the transparency does not come for free and has drawbacks: when running say =E2=80=9Cguix time-machine -C past.scm = -- build -S=E2=80=9D, all the dependencies for downloading would be the ones of past.scm. Other said, for downloading today the source code of a 5 years old package, say using =E2=80=99hg-fetch=E2=80=99, we need Python and= Mercurial as they were 5 years ago =E2=80=93 when we do not expect any difference on the content with the Python and Mercurial of today. About #3 =3D=3D=3D=3D=3D=3D=3D=3D That=E2=80=99s the very hard topic! The bootstrapping story is not fully d= one yet. Assuming trust for #1, the bootstrap of Guix starts with =E2=80=99bootstrap-seeds=E2=80=99, roughly 232KiB. Take a moment, that=E2= =80=99s impressive, :-) right? Obviously, I let aside Haskell, Ocaml@5 etc. Well, diving further. These 232K alone are not enough. It also requires helpers: tar (1.3MiB), bash (1.3MiB), mkdir (0.7MiB) and xz (0.844MiB). More, it requires two drivers: static Guile binary (14MiB) and guix-daemon. You get it: How to trust these helpers? Two approaches: (a) implement something directly in hex/assembler and/or (b) exploit the Guile binary (=C3=A0 la Scheme on bare metal). About guix-daemon, one solution is a daemon directly in Guile, and compatible with the very Guile binary. Or at least, a minimalist daemon with just enough features for building up to guix-daemon. Or another option is the =E2=80=9CExtreme bootstrapping=E2=80=9D [3] =E2=80= =93 my understanding of live-bootstrap. Somehow, remove guix-daemon from the picture and convert the derivation =E2=80=93 the one read by guix-daemon =E2=80=93 to a= minimal Guile script that would be executed during startup. See the proof-of-concept in the branch wip-system-bootstrap [4]. Just my lengthy opinion=E2=80=A6 Or maybe some ideas for GSoC. ;-) 1: https://issues.guix.gnu.org/22774#3 2: https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-fro= m-source-all-the-way-down 3: https://guix.gnu.org/en/blog/2019/reproducible-builds-summit-5th-edition 4: https://git.savannah.gnu.org/cgit/guix.git/log/?h=3Dwip-system-bootstrap Cheers, simon