From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:306:f42::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id WCN1D+AfzGFRHAAAgWs5BA (envelope-from ) for ; Wed, 29 Dec 2021 09:44:16 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id EHF8DOAfzGGLoAAAauVa8A (envelope-from ) for ; Wed, 29 Dec 2021 09:44:16 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B7072EAE2 for ; Wed, 29 Dec 2021 09:44:15 +0100 (CET) Received: from localhost ([::1]:57088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n2UZC-0005Ig-R5 for larch@yhetil.org; Wed, 29 Dec 2021 03:44:14 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n2UYe-0005IV-Sk for guix-devel@gnu.org; Wed, 29 Dec 2021 03:43:40 -0500 Received: from [2a00:1450:4864:20::42e] (port=45841 helo=mail-wr1-x42e.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n2UYc-0006yi-5D for guix-devel@gnu.org; Wed, 29 Dec 2021 03:43:39 -0500 Received: by mail-wr1-x42e.google.com with SMTP id v7so42971642wrv.12 for ; Wed, 29 Dec 2021 00:43:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=K4d3NHclGtrh6l+MhpoDOsWIw6t8rwCfQxbbTdGnUhg=; b=LLE4ez2D9m3zZXeD2mBoSvU/vpfF+6KeiXzRxrrsSEgQkqSpR5De2sSq/rCtSoNvL1 ts7Xw50fNVruBjFkY08V25fu3m86NuA/841gVmDE6KiMNepM49OVWNgslLyAa6o3lukM Ny7JulnVuwzpJaVuvcZ59CqqBw5R96Itv5DkR/J9b6rXZrDf3iZRXj9IMgxEiZRAO+oT 7grY+SkwzEfhe3etn0B+K9utP8VA0lsk67TcqjaYofi2zxYUZlLercwmItZM0OPWjOvJ ERGzSsGJcpgYSgBEYcuMPKPDzz7cRk7L4kg0p4lLLYzEDkHcmy99ZVaTtvi80e30XOFK xQ7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=K4d3NHclGtrh6l+MhpoDOsWIw6t8rwCfQxbbTdGnUhg=; b=AaaJCjil3clvTIwGUyNfrwSfgmWZAdBQSU+avpIuuLKyzhAt8v1BySdmpkcWRapITn 4T/47bQCpTUQMIKS4dw0i7DspVjT1Oje54GYYRY++xFOcU43o6FY4tMBylPZggy5j8th HtAPqWH2BjeXjF8rLyVBmDaO3pphdemGRugiePWz6Ws4HbExTW07NP0b6YToEgOo53Lw fvhkzKL5kNb+JMPOs13vkkR5HE0cPMY1f44ccHhoFJOJEqCiqZhmLuhaNiHUJNN5sDx9 k4dRLPjDrIZB1bH1+rbSYRQkGM/OUYM/2p0sW28yrzODo8CVjtjmbcdA6esPPgNWXMK0 AnAQ== X-Gm-Message-State: AOAM533XknsSh0EyBRxDa+87yiajFAzDWmPd9nC5Kq1ytJu+HRg84ALx vhNwgiPI3HX4pLLdIEZ0DtE9QF2QpG4= X-Google-Smtp-Source: ABdhPJxWzrj/ju+8614DhJ0263Tf1DT6XQqr+bxK2InAYlcMOofc35xXA7zsgaaiaLlinLleKM2YuA== X-Received: by 2002:a5d:6c6c:: with SMTP id r12mr19943607wrz.532.1640767414732; Wed, 29 Dec 2021 00:43:34 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id o5sm2847227wmc.39.2021.12.29.00.43.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Dec 2021 00:43:34 -0800 (PST) From: zimoun To: Liliana Marie Prikler , guix-devel@gnu.org Subject: Re: On raw strings in commit field In-Reply-To: <6e451a878b749d4afb6eede9b476e5faabb0d609.camel@gmail.com> References: <6e451a878b749d4afb6eede9b476e5faabb0d609.camel@gmail.com> Date: Wed, 29 Dec 2021 09:39:19 +0100 Message-ID: <86y243kdoo.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::42e (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::42e; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x42e.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1640767455; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=K4d3NHclGtrh6l+MhpoDOsWIw6t8rwCfQxbbTdGnUhg=; b=QEw0uTwb8vLOHOpZY28VxaXlvSUMBRqTFzuQ4cZBjFhlvnO7JOn3p+ldo0+cPisIOUa8FA Qa/nCCkgMWTPmq/EeOYj4lW/EKcMuDBUIrQmLUBHL1RIxpQaOO9uUQHX78A6XuUTDijdI/ tjQTABp33jfrfv3z9KqD6W/QDt81n/SnIpR/jx8eeU8TXJyMG7ha970fdvq3zzXFMEDMku c6WYmo0Y5SG2FNVHXcqfbfmzf+6D5xaigMpsQib8y/jwwOg6b1Bey2Y/xnOoEl2o3X4LUK BsYOSwr8OgS94XfiYxULxazNP7wOt+CkGu2Nya72uh5/ZWUD2yL12fWYbQ0WTQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1640767455; a=rsa-sha256; cv=none; b=igIegI+PS7lHyYa3DPZGjMtdArcywpkMCYWh9Nrq2gt3G4A1glTi+NDULsmJX1sOdQshbI +Y+uAw/tHUDoYN/q854FG7D5SpdoRw8Irkmjrn2EMJqLGKH/YMohcD8NhPApU11DCsk+Uq RxXyLY63T+aUPSAB2x2+B7juexcJ4RADyEMS9EIY6YG8gzinOky7gR/5xIsXZN/S+GE5o9 kyT2dXQiHR6YuBgPwxgVJegZCHYBGFTP3IByWtEhj05Qoga1koo7524fCk7WeNVuNr5Ly7 /UOID9oMdz7hkGVQkr2DLHH+uJWHDgwQOFx0kKs5Ij1Mpn3EoIHnbqOmuj3jKQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LLE4ez2D; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.27 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LLE4ez2D; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: B7072EAE2 X-Spam-Score: -4.27 X-Migadu-Scanner: scn1.migadu.com X-TUID: N+HSzAXcBIcE Hi, On Tue, 28 Dec 2021 at 21:55, Liliana Marie Prikler wrote: > Consider a package being added or updated in Guix. At the time of > commit, we have the tag v1.2.3 pointing towards commit deadbeef. We > therefore create a guix package with version "1.2.3" pointing to said > commit (either directly or indirectly). At this point, one of the > following holds: > (1) Guix "1.2.3" -> upstream "v1.2.3" -> upstream "deadbeef" > (2) Guix "1.2.3" -> upstream "deadbeef" <- upstream "v1.2.3" > From either, we can follow that Guix "1.2.3" =3D upstream "v1.2.3". If > upstream keeps their tags around, then both forms are equivalent, but > (1) is more convenient; it allows us to derive commit from version, > which is often done through an affine mapping. No, tags and hash commit are not equivalent. Hash commit is intrinsic: it only depends on the content. Whereas, tags are extrinsic, they depend on external choice. >From the content to the hash, three keys: 1) how to serialize and 2) how to hash and 3) how to represent the hash. For #1, Git uses their own serializer and Guix, inheriting from Nix, uses another (Nar); although the difference is minor. For #2, Git uses by default SHA-1 as hash function, although Guix uses SHA-256. And for #3, Git uses hexadecimal format and Guix uses nix-base32. The subcommand =E2=80=9Cguix hash=E2=80=9D with the options =E2=80=99-S, -H= =E2=80=99 and =E2=80=99-f=E2=80=99 exposes these 3 keys. For instance: $ cat /tmp/foo.txt | git hash-object --stdin 557db03de997c86a4a028e1ebd3a1ceb225be238 $ ./pre-inst-env guix hash -S git -H sha1 -f hex /tmp/foo.txt 557db03de997c86a4a028e1ebd3a1ceb225be238 To make it explicit, the checksum hash of =E2=80=99git-reference=E2=80=99 c= ould be removed because it is somehow redundant with the commit hash. Obviously, it cannot because security reason (SHA-1 is considered as weak). > Problems arise, when upstreams move or delete tags. At this point, > guix packages that use them break and are no longer able to fetch their > source code. Raw commits are in principle resilient to this kind of > denial of service; instead upstreams would have to actually delete the > commits themselves, including also possible backups such as SWH to > break it. There is certainly an argument for robustness to be made > here, particularly concerning `guix time-machine', though as noted it > is not infallible. =C2=A0 SWH provides =E2=80=99swh:id=E2=80=99 which is another triplet (really clos= e to Git). Basically, content means data and metadata and to make it short, SWH deals their way with metadata for reason of large scale. And SWH does snapshots of Git repositories. Therefore, to have something really robust, Guix has to rely on a map from package definition to SWH. Using Git commit hash instead of tag makes this map. For tag, to have something robust, we need an external map from checksum hash to SWH hash via Git commit hash. This =E2=80=9Cexternal=E2=80=9D is done by Disarchive. > Long-term, we might want to support having multiple in > git-fetch -- if the first one fails due to a hash mismatch, we would > warn about that instead of producing an error and thereafter continue > with the second, third, etc. similar to how we currently have mirror:// > urls for some well-known mirrored repositories. That way, we have a > system to warn us about naughty upstreams while also providing > robustness for the time machine. I think the long term is to completely remove tag and only use commit hash; as done for =E2=80=99guile-aiscm=E2=80=99. But it will not happen fo= r convenience reasons, I guess. What you are proposing is to mix extrinsic (tag, URL, etc.) with intrinsic (commit hash, checksum hash, etc.). Well, I do not know if this proposed fallback mechanism would ease the maintenance and would make Guix more robust. To me, robustness means make a map from intrinsic values to content; as Disarchive is doing for instance. Cheers, simon