From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id GJkqBta11WHkQAAAgWs5BA (envelope-from ) for ; Wed, 05 Jan 2022 16:14:30 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id cAFyOtW11WELLgEAG6o9tA (envelope-from ) for ; Wed, 05 Jan 2022 16:14:29 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7EAA928728 for ; Wed, 5 Jan 2022 16:14:29 +0100 (CET) Received: from localhost ([::1]:47980 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n57zg-0004zQ-NK for larch@yhetil.org; Wed, 05 Jan 2022 10:14:28 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54490) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n56xT-0007JY-OS for guix-patches@gnu.org; Wed, 05 Jan 2022 09:08:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:56470) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n56xO-0006bc-8E for guix-patches@gnu.org; Wed, 05 Jan 2022 09:08:07 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n56xO-0004WS-0T for guix-patches@gnu.org; Wed, 05 Jan 2022 09:08:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#50072] [PATCH WIP 0/4] Add upstream updater for git-fetch origins. Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 05 Jan 2022 14:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50072 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: zimoun Cc: Sarah Morgensen , 50072@debbugs.gnu.org Received: via spool by 50072-submit@debbugs.gnu.org id=B50072.164139164317285 (code B ref 50072); Wed, 05 Jan 2022 14:08:01 +0000 Received: (at 50072) by debbugs.gnu.org; 5 Jan 2022 14:07:23 +0000 Received: from localhost ([127.0.0.1]:39767 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n56wl-0004Uj-1T for submit@debbugs.gnu.org; Wed, 05 Jan 2022 09:07:23 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:58248) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n56wh-0004UX-T9 for 50072@debbugs.gnu.org; Wed, 05 Jan 2022 09:07:21 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by michel.telenet-ops.be with bizsmtp id f27H2600W4UW6Th0627JEg; Wed, 05 Jan 2022 15:07:18 +0100 Message-ID: <6fcbe52781b0678ea44db9beea2e1bd3f404b840.camel@telenet.be> From: Maxime Devos Date: Wed, 05 Jan 2022 14:06:55 +0000 In-Reply-To: <86pmp6730h.fsf@gmail.com> References: <20220104200643.43374-1-maximedevos@telenet.be> <20220104200643.43374-2-maximedevos@telenet.be> <867dbfcf9n.fsf_-_@gmail.com> <52e7be94d926aa06c2a0132090e8c212381e7900.camel@telenet.be> <86y23u768v.fsf@gmail.com> <86pmp6730h.fsf@gmail.com> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-RpYAu8/sIr7JlwsA4/89" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1641391638; bh=sv6wMGEusyWqG53WSwa/DHIo9A3MASNPpl4bVkKgsgY=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=GpiNml6A9ptSR6dsvOnnarPBuqdW10GSK+LrBU0R9gZQJWms2d6p+bz0WaYVIVNUe FMX5XQJJuh1XesScXo0PZ7mZIzkbX+6O/11XbueRFoL9GyRLSrTYDoixQmDzBkNdeS OPLnIUlWBaFMvklyZU3fXIzfRGPIypbpSeaVnAfP4hs12ykhbddHVYOUZ5yIp5U4eo XYeev8EH+dBiioBStpitA+BXHgQ1ZrQrme/3cGtibO6DppQGFcbkStEb2ze89maLOq m42nLl4/csCmCGq+6LWfoIBOzk+Z2jHpakYF4oYQQdGLfhFjEzCROsDc1oc1NMxS6+ +xp+2QoRbg0OQ== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641395669; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=sv6wMGEusyWqG53WSwa/DHIo9A3MASNPpl4bVkKgsgY=; b=rtpN7/D/WMvyfZ7bimC0TFNrKGA77XoaWHvTAeM6Om/aqniJLh8esrUdnag1nup2qVQqOb b1pgGJ+wJ8WAkw8t2s/AgA3k/24tWteM5vD8vj4Ae7WI1MFP2zV7hpa2qmTEE/Svb00jTX GTLz7KEZq6p4EyJ9qQrvvWVtJMvR5Pd5ocV030vw1lNfarD3nHuNzD45jG7hEdC492GGJM alnZef+Ebg/Q8zRrfUdR4FOyZyWpchZl/lr9sCpFPSwHW9w0VAW8mW1bcMEmm4NrTVSwJH geS+vomi+gB4hEQDaVfBf/FkBykia0LOow+vVGe17JMOTnk3wpPA4qj5KkHPRw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641395669; a=rsa-sha256; cv=none; b=tTGfIZtD8LdB3Vks3qzX9Kw30wv8PWSife8dH/xYdtTzOfBkhfBHrHWvETWjkV6CG4Tbs9 fTz7pNAufDFgdALiYokEH+p2Co0hEFplFvZJVbg6lKrBVo/FDXBx3hlxfKRd79DYqgJeaS iha+vJm+UvMTgFC9XUuJW58iDDOIkK/rd1s/CN6xoRPyqJ7cJwwBgjr7LXbRZGflfy5Zqr UQgw3dM3VjqMvv77a+szlh6ddDOnkh+cbL8Y6LgQQpLiZ7InI3bno5FZJpvJxiUqEl1mq4 J+woP4CHFYRKQzreVBBnLB9Tu1OdP40OP/Q0MZ/8Ra+uobEJ3ouQvOT/y+3aZA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=GpiNml6A; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.20 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=telenet.be header.s=r22 header.b=GpiNml6A; dmarc=fail reason="SPF not aligned (relaxed)" header.from=telenet.be (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 7EAA928728 X-Spam-Score: -4.20 X-Migadu-Scanner: scn1.migadu.com X-TUID: fldq34iwURcu --=-RpYAu8/sIr7JlwsA4/89 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable zimoun schreef op wo 05-01-2022 om 13:58 [+0100]: > [...] > > > --8<---------------cut here---------------start------------->8--- > > > =C2=A0 "Compute the hash of FILE with ALGORITHM.=C2=A0 If NAR-SERIALI= ZER? is > > > =C2=A0 #true, compute the combined hash (NAR hash) of FILE for which = (SELECT? > > > =C2=A0 FILE STAT) returns true. > > >=20 > > > =C2=A0 If NAR-SERIALIZER? is #false, compute the regular hash using t= he > > > =C2=A0 default serializer.=C2=A0 It is meant to be used for a regular= file. > > >=20 > > > =C2=A0 If NAR-SERIALIZER? is 'auto', when FILE is a directory, comput= e the > > > =C2=A0 combined hash (NAR hash).=C2=A0 When FILE is a regular file, c= ompute the > > > =C2=A0 regular hash using the default serializer.=C2=A0 The option = =E2=80=99auto=E2=80=99 is meant > > > =C2=A0 to apply by default the expected hash computation. > > >=20 > > > =C2=A0 Symbolic links are not dereferenced unless NAR-SERIALIZER? is = false. > > >=20 > > > =C2=A0 This procedure must only be used under controlled circumstance= s; the > > > =C2=A0 detection of symbolic links in FILE is racy. > > > --8<---------------cut here---------------end--------------->8--- >=20 > > The nar hash / regular hash difference seems a very low-level detail to > > me, that most (all?) users don't need to be bothered about. Except > > maybe if FILE denotes an executable regular file, but file-hash* is > > currently only used on tarballs/zip files/git checkouts, which aren't > > executable files unless weirdness or some kind of attack is happening. > >=20 > > I think that, the =E2=80=98least astonishing=E2=80=99 thing to do here,= is computing > > the hash that would go into the 'hash' / 'sha256' field of 'origin' > > objects by default, and not the nar hash for regular files that's > > almost never used. >=20 > I do not understand what you mean here. =E2=80=99file-hash*=E2=80=99 is = a low-level > detail, no? Whatever. :-) I don't see what it matters if 'file-hash*' is classified as low-level or high-level. =C2=A0But what I do care about, is how easy to use file-hash= * is. A low-level argument like #:nar-hash? #true/#false would make file- hash* much more complicated: this patch series uses file-hash* to compute the hash for 'origin' records, and the documentation of 'origin' doesn't mention 'nar' anywhere and if I search for 'nar hash' in the manual, I find zero results. Instead, file-hash* talks about directories, regular files, recursion and claims that the default value of #:recursive? usually does the right thing, so I don't have to look up any complicated terminology to figure out how to use file-hash* to compute hashes for 'origin' records. And in the rare situation where file-hash* doesn't do the right thing, the documentation tells me I can set #:recursive? #true/#false. =20 > Just, to be sure, I am proposing: >=20 > =C2=A01) It is v4 and ready, I guess. About =E2=80=99auto=E2=80=99, I co= uld have waken up > =C2=A0earlier. :-) And it can be still improved later as you are saying i= n > =C2=A0the other answer. So, we are done, right? I think so, yes, except for a docstring change I'll send as a v5. I'm also out of bikeshed paint. Anway, keep in mind that I'm not a committer. > =C2=A02) From my point of view, =E2=80=99#:recursive?=E2=80=99 needs to b= e adapted in > =C2=A0agreement with the discussion [1], quoting Ludo: >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Thinking more about it, I= think confusion stems from the term > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=E2=80=9Crecursive=E2=80= =9D (inherited from Nix) because, as you write, it > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0doesn=E2=80=99t necessari= ly have to do with recursion and directory > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0traversal. >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Instead, it has to do wit= h the serialization method. >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A01: >=20 > =C2=A0=C2=A0=C2=A0And I do not have a strong opinion. Just a naive remar= k. I don't think the arguments for (guix scripts hash) apply directly to (guix hash) -- (guix scripts hash) supports multiple serialisers: * none (regular in (guix hash) terminology) * git * nar * swh so something like -S nar makes a lot of sense there. But (guix hash) is only for computing the hash of something that would become a store item after interning, more specifically is is currently only used for computing the hash that would go into an (origin ...) object (though I suppose it could be extended to support git/swh/... if someone wants do that). Possibly some name like #:treat-it-as-a-directory-or-an-executable-file-or-a-symlink-and- compute-the-alternative-hash-even-if-it-is-regular? would be clearer and technically more accurate than #:recursive?, but that's a bit of a mouthful. > =C2=A03) Whatever the keyword for the current v4 =E2=80=99#:recursive?=E2= =80=99 is picked, I > =C2=A0=C2=A0still find the current docstring wording unclear. In fact, r= eading > =C2=A0=C2=A0the code is more helpful. :-) I am just proposing a reword wh= ich > =C2=A0=C2=A0appears to me clearer than the current v4 one. Maybe, I am m= issing > =C2=A0=C2=A0the obvious. Or maybe this proposed rewording is not clearer= . :-) I've reworded it a bit; it falsely claimed that the nar hash was always computed when recursive? is 'auto' (even if FILE is a regular file). It also mentions executable files and SELECT? now. Greetings, Maxime. --=-RpYAu8/sIr7JlwsA4/89 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYdWl/xccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7kV8AP4wlEra0f9+TjAh5gilOW0+dpCi A7JDyk0HGyq5KBHYqQD/YnmNTRMHePpYKxXaIUV/Z9WFmVuSkLO1RbgnAl7fcQk= =JDQE -----END PGP SIGNATURE----- --=-RpYAu8/sIr7JlwsA4/89--