From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0IieK2aT0GFpmgAAgWs5BA (envelope-from ) for ; Sat, 01 Jan 2022 18:46:14 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 0IoDKWaT0GHzWAAA9RJhRA (envelope-from ) for ; Sat, 01 Jan 2022 18:46:14 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 62E8837B99 for ; Sat, 1 Jan 2022 18:46:14 +0100 (CET) Received: from localhost ([::1]:48698 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n3iSL-0005MX-Js for larch@yhetil.org; Sat, 01 Jan 2022 12:46:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:44744) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3iS6-0005M3-Tf for guix-devel@gnu.org; Sat, 01 Jan 2022 12:45:58 -0500 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:34721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n3iS4-0000NE-Hu for guix-devel@gnu.org; Sat, 01 Jan 2022 12:45:58 -0500 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 5EB3F5C0126; Sat, 1 Jan 2022 12:45:53 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Sat, 01 Jan 2022 12:45:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=JG0W1iSaN185KjqR6blouVj/L/b9Y1+TPmqSVG2NV 8k=; b=b40z71Npne1XaZDy6J9XyQ66y6cTssbJIlIXn8v6m6u06WAjkO5a8Jcdi e5TOqH33cXvQJDCSRVj2em2U8Sy6DExjjfbx8w/PB519yFo+2AH0NvjO5Cmyh2Bs YQeLdLoueTeyj73N7DWGLRwIzNkn4tuk0K1yVE6hAw173/ETouOyo+Rp225fxf8s 3MYe3GQGGkFRmYk6k1UKzKGlwCEYJ44m2Oy+yuwpaZAb1Emaf77mpEJx1zcyn3Ey y9trr+2feJVDotdinvYjnxW1IVkfj4SJDomuPvX5xl4Bmfpg5Td1xPrJvuAIwmlS Yl3inLl2RyzXpGbQpjEKZyTsPHqlw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddruddvjedguddthecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufhffjgfkfgggtgfgsehtqhertddtreejnecuhfhrohhmpefvihhm ohhthhihucfurghmphhlvgcuoehsrghmphhlvghtsehnghihrhhordgtohhmqeenucggtf frrghtthgvrhhnpedvleevhfehgeduveejvdeftdevieelffevheevjeeggfetteetjeet ledtfeeuveenucffohhmrghinhepghhnuhdrohhrghenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsrghmphhlvghtsehnghihrhhordgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 1 Jan 2022 12:45:52 -0500 (EST) From: Timothy Sample To: Liliana Marie Prikler Subject: Re: On raw strings in commit field References: <6e451a878b749d4afb6eede9b476e5faabb0d609.camel@gmail.com> <87k0fm7v3k.fsf@netris.org> <871r1smdu6.fsf@netris.org> Date: Sat, 01 Jan 2022 12:45:51 -0500 In-Reply-To: (Liliana Marie Prikler's message of "Sat, 01 Jan 2022 12:12:33 +0100") Message-ID: <874k6nqrhs.fsf@ngyro.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=66.111.4.26; envelope-from=samplet@ngyro.com; helo=out2-smtp.messagingengine.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1641059174; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=JG0W1iSaN185KjqR6blouVj/L/b9Y1+TPmqSVG2NV8k=; b=sD4ZrY2RYhglzFZtm8NLAPFOIHUcrzyxyemHuGtVZlWArHyd4HCBarCW1R2AhTq6pDdzhx q8YGiE9vK/0W9BAb+3zFKKOFawlM9ABiAoD5vyZntEMlygkcCuHBRU1swgmz7FAXalWtC6 4T1P/i79YO3FRfahJJqhIQ9Zu+GJ+Gj1TDAs+6etBPv/3WyzUe16MtgeyfK2JvO1eDmoxm JEW7xaOU+ms1xBinFQvSsyWNUosyJYFtuWqfm5xDsa40KjX0lB0FF5FHxC31rMVtxXzufw /GpemR++0AwXg59pPTC65M4ODKv3MhPfuKJItlSpZ45QERceH/juurq65gjNHg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1641059174; a=rsa-sha256; cv=none; b=d7uraB/Ze/y1DbQW8Nrdu+b7tzWjrVi3QXedH1QMi0nsu2PEnFnh3ukyw2lu4K8kjSMMRb lGxMVC8GoXPP867C9L9VAVQTF3k17C22z3gsp0yFAba1lg3N9ZaSLfeFtYTqgaIetqlppZ VkY5leNJyJmZDKA2AXVjDF50jvT/WiNDPrV29kM7YbuNwy/yjpXPfmOqRSDAR0v+tvpxmW cyo+cIm90MMzjLQYugxBRIzZRorzPY5k3o2dJthTV8jWYW3HjrO0ESqqyoxghpE8h7UFmw S86BOLIVTAseIZY3kTN3TFN08KQyZPEo9dwmwwhKRJCugXj8RDdFl66Be8+pew== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=b40z71Np; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.98 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=b40z71Np; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 62E8837B99 X-Spam-Score: -2.98 X-Migadu-Scanner: scn0.migadu.com X-TUID: RmHcx/gPcFTX Hi all, Liliana Marie Prikler writes: > Am Freitag, dem 31.12.2021 um 20:41 -0500 schrieb Mark H Weaver: > >> If upstream later indicates that version "1.2.3" is now commit YYZ, I >> don't think that invalidates our basis for continuing to associate >> version "1.2.3" with commit XYZ.=C2=A0 The aforementioned immutable >> historical fact still remains our basis and justification for making >> that association. > > I'm pretty sure it does, particularly to a future observer who may not > have the luxury of a history to distinguish that record from one in > which a malicious committer linked those versions and tag together and > then no one bothered to check. If you want a concrete example to think through, there=E2=80=99s =E2=80=98e= clib=E2=80=99. Our package says it=E2=80=99s version =E2=80=9C20190909=E2=80=9D, but that=E2= =80=99s not what upstream calls version =E2=80=9C20190909=E2=80=9D. It looks like when we packaged =E2=80= =98eclib=E2=80=99, that tag pointed to commit 19e7e3e74268bf78bd9a1c4ba07597d5434fb166, but now it points to bfbbd7c414521e1bf5e718a2925ea8ad845a2e87. If you try to build =E2=80=98eclib=E2=80=99, everything will work great, si= nce we can grab the checkout from our servers. If you use $ guix build --check -S eclib you get a hash mismatch. We have CI jobs for sources, but they aren=E2=80= =99t checking this: . That job succeeds after downloading the checkout from our servers. There are two things I can highlight from this case. First, as expected, finding the original commit was painful. SWH did not record the old version of the tag. Comparing it with the checkout from our servers showed that the differences were very minor. With that in mind, I moved backwards through the commit history with =E2=80=98guix ha= sh=E2=80=99 until I found a match. As pointed out many times, if I had the original commit, I could just ask SWH for it directly. Second, these cases are very, very rare. (I=E2=80=99ve essentially checked every Git origin since Guix version 1.0.0, and this problem is not one that worries me). =E2=80=9CTricking Peer Review=E2=80=9D-style problems se= em to be much more prevalent. When tracking down a =E2=80=9Cdifficult=E2=80=9D Git origi= n, the first thing I do is grep the Guix Git history for a =E2=80=9Coops I committed the wrong hash=E2=80=9D message. I recommend we focus our energies there before worrying too much about replacing tags with commits or using both or whatever. >> Regarding "Tricking Peer Review": I think it would be ideal for >> package definitions to include both the git tag _and_ the git commit >> hash, and to teach our linter to raise an alarm when the expected >> tags are missing or fail to match the expected commit hash. > > That is among the solutions I've proposed here, so naturally I'd be > fine with it. Given what I wrote above, maybe we could start by updating the linter so that =E2=80=98check-source=E2=80=99 actually checks that it gets the right = result. Right now it uses a few heuristics to check that the result looks okay (for instance, it checks if the result is suspiciously small). Maybe it should just go through the whole download process and verify the hash? Alternatively (or additionally), the CI =E2=80=9Csource=E2=80=9D specificat= ion could be configured to avoid using our servers as a fallback when checking sources. I agree that adding more identifiers (commit hashes or whatever) makes things more robust, but the cost is more work when creating, updating, and reviewing packages. I think we should start by verifying the identifiers we already have (i.e., checking that the URI and method of the origin produce the right output). It would solve many existing problems and would serve as a nice foundation for future improvements. And as a bonus, if you want to be really kind to future time travellers, when fixing an errant hash, please include a nice hint as to what the original hash was for (like a commit hash). We have commit ca5a791f6285b08506ccd662d5911ccf0c4d1ece in our repo, which says: > The previous hash was from the "dev" branch of the repository. I can=E2=80=99t find the source for the previous hash, and if I could actua= lly travel through time, I would change the commit message to: > The previous hash was from commit abcd0123..., which comes from the > "dev" branch of the repository. :) -- Tim