From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id mHPzCv1BgV+eMwAA0tVLHw (envelope-from ) for ; Sat, 10 Oct 2020 05:09:17 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id mK2uBv1BgV/YfgAAB5/wlQ (envelope-from ) for ; Sat, 10 Oct 2020 05:09:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7E3089400AD for ; Sat, 10 Oct 2020 05:09:16 +0000 (UTC) Received: from localhost ([::1]:47820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kR786-0004sg-4T for larch@yhetil.org; Sat, 10 Oct 2020 01:09:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kR77u-0004sW-DU for bug-guix@gnu.org; Sat, 10 Oct 2020 01:09:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53463) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kR77u-0001GC-4d for bug-guix@gnu.org; Sat, 10 Oct 2020 01:09:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kR77u-0006Nm-00 for bug-guix@gnu.org; Sat, 10 Oct 2020 01:09:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43893: make update-guix-package produced an incorrect hash Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 10 Oct 2020 05:09:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43893 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Danny Milosavljevic Received: via spool by 43893-submit@debbugs.gnu.org id=B43893.160230652124508 (code B ref 43893); Sat, 10 Oct 2020 05:09:01 +0000 Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 05:08:41 +0000 Received: from localhost ([127.0.0.1]:36776 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kR77Y-0006ND-RM for submit@debbugs.gnu.org; Sat, 10 Oct 2020 01:08:41 -0400 Received: from mail-qt1-f178.google.com ([209.85.160.178]:42611) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kR77T-0006Mq-Se for 43893@debbugs.gnu.org; Sat, 10 Oct 2020 01:08:39 -0400 Received: by mail-qt1-f178.google.com with SMTP id t9so8964304qtp.9 for <43893@debbugs.gnu.org>; Fri, 09 Oct 2020 22:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=2rTk1Dfr7uSVsrTgPRhLiG3TO5OF4Se7SklVB6/EgrE=; b=B7PZVWGMpS9LGiX6OKcmR1mcYItmhw61yCoikc/iJcTmN4ldZNgGTL7c9EinHuRMew aAxsrK9nmdoWBs+ndhWX2BC5WRZaTCBdqLFpkIQ4jONAV9y6OrhcaD5WHshsv+tZ9hji ecRB3NszEmVd4Pa2jc8romh+fpw/EDaZeUn09KO+Fb3RFUKmvuYIA1ih6TluaiiPpiPd Cj6Gof1K+33i7sj4RPgxTp9EosoZY+mVMMVtOnUNaMrhqp326aQUWNKUVDUQcwov/QdT cDwutEa5jeCOPjtqFWQAdS7yHtqdj5Bs7OrxemJYL7mDGlOE8nqUOstodm6/H+27Uov5 4zhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=2rTk1Dfr7uSVsrTgPRhLiG3TO5OF4Se7SklVB6/EgrE=; b=cFBrNGt8qxYzuQDFxdbKALMznJL81e21akl+gU1LrEmqV9EP4T/FqhpahNot4xp29T UnqjEbgd6KKCZpWTiKrYYGl7IWqEiF3Awg7mEm4M/isKJkThwpIvr3VI6zc2HKet7Ihv GH+V46/vRsU9v5R/Q+PqnxPzFNeo1T11mnZdAFRuHeIZVwAHQGw4i6oCiGXFGLMgs1RR FQczAO3uu8NbfwZkNkCRhasnhWyVv5A+yo3ZtbmC2nAvWs/A7lV50CJyEiy++7dTJqqa vk3g2uBAkwWHDkb1TBwRk/djlmMuiHnSd4jVT3GL1WMobHlFuvpcLi6sUEScHhLTfLQa Okuw== X-Gm-Message-State: AOAM531sMN7qsk2WbRa5gfwdubHrdE3IdEpNc/0wiS1Ik0awHeXrM67j cGcfalLGXkjXGCF/xy7D/7su3N1/jMgjGw== X-Google-Smtp-Source: ABdhPJx9ObHdD4PbmxH62JKa2geylv+yQiP4MjkYd1Fiu8VGkMHwL/SHgGOyyMoN/w/QocKucfPpQw== X-Received: by 2002:ac8:3947:: with SMTP id t7mr1221566qtb.239.1602306510154; Fri, 09 Oct 2020 22:08:30 -0700 (PDT) Received: from hurd (dsl-10-148-10.b2b2c.ca. [72.10.148.10]) by smtp.gmail.com with ESMTPSA id l13sm8192475qtv.82.2020.10.09.22.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Oct 2020 22:08:29 -0700 (PDT) From: Maxim Cournoyer References: <87eem7qcxc.fsf@gmail.com> <20201010020410.3a301654@scratchpost.org> Date: Sat, 10 Oct 2020 01:08:28 -0400 In-Reply-To: <20201010020410.3a301654@scratchpost.org> (Danny Milosavljevic's message of "Sat, 10 Oct 2020 02:04:10 +0200") Message-ID: <874kn2r7lf.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 43893@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=B7PZVWGM; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: 1hDLxchyEIpe Hello Danny, Thanks for the quick investigation. Danny Milosavljevic writes: > I'm guessing it has something to do with update-guix-package using git-predicate > to add only git-known (but not necessarily committed) files to the store and then > calculating the checksum of that--but the git-fetch for the guix package not > necessarily doing the same. That's a good observation; it is indeed dangerous. In my case, my tree looks clean (no modified git-tracked files), but I had in fact modifications made to .dir-locals that I've been testing and these were hidden from the view by using: $ git update-index --skip-worktree .dir-locals But since the .dir-locals file is known to git, it was probably picked up with my invisible changes, causing the hash mismatch. > Then update-guix-package.scm does one worse and actively prevents guix from > doing the checkout from git when building that "guix" package. That means the > person invoking update-guix-package.scm can't notice even when the sha256 hash > is definitely wrong--because guix will have the source for package "guix" in > the store already (a faked entry added by update-guix-package.scm) and thus > won't fetch it again. > > Also, doesn't this entire approach have a problem? > > If you make a commit into the git repo of guix in order to update the > package "guix" to commit A, at that point you can't know what commit hash > commit A will have (since you haven't committed it yet) and yet you have > to know the commit hash of commit A in order to write it into the package > definition of package "guix". > > That cannot work. > The only way it works, more or less by accident is that, > > (1) At first, update-guix-package.scm does NOT update the "guix" package > inside, and calculates the hash of the working copy (hash A). > (2) Then, it updates the "guix" package inside to refer to hash A and to a > USER-SPECIFIED COMMIT HASH (the latter is determined by the user via > git rev-parse HEAD). > (3) Then, it commits that changed working copy as commit B. Commit B is > essentially not referred-to by anyone--it's just to make it to the > git repository so guix pull can pick it up. Yes, that's my understanding of how it works too. This means you have to be extra careful doing this while no-one else is commiting changes, else you have to start over because rebasing is not an option (it'd change the hashes, breaking the computed Guix hash). That's how I broke 'guix pull' the first time I used 'make update-guix-package' :-). But I think it's inevitable, so perhaps the best we can do is documement it well and print a warning when running the target. > That works only as long as there will be no reference to a nested-nested "guix" > package, by the eventual user. What do you mean by nested-nested Guix? Are there valid uses of such a thing? > @Maxim: I think this entire thing has to assume that > > git rev-parse HEAD > > (which it did at the very beginning of make update-guix-package) actually > refers to a commit that is available on the guix git repository on savannah. > > That means as soon as you change anything (no matter what) (and not actually > commit that) before invoking > > make update-guix-package > > the commit it refers to in the "guix" package will be one which cannot be > resolved by users. Indeed. [...] > Long story short, we should make "make update-guix-package" check for > uncommitted changes in the working copy, and fail if any such exist[1]. > There are no downsides that I can see. Even building from local working > copy still works then. Yes, that's a good step. Actually I just had an idea to use a clean worktree to do the computation, because that's even safer as it prevents subtle things like "git update-index --skip-worktree some/path" from interacting with the computed hash too. > Also, let's please document update-guix-package. I'll send a first commit. I haven't found a way to build it locally with the command in the message; it seems to create a cycle. Let me know what you think. Maxim