From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 2I/HB0dlg18qWAAA0tVLHw (envelope-from ) for ; Sun, 11 Oct 2020 20:04:23 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iFOKA0dlg1+vDQAAB5/wlQ (envelope-from ) for ; Sun, 11 Oct 2020 20:04:23 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 474A59401E3 for ; Sun, 11 Oct 2020 20:04:21 +0000 (UTC) Received: from localhost ([::1]:60246 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kRhZr-0008Fo-GT for larch@yhetil.org; Sun, 11 Oct 2020 16:04:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kRhUj-0004VQ-UR for bug-guix@gnu.org; Sun, 11 Oct 2020 15:59:01 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:57789) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kRhUj-0001QK-L3 for bug-guix@gnu.org; Sun, 11 Oct 2020 15:59:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kRhUj-00017a-KO for bug-guix@gnu.org; Sun, 11 Oct 2020 15:59:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43893: [PATCH v2] maint: update-guix-package: Prevent accidentally breaking guix pull. References: <87eem7qcxc.fsf@gmail.com> In-Reply-To: <87eem7qcxc.fsf@gmail.com> Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 11 Oct 2020 19:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43893 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 43893@debbugs.gnu.org Received: via spool by 43893-submit@debbugs.gnu.org id=B43893.16024462934246 (code B ref 43893); Sun, 11 Oct 2020 19:59:01 +0000 Received: (at 43893) by debbugs.gnu.org; 11 Oct 2020 19:58:13 +0000 Received: from localhost ([127.0.0.1]:41100 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kRhTw-00016P-Qq for submit@debbugs.gnu.org; Sun, 11 Oct 2020 15:58:13 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:42694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kRhTv-00016E-RY for 43893@debbugs.gnu.org; Sun, 11 Oct 2020 15:58:12 -0400 Received: by mail-qt1-f196.google.com with SMTP id t9so11398000qtp.9 for <43893@debbugs.gnu.org>; Sun, 11 Oct 2020 12:58:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=eeKbl+OiwT4i7R1w2hHnqUmE/Ttx6jXVIv098rlk6OY=; b=H2fM+tARE/pizmiH+cvTFkdZL2EFXpLUVA21eewzBbqAjnXRCgPwBiunBfAK9ozQwC jhv1rgrALoxr9W3hUsXRDiUC3n/xtDE3gj5TXnl6EasebLFojmLc1RMkg1Kw0jtUELPE j3KEiDVMVnC1/JNp6LEI1uCYIG4zzj/3gD2j8ju7PssBofKOdavIOxhetprKA8DhvvDd 3c8YCroLpZQ9GZXbpg+HvPS+aDsQv/ueGTU6aLm/Cb8DfmZgPZSHgWujy8cOHmJYnWCf E6oMsa95Eo7xAPQ9KuIxobrxK2PFVrRvoZDrv8YtnkF0DZhmAx2s8mBi8xbJmok82znD Wnrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=eeKbl+OiwT4i7R1w2hHnqUmE/Ttx6jXVIv098rlk6OY=; b=SNpGr2UZE4mo5zvjsGT9s6ZVzQZFcSx/p6NWAp5MjXHa9I5syJE8C33L5jiiv0lsR/ gPjZkPjT31cqRhe5SeyDGYXahtRumM+Yen2ytAJs+iqoBoIJitNbM71FD09FHFvf5baw 2Kjo9ylcd0iN9Lv7RgGB3rXoe5Tk9bB6tipEeMVSVKORXbzysJ1GZxzhTGkRry5a5GP4 28euNSqIXH96ofo1HQ6sK53WXnqpl7SO5wguZQCkBNoqCz3OMvWCAsvYG/gWOJbzUZWM ZqN6PKDy/krWo9YKIACwtyKdlZlEIQmHgJcJnhMolbV3tATOhGWh9M8uqTvIKhh6HAX2 5Mlg== X-Gm-Message-State: AOAM5320yEa9PKw49bqq1fkaGy04BfYWZsn43/73FuX1Hwv6B766mttp YrN2xgENvfTcX5CuX0PG+c81Efvvw1Y= X-Google-Smtp-Source: ABdhPJxqCfSXwOe19DVLt1LXD0zuUk/hRE3j1iDPY3mMwP7TMyZdUR2FWafUk0TeRc8Cm2z9/Hk2TQ== X-Received: by 2002:aed:27dc:: with SMTP id m28mr7195636qtg.12.1602446285768; Sun, 11 Oct 2020 12:58:05 -0700 (PDT) Received: from localhost.localdomain (dsl-10-136-39.b2b2c.ca. [72.10.136.39]) by smtp.gmail.com with ESMTPSA id l30sm11588584qta.73.2020.10.11.12.58.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Oct 2020 12:58:05 -0700 (PDT) From: Maxim Cournoyer Date: Sun, 11 Oct 2020 15:57:40 -0400 Message-Id: <20201011195740.1993-1-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Type: text/plain; charset=yes Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxim Cournoyer Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=H2fM+tAR; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: yrn0jMoH5S5F Fixes . This changes the 'update-guix-package' tool so that it: 1. Always uses a clean checkout to compute the hash of the updated 'guix' package. 2. Ensures the commit used in the updated 'guix' package definition has already been pushed upstream. * build-aux/update-guix-package.scm (%savannah-guix-git-repo-push-url): New variable. (with-input-pipe-to-string): New syntax. (find-origin-remote, git-add-worktree): New procedures. (commit-already-pushed?): New predicate. (main): Check the commit used has already been pushed upstream and compute the hash from a clean checkout. * doc/contributing.texi (Updating the Guix Package): Document it. --- build-aux/update-guix-package.scm | 103 ++++++++++++++++++++---------- doc/contributing.texi | 43 +++++++++++++ 2 files changed, 111 insertions(+), 35 deletions(-) diff --git a/build-aux/update-guix-package.scm b/build-aux/update-guix-package.scm index f695e91cfd..397b404922 100644 --- a/build-aux/update-guix-package.scm +++ b/build-aux/update-guix-package.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017, 2018 Ludovic Courtès +;;; Copyright © 2020 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,13 +25,20 @@ ;;; Code: (use-modules (guix) + (guix ui) (guix git-download) (guix upstream) (guix utils) (guix base32) (guix build utils) + (guix scripts hash) (gnu packages package-management) - (ice-9 match)) + (ice-9 match) + (ice-9 popen) + (ice-9 textual-ports) + (srfi srfi-1) + (srfi srfi-2) + (srfi srfi-26)) (define %top-srcdir (string-append (current-source-directory) "/..")) @@ -101,44 +109,69 @@ COMMIT." (exp (error "'guix' package definition is not as expected" exp))))) - -(define (main . args) - (match args - ((commit version) - (with-store store - (let* ((source (add-to-store store - "guix-checkout" ;dummy name - #t "sha256" %top-srcdir - #:select? version-controlled?)) - (hash (query-path-hash store source)) - (location (package-definition-location)) - (old-hash (content-hash-value - (origin-hash (package-source guix))))) - (edit-expression location - (update-definition commit hash - #:old-hash old-hash - #:version version)) +(define (git-add-worktree directory commit-ish) + "Create a new git worktree at DIRECTORY, detached on commit COMMIT-ISH." + (invoke "git" "worktree" "add" "--detach" directory commit-ish)) + +(define %savannah-guix-git-repo-push-url + "git.savannah.gnu.org/srv/git/guix.git") - ;; Re-add SOURCE to the store, but this time under the real name used - ;; in the 'origin'. This allows us to build the package without - ;; having to make a real checkout; thus, it also works when working - ;; on a private branch. - (reload-module - (resolve-module '(gnu packages package-management))) +(define-syntax-rule (with-input-pipe-to-string prog arg ...) + (let* ((input-pipe (open-pipe* OPEN_READ prog arg ...)) + (output (get-string-all input-pipe)) + (exit-val (status:exit-val (close-pipe input-pipe)))) + (unless (zero? exit-val) + (error (format #f "Command ~s exited with non-zero exit status: ~s" + (string-join (list prog arg ...)) exit-val))) + (string-trim-both output))) - (let* ((source (add-to-store store - (origin-file-name (package-source guix)) - #t "sha256" source)) - (root (store-path-package-name source))) +(define (find-origin-remote) + "Find the name of the git remote with the Savannah Guix git repo URL." + (and-let* ((remotes (string-split (with-input-pipe-to-string + "git" "remote" "-v") + #\newline)) + (origin-entry (find (cut string-contains <> + (string-append + %savannah-guix-git-repo-push-url + " (push)")) + remotes))) + (first (string-split origin-entry #\tab)))) - ;; Add an indirect GC root for SOURCE in the current directory. - (false-if-exception (delete-file root)) - (symlink source root) - (add-indirect-root store - (string-append (getcwd) "/" root)) +(define (commit-already-pushed? remote commit) + "True if COMMIT is found in the REMOTE repository." + (not (string-null? (with-input-pipe-to-string + "git" "branch" "-r" "--contains" commit + (string-append remote "/master"))))) - (format #t "source code for commit ~a: ~a (GC root: ~a)~%" - commit source root))))) + +(define (main . args) + (match args + ((commit version) + (with-directory-excursion %top-srcdir + (or (getenv "GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT") + (commit-already-pushed? (find-origin-remote) commit) + (leave (G_ "Commit ~a is not pushed upstream. Aborting.~%") commit)) + (dynamic-wind + (lambda () + #t) + (lambda () + (call-with-temporary-directory + (lambda (tmp-directory) + (let* ((dummy (git-add-worktree tmp-directory commit)) + (hash (nix-base32-string->bytevector + (string-trim-both + (with-output-to-string + (lambda () + (guix-hash "-rx" tmp-directory)))))) + (location (package-definition-location)) + (old-hash (content-hash-value + (origin-hash (package-source guix))))) + (edit-expression location + (update-definition commit hash + #:old-hash old-hash + #:version version)))))) + (lambda () + (invoke "git" "worktree" "prune"))))) ((commit) ;; Automatically deduce the version and revision numbers. (main commit #f)))) diff --git a/doc/contributing.texi b/doc/contributing.texi index af3601442e..11a932a9bf 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -28,6 +28,7 @@ choice. * Submitting Patches:: Share your work. * Tracking Bugs and Patches:: Using Debbugs. * Commit Access:: Pushing to the official repository. +* Updating the Guix Package:: Updating the Guix package definition. @end menu @node Building from Git @@ -1323,3 +1324,45 @@ only push their own awesome changes, but also offer some of their time @emph{reviewing} and pushing other people's changes. As a committer, you're welcome to use your expertise and commit rights to help other contributors, too! + +@node Updating the Guix Package +@section Updating the Guix Package + +@cindex update-guix-package, updating the guix package +It is sometimes desirable to update the @code{guix} package itself (the +package defined in @code{(gnu packages package-management)}), for +example to make new daemon features available for use by the +@code{guix-service-type} service type. In order to simplify this task, +the following command can be used: + +@example +make update-guix-package +@end example + +The @code{update-guix-package} make target will use the last known +@emph{commit} corresponding to @code{HEAD} in your Guix checkout, +compute the hash of the Guix sources corresponding to that commit and +update the @code{commit}, @code{revision} and hash of the @code{guix} +package definition. + +To validate that the updated @code{guix} package hashes are correct and +that it can be built successfully, the following command can be run from +the directory of your Guix checkout: + +@example +./pre-inst-env guix build guix +@end example + +To guard against accidentally updating the @code{guix} package to a +commit that others can't refer to, a check is made that the commit used +has already been pushed to the Savannah-hosted Guix git repository. + +This check can be disabled, @emph{at your own peril}, by setting the +@code{GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT} environment variable. + +To build the resulting 'guix' package when using a private commit, the +following command can be used: + +@example +./pre-inst-env guix build guix --with-git-url=guix=$PWD +@end example -- 2.28.0