From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CO3JMTNuhl8mSgAA0tVLHw (envelope-from ) for ; Wed, 14 Oct 2020 03:19:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id iF2wLTNuhl+HTAAA1q6Kng (envelope-from ) for ; Wed, 14 Oct 2020 03:19:15 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A77A294051F for ; Wed, 14 Oct 2020 03:19:14 +0000 (UTC) Received: from localhost ([::1]:40960 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kSXJo-0006KO-Jf for larch@yhetil.org; Tue, 13 Oct 2020 23:19:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kSXJe-0006JY-5a for bug-guix@gnu.org; Tue, 13 Oct 2020 23:19:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38001) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kSXJd-0000u3-ST for bug-guix@gnu.org; Tue, 13 Oct 2020 23:19:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kSXJd-000446-Nl for bug-guix@gnu.org; Tue, 13 Oct 2020 23:19:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43893: [PATCH v3] maint: update-guix-package: Prevent accidentally breaking guix pull. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 14 Oct 2020 03:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43893 X-GNU-PR-Package: guix X-GNU-PR-Keywords: patch To: 43893@debbugs.gnu.org Received: via spool by 43893-submit@debbugs.gnu.org id=B43893.160264548415531 (code B ref 43893); Wed, 14 Oct 2020 03:19:01 +0000 Received: (at 43893) by debbugs.gnu.org; 14 Oct 2020 03:18:04 +0000 Received: from localhost ([127.0.0.1]:49547 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kSXIh-00042R-RS for submit@debbugs.gnu.org; Tue, 13 Oct 2020 23:18:04 -0400 Received: from mail-il1-f196.google.com ([209.85.166.196]:39448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kSXIf-00041w-34 for 43893@debbugs.gnu.org; Tue, 13 Oct 2020 23:18:02 -0400 Received: by mail-il1-f196.google.com with SMTP id q1so3548012ilt.6 for <43893@debbugs.gnu.org>; Tue, 13 Oct 2020 20:18:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NpxD8fVmrvRyMrFF4/r3MAZoT01J8fBQGWi+bQMEcME=; b=T+K2eDQHtDTRk35MX1nnAA5U0HtlciFO0VGn3TrDK/xaY0K5IN8yLCzjYicJaDuRmV JXNzYc/Ckh8zbghvLSj11IvW1dfOx/YwPg8zien6Fz+h3BHhlJOGAVgDs2pZzKq95QEU svdgmmyqMnSwBRwrGQaJYDVSMx6K1JtA/vukYqIa9728VKVoCyTmNwo3LigyXIqM+jmc 26OWcV6D/hbNU78v54khmJmwdvXGagP5HjxDvjrI2vwbFZV2rMWCuUNyn4nD9EjHr0pl uvYBV34MZUdgbTH2HRVUpCXwkI30YQ9CQppljSKz3VWn0FIiW9L3WpvdGLnJHDWVM5CD 8o/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NpxD8fVmrvRyMrFF4/r3MAZoT01J8fBQGWi+bQMEcME=; b=OLfFos8c8ZJNHkl2JoZWj6Kx/cf7iPYE3IPt/7K+sovl+5Lx29H+l70cO8719uEHeI othpRvFiQuuyaiiItvPqI8pIncfFrVr4FwD0Wh7TyWzsi0HEkc/iT8MjH4Ubd6G+WRNK KwrfOs/ETwVqcOzfcBza4lH17feNN3mhaK4k+OH3yIieHmaThywYQ+V3iUpaoe+4Khil g7eRdKM+oYkS2zrUAJnN5MiOtGAO/yQXx7VX1tOqkgvD3IOj6vy2c8PSwXdoQh5rO5HP qQXlBDbUrFFBSUing7tcoCdB5TSiiF+LoMnCea/e9E3hJoUJ+OUoNy0tioWsewhU9gt8 z7Pg== X-Gm-Message-State: AOAM530FAHG2DjdN0py4TDny3jZdKF0Dj69JQuwQXdJcP+ic7AJwz13I Vexax20TNZ2nsPvmnPrTln6KhwewVu6CPg== X-Google-Smtp-Source: ABdhPJxbDU0D24P+6O81XetRg+6YfB/N7W/pSh7CjmcLhH4mM1Hu29n/TWgdPuvlg4kE46YwwZxJ2g== X-Received: by 2002:a92:3608:: with SMTP id d8mr2294983ila.2.1602645475057; Tue, 13 Oct 2020 20:17:55 -0700 (PDT) Received: from localhost.localdomain (dsl-155-33.b2b2c.ca. [66.158.155.33]) by smtp.gmail.com with ESMTPSA id p23sm1682747ioo.10.2020.10.13.20.17.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2020 20:17:54 -0700 (PDT) From: Maxim Cournoyer Date: Tue, 13 Oct 2020 23:17:05 -0400 Message-Id: <20201014031705.4516-1-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <87imbedsko.fsf@gnu.org> References: <87imbedsko.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=yes Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Maxim Cournoyer Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=T+K2eDQH; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 1.59 X-TUID: bGKXdocZpf6p Fixes . This changes the 'update-guix-package' tool so that it: 1. Always uses a clean checkout to compute the hash of the updated 'guix' package. 2. Ensures the commit used in the updated 'guix' package definition has already been pushed upstream. * build-aux/update-guix-package.scm (%savannah-guix-git-repo-push-url): New variable. (with-input-pipe-to-string, with-temporary-git-worktree): New syntaxes. (find-origin-remote, git-add-worktree): New procedures. (commit-already-pushed?): New predicate. (main): Check the commit used has already been pushed upstream and compute the hash from a clean checkout. * doc/contributing.texi (Updating the Guix Package): Document it. * .dir-locals.el (scheme-mode): Fix indentation of with-temporary-git-worktree. --- .dir-locals.el | 1 + build-aux/update-guix-package.scm | 98 +++++++++++++++++++++---------- doc/contributing.texi | 43 ++++++++++++++ 3 files changed, 112 insertions(+), 30 deletions(-) diff --git a/.dir-locals.el b/.dir-locals.el index 7f310d2612..19f15b3e1a 100644 --- a/.dir-locals.el +++ b/.dir-locals.el @@ -103,6 +103,7 @@ (eval . (put 'call-with-progress-reporter 'scheme-indent-function 1)) (eval . (put 'with-repository 'scheme-indent-function 2)) (eval . (put 'with-temporary-git-repository 'scheme-indent-function 2)) + (eval . (put 'with-temporary-git-worktree 'scheme-indent-function 2)) (eval . (put 'with-environment-variables 'scheme-indent-function 1)) (eval . (put 'with-fresh-gnupg-setup 'scheme-indent-function 1)) diff --git a/build-aux/update-guix-package.scm b/build-aux/update-guix-package.scm index f695e91cfd..9b03b06c7c 100644 --- a/build-aux/update-guix-package.scm +++ b/build-aux/update-guix-package.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017, 2018 Ludovic Courtès +;;; Copyright © 2020 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,13 +25,20 @@ ;;; Code: (use-modules (guix) + (guix ui) (guix git-download) (guix upstream) (guix utils) (guix base32) (guix build utils) + (guix scripts hash) (gnu packages package-management) - (ice-9 match)) + (ice-9 match) + (ice-9 popen) + (ice-9 textual-ports) + (srfi srfi-1) + (srfi srfi-2) + (srfi srfi-26)) (define %top-srcdir (string-append (current-source-directory) "/..")) @@ -101,44 +109,74 @@ COMMIT." (exp (error "'guix' package definition is not as expected" exp))))) +(define (git-add-worktree directory commit) + "Create a new git worktree at DIRECTORY, detached on commit COMMIT." + (invoke "git" "worktree" "add" "--detach" directory commit)) + +(define-syntax-rule (with-temporary-git-worktree commit body ...) + "Execute BODY in the context of a temporary git worktree created from COMMIT." + (call-with-temporary-directory + (lambda (tmp-directory) + (dynamic-wind + (lambda () + #t) + (lambda () + (git-add-worktree tmp-directory commit) + (with-directory-excursion tmp-directory body ...)) + (lambda () + (invoke "git" "worktree" "remove" "--force" tmp-directory)))))) + +(define %savannah-guix-git-repo-push-url + "git.savannah.gnu.org/srv/git/guix.git") + +(define-syntax-rule (with-input-pipe-to-string prog arg ...) + (let* ((input-pipe (open-pipe* OPEN_READ prog arg ...)) + (output (get-string-all input-pipe)) + (exit-val (status:exit-val (close-pipe input-pipe)))) + (unless (zero? exit-val) + (error (format #f "Command ~s exited with non-zero exit status: ~s" + (string-join (list prog arg ...)) exit-val))) + (string-trim-both output))) + +(define (find-origin-remote) + "Find the name of the git remote with the Savannah Guix git repo URL." + (and-let* ((remotes (string-split (with-input-pipe-to-string + "git" "remote" "-v") + #\newline)) + (origin-entry (find (cut string-contains <> + (string-append + %savannah-guix-git-repo-push-url + " (push)")) + remotes))) + (first (string-split origin-entry #\tab)))) + +(define (commit-already-pushed? remote commit) + "True if COMMIT is found in the REMOTE repository." + (not (string-null? (with-input-pipe-to-string + "git" "branch" "-r" "--contains" commit + (string-append remote "/master"))))) + (define (main . args) (match args ((commit version) - (with-store store - (let* ((source (add-to-store store - "guix-checkout" ;dummy name - #t "sha256" %top-srcdir - #:select? version-controlled?)) - (hash (query-path-hash store source)) + (with-directory-excursion %top-srcdir + (or (getenv "GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT") + (commit-already-pushed? (find-origin-remote) commit) + (leave (G_ "Commit ~a is not pushed upstream. Aborting.~%") commit)) + (let* ((hash (with-temporary-git-worktree commit + (nix-base32-string->bytevector + (string-trim-both + (with-output-to-string + (lambda () + (guix-hash "-rx" "."))))))) (location (package-definition-location)) (old-hash (content-hash-value - (origin-hash (package-source guix))))) + (origin-hash (package-source guix))))) (edit-expression location (update-definition commit hash #:old-hash old-hash - #:version version)) - - ;; Re-add SOURCE to the store, but this time under the real name used - ;; in the 'origin'. This allows us to build the package without - ;; having to make a real checkout; thus, it also works when working - ;; on a private branch. - (reload-module - (resolve-module '(gnu packages package-management))) - - (let* ((source (add-to-store store - (origin-file-name (package-source guix)) - #t "sha256" source)) - (root (store-path-package-name source))) - - ;; Add an indirect GC root for SOURCE in the current directory. - (false-if-exception (delete-file root)) - (symlink source root) - (add-indirect-root store - (string-append (getcwd) "/" root)) - - (format #t "source code for commit ~a: ~a (GC root: ~a)~%" - commit source root))))) + #:version version))))) ((commit) ;; Automatically deduce the version and revision numbers. (main commit #f)))) diff --git a/doc/contributing.texi b/doc/contributing.texi index af3601442e..11a932a9bf 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -28,6 +28,7 @@ choice. * Submitting Patches:: Share your work. * Tracking Bugs and Patches:: Using Debbugs. * Commit Access:: Pushing to the official repository. +* Updating the Guix Package:: Updating the Guix package definition. @end menu @node Building from Git @@ -1323,3 +1324,45 @@ only push their own awesome changes, but also offer some of their time @emph{reviewing} and pushing other people's changes. As a committer, you're welcome to use your expertise and commit rights to help other contributors, too! + +@node Updating the Guix Package +@section Updating the Guix Package + +@cindex update-guix-package, updating the guix package +It is sometimes desirable to update the @code{guix} package itself (the +package defined in @code{(gnu packages package-management)}), for +example to make new daemon features available for use by the +@code{guix-service-type} service type. In order to simplify this task, +the following command can be used: + +@example +make update-guix-package +@end example + +The @code{update-guix-package} make target will use the last known +@emph{commit} corresponding to @code{HEAD} in your Guix checkout, +compute the hash of the Guix sources corresponding to that commit and +update the @code{commit}, @code{revision} and hash of the @code{guix} +package definition. + +To validate that the updated @code{guix} package hashes are correct and +that it can be built successfully, the following command can be run from +the directory of your Guix checkout: + +@example +./pre-inst-env guix build guix +@end example + +To guard against accidentally updating the @code{guix} package to a +commit that others can't refer to, a check is made that the commit used +has already been pushed to the Savannah-hosted Guix git repository. + +This check can be disabled, @emph{at your own peril}, by setting the +@code{GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT} environment variable. + +To build the resulting 'guix' package when using a private commit, the +following command can be used: + +@example +./pre-inst-env guix build guix --with-git-url=guix=$PWD +@end example -- 2.28.0