From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id SM31IRdNzl+PfQAA0tVLHw (envelope-from ) for ; Mon, 07 Dec 2020 15:41:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id oGjPHRdNzl8zMAAA1q6Kng (envelope-from ) for ; Mon, 07 Dec 2020 15:41:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3CED7940485 for ; Mon, 7 Dec 2020 15:41:11 +0000 (UTC) Received: from localhost ([::1]:38874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIdS-0003LV-64 for larch@yhetil.org; Mon, 07 Dec 2020 10:41:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIdK-0003L4-Kh for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43459) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kmIdK-0001R9-DP for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kmIdK-0004Gn-Bi for guix-patches@gnu.org; Mon, 07 Dec 2020 10:41:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45104] pull: Add a "with-substitutes" option. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45104 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 45104@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160735560516320 (code B ref -1); Mon, 07 Dec 2020 15:41:02 +0000 Received: (at submit) by debbugs.gnu.org; 7 Dec 2020 15:40:05 +0000 Received: from localhost ([127.0.0.1]:55002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIcP-0004F9-23 for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:40:05 -0500 Received: from lists.gnu.org ([209.51.188.17]:59678) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIcM-0004Ez-Md for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:40:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIcM-0002Zp-Dc for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:02 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57408) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIcM-0000wv-4V for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:02 -0500 Received: from [2a01:e0a:19b:d9a0:34f0:bcad:cad1:16d6] (port=34604 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmIcL-0006bB-7Q for guix-patches@gnu.org; Mon, 07 Dec 2020 10:40:01 -0500 From: Mathieu Othacehe Date: Mon, 07 Dec 2020 16:39:59 +0100 Message-ID: <87eek1vd4g.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.80 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 3CED7940485 X-Spam-Score: -2.80 X-Migadu-Scanner: ns3122888.ip-94-23-21.eu X-TUID: L3hsAJLrLNkf --=-=-= Content-Type: text/plain Hello, When "guix pull" is run before the CI server is done baking substitutes, the user may need to build some derivations, which can be long on low-end hardware. This patch adds an option to "guix pull" so that it updates to the latest commit with available substitutes. This work is still in progress, but I'd like to gather some impressions on that before going further. Thanks, Mathieu --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: inline; filename=0001-scripts-pull-Add-with-substitutes-option.patch Content-Transfer-Encoding: quoted-printable >From d399f8dbb9e38a82241b9048b8b04758fae10005 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Mon, 7 Dec 2020 16:12:22 +0100 Subject: [PATCH] scripts: pull: Add "with-substitutes" option. --- guix/scripts/pull.scm | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index 83cdc1d1eb..4609f8614e 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -2,6 +2,7 @@ ;;; Copyright =C2=A9 2013, 2014, 2015, 2017, 2018, 2019, 2020 Ludovic Cour= t=C3=A8s ;;; Copyright =C2=A9 2017 Marius Bakke ;;; Copyright =C2=A9 2020 Tobias Geerinckx-Rice +;;; Copyright =C2=A9 2020 Mathieu Othacehe ;;; ;;; This file is part of GNU Guix. ;;; @@ -20,6 +21,7 @@ =20 (define-module (guix scripts pull) #:use-module ((guix ui) #:hide (display-profile-content)) + #:use-module (guix ci) #:use-module (guix colors) #:use-module (guix utils) #:use-module ((guix status) #:select (with-status-verbosity)) @@ -64,6 +66,7 @@ #:re-export (display-profile-content channel-commit-hyperlink) #:export (channel-list + find-lastest-commit-with-substitutes guix-pull)) =20 @@ -169,6 +172,9 @@ Download and deploy the latest version of Guix.\n")) (lambda (opt name arg result) (alist-cons 'validate-pull warn-about-backward-updates result))) + (option '("with-substitutes") #f #f + (lambda (opt name arg result) + (alist-cons 'with-substitutes? #t result))) (option '("disable-authentication") #f #f (lambda (opt name arg result) (alist-cons 'authenticate-channels? #f result))) @@ -526,6 +532,26 @@ true, display what would be built without actually bui= lding it." (leave (G_ "while creating symlink '~a': ~a~%") link (strerror (system-error-errno args)))))))) =20 + +;;; +;;; Substitutes. +;;; + +(define (find-lastest-commit-with-substitutes) + (let ((urls %default-substitute-urls)) + (any (lambda (url) + (let* ((build (match (latest-builds url 1 + #:job "guix.x86_64-linux" + #:status 0) ;success + ((build) build))) + (evaluation (evaluation url + (build-evaluation build))) + (commit (match (evaluation-checkouts evaluation) + ((checkout) + (checkout-commit checkout))))) + commit)) + urls))) + ;;; ;;; Queries. @@ -731,8 +757,9 @@ Use '~/.config/guix/channels.scm' instead.")) =20 (let ((ref (assoc-ref opts 'ref)) (url (or (assoc-ref opts 'repository-url) - (environment-variable)))) - (if (or ref url) + (environment-variable))) + (with-substitutes? (assoc-ref opts 'with-substitutes?))) + (if (or ref url with-substitutes?) (match (find guix-channel? channels) ((? channel? guix) ;; Apply '--url', '--commit', and '--branch' to the 'guix' chan= nel. @@ -745,7 +772,12 @@ Use '~/.config/guix/channels.scm' instead.")) (channel (inherit guix) (url url) (commit #f) (branch branch))) (#f - (channel (inherit guix) (url url)))) + (let ((commit + (and with-substitutes? + (find-lastest-commit-with-substitutes)))) + (channel (inherit guix) + (url url) + (commit commit))))) (remove guix-channel? channels)))) (#f ;no 'guix' channel, failure will e= nsue channels)) --=20 2.29.2 --=-=-=--