From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 2DxmFhUQFGAbbAAA0tVLHw (envelope-from ) for ; Fri, 29 Jan 2021 13:39:33 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id kYoQEhUQFGCmbwAAB5/wlQ (envelope-from ) for ; Fri, 29 Jan 2021 13:39:33 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C217E940309 for ; Fri, 29 Jan 2021 13:39:32 +0000 (UTC) Received: from localhost ([::1]:58528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5Tzn-00018F-Mp for larch@yhetil.org; Fri, 29 Jan 2021 08:39:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33606) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5TxO-0007Do-MJ for guix-patches@gnu.org; Fri, 29 Jan 2021 08:37:09 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:39836) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l5TxN-0004js-Mm for guix-patches@gnu.org; Fri, 29 Jan 2021 08:37:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1l5TxN-0003aq-Is for guix-patches@gnu.org; Fri, 29 Jan 2021 08:37:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45104] pull: Add a "with-substitutes" option. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 29 Jan 2021 13:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45104 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 45104-submit@debbugs.gnu.org id=B45104.161192740213786 (code B ref 45104); Fri, 29 Jan 2021 13:37:01 +0000 Received: (at 45104) by debbugs.gnu.org; 29 Jan 2021 13:36:42 +0000 Received: from localhost ([127.0.0.1]:51382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l5Tx4-0003aH-5q for submit@debbugs.gnu.org; Fri, 29 Jan 2021 08:36:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43922) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l5Tx2-0003a5-7Q for 45104@debbugs.gnu.org; Fri, 29 Jan 2021 08:36:40 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:50140) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5Tww-0004Wt-Lr; Fri, 29 Jan 2021 08:36:34 -0500 Received: from [2a01:e0a:19b:d9a0:10b6:a2c3:9f4e:6dae] (port=56768 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l5Twu-0002Am-Hp; Fri, 29 Jan 2021 08:36:34 -0500 From: Mathieu Othacehe References: <87eek1vd4g.fsf@gnu.org> <87a6uohztw.fsf@cbaines.net> <877dpktzot.fsf@gnu.org> <878s9zfjt4.fsf@gnu.org> <875z52loam.fsf@gnu.org> <871re3khbt.fsf@gnu.org> Date: Fri, 29 Jan 2021 14:36:30 +0100 In-Reply-To: <871re3khbt.fsf@gnu.org> (Mathieu Othacehe's message of "Fri, 29 Jan 2021 14:23:18 +0100") Message-ID: <87y2gbj25d.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 45104@debbugs.gnu.org Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.85 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: C217E940309 X-Spam-Score: -2.85 X-Migadu-Scanner: scn1.migadu.com X-TUID: k1jJumLo/Hnx --=-=-= Content-Type: text/plain Here's a v2 with the missing docstrings. Thanks, Mathieu --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: inline; filename=0001-guix-channels-Introduce-channel-with-substitutes-ava.patch Content-Transfer-Encoding: quoted-printable >From 31dad6456825a329ba0f07c95e3e99258d186a8f Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Fri, 29 Jan 2021 13:48:44 +0100 Subject: [PATCH v2] guix: channels: Introduce "channel-with-substitutes-available". * guix/channels.scm (find-latest-commit-with-substitutes, channel-with-substitutes-available): New procedures. * guix/scripts/pull.scm (guix-pull): Move "channel-list" call inside the %current-system parameter scope. * doc/guix.texi (Channels with substitutes): New section. --- doc/guix.texi | 27 ++++++++++- guix/channels.scm | 37 ++++++++++++++- guix/scripts/pull.scm | 103 +++++++++++++++++++++--------------------- 3 files changed, 114 insertions(+), 53 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index ff9e8da2e0..6587a49d0e 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -40,7 +40,7 @@ Copyright @copyright{} 2016, 2017, 2018, 2019, 2020 Julie= n Lepiller@* Copyright @copyright{} 2016 Alex ter Weele@* Copyright @copyright{} 2016, 2017, 2018, 2019 Christopher Baines@* Copyright @copyright{} 2017, 2018, 2019 Cl=C3=A9ment Lassieur@* -Copyright @copyright{} 2017, 2018, 2020 Mathieu Othacehe@* +Copyright @copyright{} 2017, 2018, 2020, 2021 Mathieu Othacehe@* Copyright @copyright{} 2017 Federico Beffa@* Copyright @copyright{} 2017, 2018 Carlo Zancanaro@* Copyright @copyright{} 2017 Thomas Danckaert@* @@ -245,6 +245,7 @@ Channels * Specifying Channel Authorizations:: Defining channel authors authorizat= ions. * Primary URL:: Distinguishing mirror to original. * Writing Channel News:: Communicating information to channel's use= rs. +* Channels with substitutes:: Using channels with available substitutes. =20 Development =20 @@ -4919,6 +4920,7 @@ updates. * Specifying Channel Authorizations:: Defining channel authors authorizat= ions. * Primary URL:: Distinguishing mirror to original. * Writing Channel News:: Communicating information to channel's use= rs. +* Channels with substitutes:: Using channels with available substitutes. @end menu =20 @node Specifying Additional Channels @@ -5390,6 +5392,29 @@ xgettext -o news.po -l scheme -ken etc/news.txt To sum up, yes, you could use your channel as a blog. But beware, this is @emph{not quite} what your users might expect. =20 +@node Channels with substitutes +@section Channels with substitutes + +When running @command{guix pull}, Guix will first compile the +definitions of every available package. This is an expensive operation +for which substitutes (@pxref{Substitutes}) may be available. The +following snippet in @file{channels.scm} will ensure that @command{guix +pull} uses the latest commit with available substitutes for the package +definitions. This is done by querying the continuous integration +server at @url{https://ci.guix.gnu.org}. + +Be careful, this does not mean that all the packages that you will +install after running @command{guix pull} will have available +substitutes. It only ensures that @command{guix pull} will not try to +compile package definitions. This is particularly useful when using +machines with limited resources. + +@lisp +(list + (channel-with-substitutes-available + %default-guix-channel + "https://ci.guix.gnu.org")) +@end lisp =20 @c ********************************************************************* @node Development diff --git a/guix/channels.scm b/guix/channels.scm index 0c84eed477..a04eae1c10 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -20,6 +20,7 @@ =20 (define-module (guix channels) #:use-module (git) + #:use-module (guix ci) #:use-module (guix git) #:use-module (guix git-authenticate) #:use-module ((guix openpgp) @@ -98,7 +99,9 @@ channel-news-entry-title channel-news-entry-body =20 - channel-news-for-commit)) + channel-news-for-commit + + channel-with-substitutes-available)) =20 ;;; Commentary: ;;; @@ -1044,6 +1047,38 @@ NEW. When OLD is omitted or is #f, return all the n= ews entries of CHANNEL." '() (apply throw key error rest))))) =20 +(define (find-latest-commit-with-substitutes url) + "Return the latest commit with available substitutes for the Guix package +definitions at URL. Return false if no commit were found." + (let* ((job-name (string-append "guix." (%current-system))) + (build (match (latest-builds url 1 + #:job job-name + #:status 0) ;success + ((build) build) + (_ #f))) + (evaluation (and build + (evaluation url (build-evaluation build)))) + (commit (and evaluation + (match (evaluation-checkouts evaluation) + ((checkout) + (checkout-commit checkout)))))) + commit)) + +(define (channel-with-substitutes-available chan url) + "Return a channel inheriting from CHAN but which commit field is set to = the +latest commit with available substitutes for the Guix package definitions = at +URL. The current system is taken into account. + +If no commit with available substitutes were found, the commit field is se= t to +false and a warning message is printed." + (let ((commit (find-latest-commit-with-substitutes url))) + (unless commit + (warning (G_ "could not find available substitutes at ~a~%") + url)) + (channel + (inherit chan) + (commit commit)))) + ;;; Local Variables: ;;; eval: (put 'with-guile 'scheme-indent-function 1) ;;; End: diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm index 83cdc1d1eb..4e0ab5d341 100644 --- a/guix/scripts/pull.scm +++ b/guix/scripts/pull.scm @@ -765,60 +765,61 @@ Use '~/.config/guix/channels.scm' instead.")) #:argument-handler no-argume= nts)) (substitutes? (assoc-ref opts 'substitutes?)) (dry-run? (assoc-ref opts 'dry-run?)) - (channels (channel-list opts)) (profile (or (assoc-ref opts 'profile) %current-profile)) (current-channels (profile-channels profile)) (validate-pull (assoc-ref opts 'validate-pull)) (authenticate? (assoc-ref opts 'authenticate-channels?))) - (cond ((assoc-ref opts 'query) - (process-query opts profile)) - ((assoc-ref opts 'generation) - (process-generation-change opts profile)) - (else - (with-store store - (with-status-verbosity (assoc-ref opts 'verbosity) - (parameterize ((%current-system (assoc-ref opts 'system)) - (%graft? (assoc-ref opts 'graft?))) - (with-build-handler (build-notifier #:use-substitutes? - substitutes? - #:verbosity - (assoc-ref opts 'v= erbosity) - #:dry-run? dry-run= ?) - (set-build-options-from-command-line store opts) - (ensure-default-profile) - (honor-x509-certificates store) - - (let ((instances - (latest-channel-instances store channels - #:current-channels - current-channels - #:validate-pull - validate-pull - #:authenticate? - authenticate?))) - (format (current-error-port) - (N_ "Building from this channel:~%" - "Building from these channels:~%" - (length instances))) - (for-each (lambda (instance) - (let ((channel - (channel-instance-channel insta= nce))) - (format (current-error-port) - " ~10a~a\t~a~%" - (channel-name channel) - (channel-url channel) - (string-take - (channel-instance-commit in= stance) - 7)))) - instances) - (parameterize ((%guile-for-build - (package-derivation - store - (if (assoc-ref opts 'bootstrap?) - %bootstrap-guile - (default-guile))))) - (with-profile-lock profile - (run-with-store store - (build-and-install instances profile))))))))= ))))))) + (cond + ((assoc-ref opts 'query) + (process-query opts profile)) + ((assoc-ref opts 'generation) + (process-generation-change opts profile)) + (else + (with-store store + (with-status-verbosity (assoc-ref opts 'verbosity) + (parameterize ((%current-system (assoc-ref opts 'system)) + (%graft? (assoc-ref opts 'graft?))) + (with-build-handler (build-notifier #:use-substitutes? + substitutes? + #:verbosity + (assoc-ref opts 'verbos= ity) + #:dry-run? dry-run?) + (set-build-options-from-command-line store opts) + (ensure-default-profile) + (honor-x509-certificates store) + + (let* ((channels (channel-list opts)) + (instances + (latest-channel-instances store channels + #:current-channels + current-channels + #:validate-pull + validate-pull + #:authenticate? + authenticate?))) + (format (current-error-port) + (N_ "Building from this channel:~%" + "Building from these channels:~%" + (length instances))) + (for-each (lambda (instance) + (let ((channel + (channel-instance-channel instance))) + (format (current-error-port) + " ~10a~a\t~a~%" + (channel-name channel) + (channel-url channel) + (string-take + (channel-instance-commit instanc= e) + 7)))) + instances) + (parameterize ((%guile-for-build + (package-derivation + store + (if (assoc-ref opts 'bootstrap?) + %bootstrap-guile + (default-guile))))) + (with-profile-lock profile + (run-with-store store + (build-and-install instances profile)))))))))))))= )) =20 ;;; pull.scm ends here --=20 2.29.2 --=-=-=--