From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id kKGaONoIyGLQfAAAbAwnHQ (envelope-from ) for ; Fri, 08 Jul 2022 12:37:15 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id UPfLN9oIyGLtQAAAG6o9tA (envelope-from ) for ; Fri, 08 Jul 2022 12:37:14 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3AB1638B7B for ; Fri, 8 Jul 2022 12:37:14 +0200 (CEST) Received: from localhost ([::1]:60626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o9lMH-0004WJ-8r for larch@yhetil.org; Fri, 08 Jul 2022 06:37:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o9lM6-0004W0-Rb for bug-guix@gnu.org; Fri, 08 Jul 2022 06:37:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36800) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o9lM6-0000sZ-Jf for bug-guix@gnu.org; Fri, 08 Jul 2022 06:37:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o9lM6-00086V-HA for bug-guix@gnu.org; Fri, 08 Jul 2022 06:37:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#56441: guix time-machine broken by profiles speed-up Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 08 Jul 2022 10:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56441 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ricardo Wurmus Cc: 56441@debbugs.gnu.org, zimoun Received: via spool by 56441-submit@debbugs.gnu.org id=B56441.165727660831110 (code B ref 56441); Fri, 08 Jul 2022 10:37:02 +0000 Received: (at 56441) by debbugs.gnu.org; 8 Jul 2022 10:36:48 +0000 Received: from localhost ([127.0.0.1]:58929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o9lLr-00085e-Fx for submit@debbugs.gnu.org; Fri, 08 Jul 2022 06:36:48 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39968) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o9lLp-00085F-3N for 56441@debbugs.gnu.org; Fri, 08 Jul 2022 06:36:45 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52984) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o9lLj-0000qG-KS; Fri, 08 Jul 2022 06:36:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=yeY5En7qp2r0sYiE2RHW5mFwQSSaS6jXRA99DhxqHA8=; b=fuquK8nwCZjM+0I8cPv9 +5buKGyFFjWlDzuMyp6oEOb+Oefkg3yUoGq4ZdLdcApcTZm6PnVyJJ/RyBhPzzSPgUmFRw32MKabH CLBCh1/hmzULL5xNFM9nT/dXdPn9FGH4ZEsLs7M6MbfzRFXGj3cqjzsabmf06ry31wbm/11Vw0sII mMP6IARbOTxJvNeOD1uDrju+qKR1g2+4XnRl0Se73/9Hhr0/SIhc+F5mGl9ykDVyz4B5s5ZOHWloz D0r8dVjYoO4SoIVEdBVcq1OcmPSnwRjttq4236n4Ic6EeW830YquHJJBcvPOSFhCjVYQp6wKfxIl/ zMrwZBnC0K7q9g==; Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=58168 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o9lLj-00017O-4K; Fri, 08 Jul 2022 06:36:39 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <867d4oq3gr.fsf@gmail.com> <87wncozun2.fsf@elephly.net> <87sfncztuq.fsf@elephly.net> <87fsjcx9yc.fsf@gnu.org> Date: Fri, 08 Jul 2022 12:36:36 +0200 In-Reply-To: <87fsjcx9yc.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 08 Jul 2022 10:52:43 +0200") Message-ID: <87zghjx557.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1657276634; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=yeY5En7qp2r0sYiE2RHW5mFwQSSaS6jXRA99DhxqHA8=; b=pdPTYNKCnDkSC2aFJ3PwCxiPEHz5mOYIfpCZmEoQ0QcLekrXaYVFKbTvxhw2wC/Ec18ZZ9 /V6+O3jrTPHueFwnWNZHM3XZd9+e9XuNGnn4J1UX2jbu23Qfve1DJb1Xj1e+dmNIL30VlO L2tbZaBsCQLMxR9iQSJIr6PbWcMXb1g6TKZuelPn2RQm0ug1G6hXohK1o2eaxC8315GDo1 JB0xnMRpcYUTSUZfeqxFIXoWX45y+U9WKWCqNwE1g31Z1FuWoW0SLl3eId4PtIHEv2dD7M YWm7JAJIV6a6eGNJanQOcgAFheTAugn84QhTevpdaeW2TRhjXvwln3gvvFlyJg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1657276634; a=rsa-sha256; cv=none; b=IhLnyGk1RmMJCmjf/KZBwBZLPDA2JDWtDnhb/70ZKo+32/F4hnvr7cSJUsEPOHICY6lFXG WNnwUHUWgD1M0RrymHH23MNp7+TUkoTjqm/rOUtD122q9rW3gHPYVf9z/NsQrV0IPk6E41 IWPD4hUzq33Ci+e9G9Y23mtCg+a4cbZMKulEg9ZP6f1YmblSiBMQaudHaFjowy4ywTT4yz YuoA6qa99EgNm0D1UWfPIbHRmJGZz/e78is9Yd8Mq5pj+i9YLcbROQ45J6udd2UThll1Jf c82sD4ACWnI/mHt+22/wYvv8yfydbqrNOW1hYtYsEI3LNeZWf0/vSM2K+/RXNQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=fuquK8nw; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.64 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=fuquK8nw; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3AB1638B7B X-Spam-Score: -4.64 X-Migadu-Scanner: scn0.migadu.com X-TUID: q3/eSKSeKEpJ --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Ludovic Court=C3=A8s skribis: > Another option (thinking out loud): > > =E2=80=A2 in =E2=80=98package-cache-file=E2=80=99, unconditionally gene= rate a v3 profile (we > could add a =E2=80=98version=E2=80=99 field to etc.); > > =E2=80=A2 likewise in =E2=80=98channel-instances->derivation=E2=80=99. The patches below do that. As discussed on IRC, it=E2=80=99s not pretty but it=E2=80=99s pragmatic. Tested with: ./pre-inst-env guix time-machine \ --commit=3D85a5110de79f4fe9fd822ede3915654ee699d6c5 -- describe How does that sound? > Problem: it doesn=E2=80=99t address the case where you install the =E2=80= =98guix=E2=80=99 > package in a regular profile (I think?). I haven=E2=80=99t yet checked whether this is the case. Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-profiles-Support-the-creation-of-profiles-with-versi.patch Content-Transfer-Encoding: quoted-printable >From e1d7117cb4f81f4f590ff9c9a8fe14797cc210f1 Mon Sep 17 00:00:00 2001 From: =3D?UTF-8?q?Ludovic=3D20Court=3DC3=3DA8s?=3D Date: Fri, 8 Jul 2022 12:26:50 +0200 Subject: [PATCH 1/2] profiles: Support the creation of profiles with versio= n 3 manifests. * guix/profiles.scm (%manifest-format-version): New variable. (manifest->gexp): Add optional 'format-version' parameter. [optional, entry->gexp]: Honor it. (profile-derivation): Add #:format-version parameter and honor it. ()[format-version]: New field. (profile-compiler): Honor it. * guix/build/profiles.scm (manifest-sexp->inputs+search-paths): Support both versions 3 and 4. Remove unused 'properties' variable. * tests/profiles.scm ("profile-derivation format version 3"): New test. --- guix/build/profiles.scm | 6 +++--- guix/profiles.scm | 48 ++++++++++++++++++++++++++++++----------- tests/profiles.scm | 28 ++++++++++++++++++++++++ 3 files changed, 66 insertions(+), 16 deletions(-) diff --git a/guix/build/profiles.scm b/guix/build/profiles.scm index 2ab76bde74..0c92f222b4 100644 --- a/guix/build/profiles.scm +++ b/guix/build/profiles.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2015, 2017, 2018, 2019, 2020, 2021 Ludovic Court=C3= =A8s +;;; Copyright =C2=A9 2015, 2017-2022 Ludovic Court=C3=A8s ;;; ;;; This file is part of GNU Guix. ;;; @@ -162,7 +162,7 @@ (define-syntax let-fields (begin body ...)))) =20 (match manifest ;this must match 'manifest->g= exp' - (('manifest ('version 4) + (('manifest ('version (or 3 4)) ('packages (entries ...))) (let loop ((entries entries) (inputs '()) @@ -170,7 +170,7 @@ (define-syntax let-fields (match entries (((name version output item fields ...) . rest) (let ((paths search-paths)) - (let-fields fields (propagated-inputs search-paths properties) + (let-fields fields (propagated-inputs search-paths) (loop (append rest propagated-inputs) ;breadth-first travers= al (cons item inputs) (append search-paths paths))))) diff --git a/guix/profiles.scm b/guix/profiles.scm index a21cc432dc..d1dfa13e98 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -452,12 +452,23 @@ (define (inferior->entry) packages) manifest-entry=3D?))) =20 -(define (manifest->gexp manifest) - "Return a representation of MANIFEST as a gexp." +(define %manifest-format-version + ;; The current manifest format version. + 4) + +(define* (manifest->gexp manifest #:optional + (format-version %manifest-format-version)) + "Return a representation in FORMAT-VERSION of MANIFEST as a gexp." (define (optional name value) - (if (null? value) - #~() - #~((#$name #$value)))) + (match format-version + (4 + (if (null? value) + #~() + #~((#$name #$value)))) + (3 + (match name + ('properties #~((#$name #$@value))) + (_ #~((#$name #$value))))))) =20 (define (entry->gexp entry) ;; Maintain in state monad a vhash of visited entries, indexed by their @@ -467,10 +478,11 @@ (define (entry->gexp entry) ;; the presence of propagated inputs, where we could otherwise end up ;; repeating large trees. (mlet %state-monad ((visited (current-state))) - (if (match (vhash-assq (manifest-entry-item entry) visited) - ((_ . previous-entry) - (manifest-entry=3D? previous-entry entry)) - (#f #f)) + (if (and (=3D format-version 4) + (match (vhash-assq (manifest-entry-item entry) visited) + ((_ . previous-entry) + (manifest-entry=3D? previous-entry entry)) + (#f #f))) (return #~(repeated #$(manifest-entry-name entry) #$(manifest-entry-version entry) (ungexp (manifest-entry-item entry) @@ -500,9 +512,14 @@ (define (entry->gexp entry) search-paths)) #$@(optional 'properties properties)))))))))) =20 + (unless (memq format-version '(3 4)) + (raise (formatted-message + (G_ "cannot emit manifests formatted as version ~a") + format-version))) + (match manifest (($ (entries ...)) - #~(manifest (version 4) + #~(manifest (version #$format-version) (packages #$(run-with-state (mapm %state-monad entry->gexp entries) vlist-null)))))) @@ -1883,6 +1900,7 @@ (define* (profile-derivation manifest (allow-unsupported-packages? #f) (allow-collisions? #f) (relative-symlinks? #f) + (format-version %manifest-format-version) system target) "Return a derivation that builds a profile (aka. 'user environment') with the given MANIFEST. The profile includes additional derivations returned = by @@ -1968,7 +1986,7 @@ (define builder =20 #+(if locales? set-utf8-locale #t) =20 - (build-profile #$output '#$(manifest->gexp manifest) + (build-profile #$output '#$(manifest->gexp manifest format-ver= sion) #:extra-inputs '#$extra-inputs #:symlink #$(if relative-symlinks? #~symlink-relative @@ -2007,19 +2025,23 @@ (define-record-type* profile make-profile (allow-collisions? profile-allow-collisions? ;Boolean (default #f)) (relative-symlinks? profile-relative-symlinks? ;Boolean - (default #f))) + (default #f)) + (format-version profile-format-version ;integer + (default %manifest-format-version))) =20 (define-gexp-compiler (profile-compiler (profile ) system target) "Compile PROFILE to a derivation." (match profile (($ name manifest hooks - locales? allow-collisions? relative-symlinks?) + locales? allow-collisions? relative-symlinks? + format-version) (profile-derivation manifest #:name name #:hooks hooks #:locales? locales? #:allow-collisions? allow-collisions? #:relative-symlinks? relative-symlinks? + #:format-version format-version #:system system #:target target)))) =20 (define* (profile-search-paths profile diff --git a/tests/profiles.scm b/tests/profiles.scm index f002dfc5e4..7bed946bf3 100644 --- a/tests/profiles.scm +++ b/tests/profiles.scm @@ -286,6 +286,34 @@ (define transform1 (string=3D? (dirname (readlink bindir)) (derivation->output-path guile)))))) =20 +(test-assertm "profile-derivation format version 3" + ;; Make sure we can create and read a version 3 manifest. + (mlet* %store-monad + ((entry -> (package->manifest-entry %bootstrap-guile + #:properties '((answer . 42))= )) + (manifest -> (manifest (list entry))) + (drv1 (profile-derivation manifest + #:format-version 3 ;old version + #:hooks '() + #:locales? #f)) + (drv2 (profile-derivation manifest + #:hooks '() + #:locales? #f)) + (profile1 -> (derivation->output-path drv1)) + (profile2 -> (derivation->output-path drv2)) + (_ (built-derivations (list drv1 drv2)))) + (return (let ((manifest1 (profile-manifest profile1)) + (manifest2 (profile-manifest profile2))) + (match (manifest-entries manifest1) + ((entry1) + (match (manifest-entries manifest2) + ((entry2) + (and (manifest-entry=3D? entry1 entry2) + (equal? (manifest-entry-properties entry1) + '((answer . 42))) + (equal? (manifest-entry-properties entry2) + '((answer . 42)))))))))))) + (test-assertm "profile-derivation, ordering & collisions" ;; ENTRY1 and ENTRY2 both provide 'bin/guile'--a collision. Make sure ;; ENTRY1 "wins" over ENTRY2. See . base-commit: 33179b180a5b02b730b4b37369ccf48204597940 --=20 2.36.1 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0002-channels-Emit-version-3-profiles.patch Content-Transfer-Encoding: quoted-printable >From d404ef913482c6a9c692bad2142cf7202ebc1cc4 Mon Sep 17 00:00:00 2001 From: =3D?UTF-8?q?Ludovic=3D20Court=3DC3=3DA8s?=3D Date: Fri, 8 Jul 2022 12:31:25 +0200 Subject: [PATCH 2/2] channels: Emit version 3 profiles. Fixes . Reported by zimoun . * guix/channels.scm (package-cache-file): Add 'format-version' field to PROFILE. (channel-instances->derivation): Pass #:format-version to 'profile-derivation'. --- guix/channels.scm | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/guix/channels.scm b/guix/channels.scm index ce1a60436f..689b30e0eb 100644 --- a/guix/channels.scm +++ b/guix/channels.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2018, 2019, 2020, 2021 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2018-2022 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2018 Ricardo Wurmus ;;; Copyright =C2=A9 2019 Jan (janneke) Nieuwenhuizen ;;; Copyright =C2=A9 2021 Brice Waegeneire @@ -896,7 +896,12 @@ (define (instance->entry instance drv) (define (package-cache-file manifest) "Build a package cache file for the instance in MANIFEST. This is meant= to be used as a profile hook." - (let ((profile (profile (content manifest) (hooks '())))) + ;; Note: Emit a profile in format version 3, which was introduced in 2017 + ;; and is readable by Guix since before version 1.0. This ensures that = the + ;; Guix in MANIFEST is able to read the manifest file created for its own + ;; profile below. See . + (let ((profile (profile (content manifest) (hooks '()) + (format-version 3)))) (define build #~(begin (use-modules (gnu packages)) @@ -937,8 +942,12 @@ (define (channel-instances->derivation instances) "Return the derivation of the profile containing INSTANCES, a list of channel instances." (mlet %store-monad ((manifest (channel-instances->manifest instances))) + ;; Emit a profile in format version so that, if INSTANCES denotes an o= ld + ;; Guix, it can still read that profile, for instance for the purposes= of + ;; 'guix describe'. (profile-derivation manifest - #:hooks %channel-profile-hooks))) + #:hooks %channel-profile-hooks + #:format-version 3))) =20 (define latest-channel-instances* (store-lift latest-channel-instances)) --=20 2.36.1 --=-=-=--