* [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest
@ 2024-12-02 16:51 Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory Ludovic Courtès
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:51 UTC (permalink / raw)
To: 74654; +Cc: Ludovic Courtès
Hello Guix!
This is a followup to <https://issues.guix.gnu.org/74542>:
• Optimizing ‘all-packages’ significantly (speeds up
‘guix build -P’ and ‘guix refresh -l’);
• Moving manifests to ‘etc/manifests’ (we’ll have to
update Cuirass specs that refer to these files);
• Adding an “ungrafting” package that rebuilds everything
without grafts, just like what we’d do manually.
The goal is to have that ungrafting manifest under CI, such
that anytime the corresponding jobset is green, we can ungraft
everything with the guarantee that binaries are available.
Thoughts?
Ludo’.
Ludovic Courtès (5):
etc: Move manifests to a separate directory.
packages: Optimize ‘all-packages’.
build-system/cargo: Simplify ‘crate-closure’.
guix build: Last argument of ‘dependents’ is optional.
maint: Add ungrafting manifest.
Makefile.am | 21 ++++----
.../disarchive.scm} | 2 +-
etc/{hurd-manifest.scm => manifests/hurd.scm} | 0
.../kernels.scm} | 0
.../release.scm} | 0
.../source.scm} | 0
etc/{ => manifests}/system-tests.scm | 0
.../time-travel.scm} | 0
etc/manifests/ungraft.scm | 49 +++++++++++++++++++
.../upgrade.scm} | 0
gnu/packages.scm | 30 +++++++-----
guix/build-system/cargo.scm | 12 ++---
guix/scripts/build.scm | 4 +-
13 files changed, 87 insertions(+), 31 deletions(-)
rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
rename etc/{release-manifest.scm => manifests/release.scm} (100%)
rename etc/{source-manifest.scm => manifests/source.scm} (100%)
rename etc/{ => manifests}/system-tests.scm (100%)
rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
create mode 100644 etc/manifests/ungraft.scm
rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)
base-commit: 522732d5c15e44fc9e061f36a41f7129edfee66f
--
2.46.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory.
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
@ 2024-12-02 16:53 ` Ludovic Courtès
2024-12-03 3:05 ` Z572
2024-12-02 16:53 ` [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’ Ludovic Courtès
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:53 UTC (permalink / raw)
To: 74654; +Cc: Ludovic Courtès
* etc/disarchive-manifest.scm, etc/hurd-manifest.scm,
etc/kernels-manifest.scm, etc/release-manifest.scm,
etc/source-manifest.scm, etc/system-tests.scm,
etc/time-travel-manifest.scm, etc/upgrade-manifest.scm: Move to…
* etc/manifests: … here, and drop “-manifest” from file name.
* Makefile.am (EXTRA_DIST, assert-binaries-available, check-system):
Adjust accordingly.
Change-Id: Iedee3d0cdd42e72ef8bbf654ea5d3b47dca95874
---
Makefile.am | 20 +++++++++----------
.../disarchive.scm} | 2 +-
etc/{hurd-manifest.scm => manifests/hurd.scm} | 0
.../kernels.scm} | 0
.../release.scm} | 0
.../source.scm} | 0
etc/{ => manifests}/system-tests.scm | 0
.../time-travel.scm} | 0
.../upgrade.scm} | 0
9 files changed, 11 insertions(+), 11 deletions(-)
rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
rename etc/{release-manifest.scm => manifests/release.scm} (100%)
rename etc/{source-manifest.scm => manifests/source.scm} (100%)
rename etc/{ => manifests}/system-tests.scm (100%)
rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)
diff --git a/Makefile.am b/Makefile.am
index 0cff32c607..8c763c68aa 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -686,7 +686,7 @@ endif !CAN_RUN_TESTS
check-system: $(GOBJECTS)
$(AM_V_at)$(top_builddir)/pre-inst-env \
- guix build -m $(top_srcdir)/etc/system-tests.scm -K
+ guix build -m $(top_srcdir)/etc/manifests/system-tests.scm -K
# Public keys used to sign substitutes.
dist_pkgdata_DATA = \
@@ -733,17 +733,17 @@ EXTRA_DIST += \
build-aux/update-guix-package.scm \
build-aux/xgettext.scm \
doc/build.scm \
- etc/disarchive-manifest.scm \
etc/guix-install.sh \
etc/historical-authorizations \
etc/news.scm \
- etc/hurd-manifest.scm \
- etc/kernels-manifest.scm \
- etc/release-manifest.scm \
- etc/source-manifest.scm \
- etc/system-tests.scm \
- etc/time-travel-manifest.scm \
- etc/upgrade-manifest.scm \
+ etc/manifests/disarchive.scm \
+ etc/manifests/hurd.scm \
+ etc/manifests/kernels.scm \
+ etc/manifests/release.scm \
+ etc/manifests/source.scm \
+ etc/manifests/system-tests.scm \
+ etc/manifests/time-travel.scm \
+ etc/manifests/upgrade.scm \
scripts/guix.in \
tests/cve-sample.json \
tests/keys/civodul.pub \
@@ -1194,7 +1194,7 @@ assert-no-store-file-names:
# server so that '--display-missing' doesn't print two lists.
assert-binaries-available: $(GOBJECTS)
$(AM_V_at)$(top_builddir)/pre-inst-env \
- guix weather -m "$(top_srcdir)/etc/release-manifest.scm" \
+ guix weather -m "$(top_srcdir)/etc/manifests/release.scm" \
--substitute-urls="https://ci.guix.gnu.org" \
--display-missing
diff --git a/etc/disarchive-manifest.scm b/etc/manifests/disarchive.scm
similarity index 99%
rename from etc/disarchive-manifest.scm
rename to etc/manifests/disarchive.scm
index 3dbfa356df..a7f71414b6 100644
--- a/etc/disarchive-manifest.scm
+++ b/etc/manifests/disarchive.scm
@@ -24,7 +24,7 @@
(guix base16)
(gnu packages))
-(include "source-manifest.scm")
+(include "source.scm")
(define (tarball-origin? origin)
(match (origin-actual-file-name origin)
diff --git a/etc/hurd-manifest.scm b/etc/manifests/hurd.scm
similarity index 100%
rename from etc/hurd-manifest.scm
rename to etc/manifests/hurd.scm
diff --git a/etc/kernels-manifest.scm b/etc/manifests/kernels.scm
similarity index 100%
rename from etc/kernels-manifest.scm
rename to etc/manifests/kernels.scm
diff --git a/etc/release-manifest.scm b/etc/manifests/release.scm
similarity index 100%
rename from etc/release-manifest.scm
rename to etc/manifests/release.scm
diff --git a/etc/source-manifest.scm b/etc/manifests/source.scm
similarity index 100%
rename from etc/source-manifest.scm
rename to etc/manifests/source.scm
diff --git a/etc/system-tests.scm b/etc/manifests/system-tests.scm
similarity index 100%
rename from etc/system-tests.scm
rename to etc/manifests/system-tests.scm
diff --git a/etc/time-travel-manifest.scm b/etc/manifests/time-travel.scm
similarity index 100%
rename from etc/time-travel-manifest.scm
rename to etc/manifests/time-travel.scm
diff --git a/etc/upgrade-manifest.scm b/etc/manifests/upgrade.scm
similarity index 100%
rename from etc/upgrade-manifest.scm
rename to etc/manifests/upgrade.scm
--
2.46.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’.
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory Ludovic Courtès
@ 2024-12-02 16:53 ` Ludovic Courtès
2024-12-02 23:47 ` David Elsing
2024-12-02 16:53 ` [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’ Ludovic Courtès
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:53 UTC (permalink / raw)
To: 74654; +Cc: Ludovic Courtès
On my laptop, wall-clock time for (all-packages) goes from 27s to 1s.
* gnu/packages.scm (all-packages): Use a hash table to remember visited
packages instead of calling ‘delete-duplicates’ on the final list.
Change-Id: I4aae804656b56ef2095993e91f0572a5891f419f
---
gnu/packages.scm | 30 +++++++++++++++++++-----------
1 file changed, 19 insertions(+), 11 deletions(-)
diff --git a/gnu/packages.scm b/gnu/packages.scm
index 1af3b8d440..bdd5d21940 100644
--- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -258,18 +258,26 @@ (define all-packages
(mlambda ()
"Return the list of all public packages, including replacements and hidden
packages, excluding superseded packages."
- (delete-duplicates
- (fold-packages (lambda (package result)
- (match (package-replacement package)
- ((? package? replacement)
- (cons* replacement package result))
- (#f
- (cons package result))))
- '()
+ ;; Note: 'fold-packages' never traverses the same package twice but
+ ;; replacements break that (they may or may not be visible to
+ ;; 'fold-packages'), hence this hash table to track visited packages.
+ (define visited (make-hash-table))
- ;; Dismiss deprecated packages but keep hidden packages.
- #:select? (negate package-superseded))
- eq?)))
+ (fold-packages (lambda (package result)
+ (if (hashq-ref visited package)
+ result
+ (begin
+ (hashq-set! visited package #t)
+ (match (package-replacement package)
+ ((? package? replacement)
+ (hashq-set! visited replacement #t)
+ (cons* replacement package result))
+ (#f
+ (cons package result))))))
+ '()
+
+ ;; Dismiss deprecated packages but keep hidden packages.
+ #:select? (negate package-superseded))))
(define %package-cache-file
;; Location of the package cache.
--
2.46.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’ Ludovic Courtès
@ 2024-12-02 16:53 ` Ludovic Courtès
2024-12-02 19:11 ` Efraim Flashner
2024-12-02 16:53 ` [bug#74654] [PATCH 4/5] guix build: Last argument of ‘dependents’ is optional Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 5/5] maint: Add ungrafting manifest Ludovic Courtès
4 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:53 UTC (permalink / raw)
To: 74654; +Cc: Ludovic Courtès, Efraim Flashner
* guix/build-system/cargo.scm (crate-closure): Remove ‘first?’.
Change-Id: Ica1a661eb422a882ae3a1a54819b2dd7d68449de
---
guix/build-system/cargo.scm | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 658a2e525e..0e9a4b1d23 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013-2016, 2019, 2021, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
@@ -227,24 +227,22 @@ (define (crate-closure inputs)
(let loop ((inputs inputs)
(result '())
(propagated '())
- (first? #t)
(seen vlist-null))
(match inputs
(()
(if (null? propagated)
(reverse result)
- (loop (reverse (concatenate propagated)) result '() #f seen)))
+ (loop (reverse (concatenate propagated)) result '() seen)))
(((and input (label (? package? package))) rest ...)
- (if (and (not first?) (seen? seen package))
- (loop rest result propagated first? seen)
+ (if (seen? seen package)
+ (loop rest result propagated seen)
(loop rest
(cons input result)
(cons (package-cargo-inputs package)
propagated)
- first?
(vhash-consq package package seen))))
((input rest ...)
- (loop rest (cons input result) propagated first? seen)))))
+ (loop rest (cons input result) propagated seen)))))
(define (expand-crate-sources cargo-inputs cargo-development-inputs)
"Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
--
2.46.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 4/5] guix build: Last argument of ‘dependents’ is optional.
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
` (2 preceding siblings ...)
2024-12-02 16:53 ` [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’ Ludovic Courtès
@ 2024-12-02 16:53 ` Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 5/5] maint: Add ungrafting manifest Ludovic Courtès
4 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:53 UTC (permalink / raw)
To: 74654
Cc: Ludovic Courtès, Christopher Baines, Josselin Poiret,
Ludovic Courtès, Mathieu Othacehe, Simon Tournier,
Tobias Geerinckx-Rice
* guix/scripts/build.scm (dependents): Make ‘max-depth’ optional.
Improve docstring.
Change-Id: Ibeffcf15297ecba3d4000b3adb4e3bc7853f6957
---
guix/scripts/build.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 0080f1fe58..f0a637a2ef 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -564,8 +564,8 @@ (define %options
%standard-cross-build-options
%standard-native-build-options)))
-(define (dependents store packages max-depth)
- "List all the things that would need to be rebuilt if PACKAGES are changed."
+(define* (dependents store packages #:optional (max-depth +inf.0))
+ "Return the list of dependents of all of PACKAGES up to distance MAX-DEPTH."
;; Using %BAG-NODE-TYPE is more accurate than using %PACKAGE-NODE-TYPE
;; because it includes implicit dependencies.
(define (get-dependents packages edges)
--
2.46.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 5/5] maint: Add ungrafting manifest.
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
` (3 preceding siblings ...)
2024-12-02 16:53 ` [bug#74654] [PATCH 4/5] guix build: Last argument of ‘dependents’ is optional Ludovic Courtès
@ 2024-12-02 16:53 ` Ludovic Courtès
4 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 16:53 UTC (permalink / raw)
To: 74654; +Cc: Ludovic Courtès
* etc/manifests/ungraft.scm: New file.
* Makefile.am (EXTRA_DIST): Add it.
Change-Id: If4aa7649721b93d3297ecd967d5db4a6b71ec6eb
---
Makefile.am | 1 +
etc/manifests/ungraft.scm | 49 +++++++++++++++++++++++++++++++++++++++
2 files changed, 50 insertions(+)
create mode 100644 etc/manifests/ungraft.scm
diff --git a/Makefile.am b/Makefile.am
index 8c763c68aa..d0d195636b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -743,6 +743,7 @@ EXTRA_DIST += \
etc/manifests/source.scm \
etc/manifests/system-tests.scm \
etc/manifests/time-travel.scm \
+ etc/manifests/ungraft.scm \
etc/manifests/upgrade.scm \
scripts/guix.in \
tests/cve-sample.json \
diff --git a/etc/manifests/ungraft.scm b/etc/manifests/ungraft.scm
new file mode 100644
index 0000000000..3e42b98ece
--- /dev/null
+++ b/etc/manifests/ungraft.scm
@@ -0,0 +1,49 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+;; This manifest "ungrafts" all the currently grafted packages and returns
+;; said packages and all their dependents.
+
+(use-modules (guix diagnostics)
+ (guix i18n)
+ (guix packages)
+ (guix profiles)
+ (guix store)
+ ((guix scripts build) #:select (dependents))
+ ((gnu packages) #:select (all-packages))
+ (srfi srfi-1))
+
+(define (grafted-packages)
+ (info (G_ "enumerating grafted packages...~%"))
+ (let ((result (filter package-replacement (all-packages))))
+ (info (G_ "found ~d grafted packages:~{ ~a~}~%")
+ (length result) (map package-full-name result))
+ result))
+
+(manifest
+ (with-store store
+ (let* ((grafted (grafted-packages))
+ (ungraft-all (package-input-rewriting
+ (map (lambda (package)
+ `(,package . ,(package-replacement package)))
+ grafted))))
+ (map (lambda (package)
+ (manifest-entry
+ (inherit (package->manifest-entry (ungraft-all package)))
+ (name (string-append (package-name package) "-ungrafted"))))
+ (dependents store grafted)))))
--
2.46.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.
2024-12-02 16:53 ` [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’ Ludovic Courtès
@ 2024-12-02 19:11 ` Efraim Flashner
2024-12-02 22:24 ` Ludovic Courtès
0 siblings, 1 reply; 10+ messages in thread
From: Efraim Flashner @ 2024-12-02 19:11 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 74654
[-- Attachment #1: Type: text/plain, Size: 2701 bytes --]
I test applied this on master and then went to build my rusty-packages
manifest. The time on './pre-inst-env guix build -m ... -n' went from 35
seconds to 20 seconds and the derivations didn't change.
On Mon, Dec 02, 2024 at 05:53:28PM +0100, Ludovic Courtès wrote:
> * guix/build-system/cargo.scm (crate-closure): Remove ‘first?’.
>
> Change-Id: Ica1a661eb422a882ae3a1a54819b2dd7d68449de
> ---
> guix/build-system/cargo.scm | 12 +++++-------
> 1 file changed, 5 insertions(+), 7 deletions(-)
>
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 658a2e525e..0e9a4b1d23 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -1,5 +1,5 @@
> ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2021 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2013-2016, 2019, 2021, 2024 Ludovic Courtès <ludo@gnu.org>
> ;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
> ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
> ;;; Copyright © 2016 David Craven <david@craven.ch>
> @@ -227,24 +227,22 @@ (define (crate-closure inputs)
> (let loop ((inputs inputs)
> (result '())
> (propagated '())
> - (first? #t)
> (seen vlist-null))
> (match inputs
> (()
> (if (null? propagated)
> (reverse result)
> - (loop (reverse (concatenate propagated)) result '() #f seen)))
> + (loop (reverse (concatenate propagated)) result '() seen)))
> (((and input (label (? package? package))) rest ...)
> - (if (and (not first?) (seen? seen package))
> - (loop rest result propagated first? seen)
> + (if (seen? seen package)
> + (loop rest result propagated seen)
> (loop rest
> (cons input result)
> (cons (package-cargo-inputs package)
> propagated)
> - first?
> (vhash-consq package package seen))))
> ((input rest ...)
> - (loop rest (cons input result) propagated first? seen)))))
> + (loop rest (cons input result) propagated seen)))))
>
> (define (expand-crate-sources cargo-inputs cargo-development-inputs)
> "Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
> --
> 2.46.0
>
>
>
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.
2024-12-02 19:11 ` Efraim Flashner
@ 2024-12-02 22:24 ` Ludovic Courtès
0 siblings, 0 replies; 10+ messages in thread
From: Ludovic Courtès @ 2024-12-02 22:24 UTC (permalink / raw)
To: Efraim Flashner; +Cc: 74654
Efraim Flashner <efraim@flashner.co.il> skribis:
> I test applied this on master and then went to build my rusty-packages
> manifest. The time on './pre-inst-env guix build -m ... -n' went from 35
> seconds to 20 seconds and the derivations didn't change.
Thanks for testing! I didn’t expect this to have an impact on
performance, interesting.
Ludo’.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’.
2024-12-02 16:53 ` [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’ Ludovic Courtès
@ 2024-12-02 23:47 ` David Elsing
0 siblings, 0 replies; 10+ messages in thread
From: David Elsing @ 2024-12-02 23:47 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 74654
Hi,
Ludovic Courtès <ludo@gnu.org> writes:
> On my laptop, wall-clock time for (all-packages) goes from 27s to 1s.
>
> * gnu/packages.scm (all-packages): Use a hash table to remember visited
> packages instead of calling ‘delete-duplicates’ on the final list.
Thanks for fixing this, commit 4b5dae8 also made
`guix graph -t reverse-package` much slower, because `delete-duplicates`
had not been used in the old `all-packages` procedure before. I wanted
to make a patch, but then saw your series already. :)
Cheers,
David
^ permalink raw reply [flat|nested] 10+ messages in thread
* [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory.
2024-12-02 16:53 ` [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory Ludovic Courtès
@ 2024-12-03 3:05 ` Z572
0 siblings, 0 replies; 10+ messages in thread
From: Z572 @ 2024-12-03 3:05 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 74654
[-- Attachment #1: Type: text/plain, Size: 5384 bytes --]
Ludovic Courtès <ludo@gnu.org> writes:
> * etc/disarchive-manifest.scm, etc/hurd-manifest.scm,
> etc/kernels-manifest.scm, etc/release-manifest.scm,
> etc/source-manifest.scm, etc/system-tests.scm,
> etc/time-travel-manifest.scm, etc/upgrade-manifest.scm: Move to…
> * etc/manifests: … here, and drop “-manifest” from file name.
> * Makefile.am (EXTRA_DIST, assert-binaries-available, check-system):
> Adjust accordingly.
Once these files are moved, it may be necessary to mention them in
etc/news.scm to inform others that they need to update their cuirass
specs.
>
> Change-Id: Iedee3d0cdd42e72ef8bbf654ea5d3b47dca95874
> ---
> Makefile.am | 20 +++++++++----------
> .../disarchive.scm} | 2 +-
> etc/{hurd-manifest.scm => manifests/hurd.scm} | 0
> .../kernels.scm} | 0
> .../release.scm} | 0
> .../source.scm} | 0
> etc/{ => manifests}/system-tests.scm | 0
> .../time-travel.scm} | 0
> .../upgrade.scm} | 0
> 9 files changed, 11 insertions(+), 11 deletions(-)
> rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
> rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
> rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
> rename etc/{release-manifest.scm => manifests/release.scm} (100%)
> rename etc/{source-manifest.scm => manifests/source.scm} (100%)
> rename etc/{ => manifests}/system-tests.scm (100%)
> rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
> rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)
>
> diff --git a/Makefile.am b/Makefile.am
> index 0cff32c607..8c763c68aa 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -686,7 +686,7 @@ endif !CAN_RUN_TESTS
>
> check-system: $(GOBJECTS)
> $(AM_V_at)$(top_builddir)/pre-inst-env \
> - guix build -m $(top_srcdir)/etc/system-tests.scm -K
> + guix build -m $(top_srcdir)/etc/manifests/system-tests.scm -K
>
> # Public keys used to sign substitutes.
> dist_pkgdata_DATA = \
> @@ -733,17 +733,17 @@ EXTRA_DIST += \
> build-aux/update-guix-package.scm \
> build-aux/xgettext.scm \
> doc/build.scm \
> - etc/disarchive-manifest.scm \
> etc/guix-install.sh \
> etc/historical-authorizations \
> etc/news.scm \
> - etc/hurd-manifest.scm \
> - etc/kernels-manifest.scm \
> - etc/release-manifest.scm \
> - etc/source-manifest.scm \
> - etc/system-tests.scm \
> - etc/time-travel-manifest.scm \
> - etc/upgrade-manifest.scm \
> + etc/manifests/disarchive.scm \
> + etc/manifests/hurd.scm \
> + etc/manifests/kernels.scm \
> + etc/manifests/release.scm \
> + etc/manifests/source.scm \
> + etc/manifests/system-tests.scm \
> + etc/manifests/time-travel.scm \
> + etc/manifests/upgrade.scm \
> scripts/guix.in \
> tests/cve-sample.json \
> tests/keys/civodul.pub \
> @@ -1194,7 +1194,7 @@ assert-no-store-file-names:
> # server so that '--display-missing' doesn't print two lists.
> assert-binaries-available: $(GOBJECTS)
> $(AM_V_at)$(top_builddir)/pre-inst-env \
> - guix weather -m "$(top_srcdir)/etc/release-manifest.scm" \
> + guix weather -m "$(top_srcdir)/etc/manifests/release.scm" \
> --substitute-urls="https://ci.guix.gnu.org" \
> --display-missing
>
> diff --git a/etc/disarchive-manifest.scm b/etc/manifests/disarchive.scm
> similarity index 99%
> rename from etc/disarchive-manifest.scm
> rename to etc/manifests/disarchive.scm
> index 3dbfa356df..a7f71414b6 100644
> --- a/etc/disarchive-manifest.scm
> +++ b/etc/manifests/disarchive.scm
> @@ -24,7 +24,7 @@
> (guix base16)
> (gnu packages))
>
> -(include "source-manifest.scm")
> +(include "source.scm")
>
> (define (tarball-origin? origin)
> (match (origin-actual-file-name origin)
> diff --git a/etc/hurd-manifest.scm b/etc/manifests/hurd.scm
> similarity index 100%
> rename from etc/hurd-manifest.scm
> rename to etc/manifests/hurd.scm
> diff --git a/etc/kernels-manifest.scm b/etc/manifests/kernels.scm
> similarity index 100%
> rename from etc/kernels-manifest.scm
> rename to etc/manifests/kernels.scm
> diff --git a/etc/release-manifest.scm b/etc/manifests/release.scm
> similarity index 100%
> rename from etc/release-manifest.scm
> rename to etc/manifests/release.scm
> diff --git a/etc/source-manifest.scm b/etc/manifests/source.scm
> similarity index 100%
> rename from etc/source-manifest.scm
> rename to etc/manifests/source.scm
> diff --git a/etc/system-tests.scm b/etc/manifests/system-tests.scm
> similarity index 100%
> rename from etc/system-tests.scm
> rename to etc/manifests/system-tests.scm
> diff --git a/etc/time-travel-manifest.scm b/etc/manifests/time-travel.scm
> similarity index 100%
> rename from etc/time-travel-manifest.scm
> rename to etc/manifests/time-travel.scm
> diff --git a/etc/upgrade-manifest.scm b/etc/manifests/upgrade.scm
> similarity index 100%
> rename from etc/upgrade-manifest.scm
> rename to etc/manifests/upgrade.scm
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2024-12-03 3:06 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-12-02 16:51 [bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory Ludovic Courtès
2024-12-03 3:05 ` Z572
2024-12-02 16:53 ` [bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’ Ludovic Courtès
2024-12-02 23:47 ` David Elsing
2024-12-02 16:53 ` [bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’ Ludovic Courtès
2024-12-02 19:11 ` Efraim Flashner
2024-12-02 22:24 ` Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 4/5] guix build: Last argument of ‘dependents’ is optional Ludovic Courtès
2024-12-02 16:53 ` [bug#74654] [PATCH 5/5] maint: Add ungrafting manifest Ludovic Courtès
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.