[bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest

[bug#74654] [PATCH 0/5] Optimize 'all-packages'; add ungrafting manifest

[Ludovic Courtès]

Hello Guix!

This is a followup to <https://issues.guix.gnu.org/74542>:

  • Optimizing ‘all-packages’ significantly (speeds up
    ‘guix build -P’ and ‘guix refresh -l’);

  • Moving manifests to ‘etc/manifests’ (we’ll have to
    update Cuirass specs that refer to these files);

  • Adding an “ungrafting” package that rebuilds everything
    without grafts, just like what we’d do manually.

The goal is to have that ungrafting manifest under CI, such
that anytime the corresponding jobset is green, we can ungraft
everything with the guarantee that binaries are available.

Thoughts?

Ludo’.

Ludovic Courtès (5):
  etc: Move manifests to a separate directory.
  packages: Optimize ‘all-packages’.
  build-system/cargo: Simplify ‘crate-closure’.
  guix build: Last argument of ‘dependents’ is optional.
  maint: Add ungrafting manifest.

 Makefile.am                                   | 21 ++++----
 .../disarchive.scm}                           |  2 +-
 etc/{hurd-manifest.scm => manifests/hurd.scm} |  0
 .../kernels.scm}                              |  0
 .../release.scm}                              |  0
 .../source.scm}                               |  0
 etc/{ => manifests}/system-tests.scm          |  0
 .../time-travel.scm}                          |  0
 etc/manifests/ungraft.scm                     | 49 +++++++++++++++++++
 .../upgrade.scm}                              |  0
 gnu/packages.scm                              | 30 +++++++-----
 guix/build-system/cargo.scm                   | 12 ++---
 guix/scripts/build.scm                        |  4 +-
 13 files changed, 87 insertions(+), 31 deletions(-)
 rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
 rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
 rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
 rename etc/{release-manifest.scm => manifests/release.scm} (100%)
 rename etc/{source-manifest.scm => manifests/source.scm} (100%)
 rename etc/{ => manifests}/system-tests.scm (100%)
 rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
 create mode 100644 etc/manifests/ungraft.scm
 rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)


base-commit: 522732d5c15e44fc9e061f36a41f7129edfee66f
-- 
2.46.0

[bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory.

[Ludovic Courtès]

* etc/disarchive-manifest.scm, etc/hurd-manifest.scm,
etc/kernels-manifest.scm, etc/release-manifest.scm,
etc/source-manifest.scm, etc/system-tests.scm,
etc/time-travel-manifest.scm, etc/upgrade-manifest.scm: Move to…
* etc/manifests: … here, and drop “-manifest” from file name.
* Makefile.am (EXTRA_DIST, assert-binaries-available, check-system):
Adjust accordingly.

Change-Id: Iedee3d0cdd42e72ef8bbf654ea5d3b47dca95874
---
 Makefile.am                                   | 20 +++++++++----------
 .../disarchive.scm}                           |  2 +-
 etc/{hurd-manifest.scm => manifests/hurd.scm} |  0
 .../kernels.scm}                              |  0
 .../release.scm}                              |  0
 .../source.scm}                               |  0
 etc/{ => manifests}/system-tests.scm          |  0
 .../time-travel.scm}                          |  0
 .../upgrade.scm}                              |  0
 9 files changed, 11 insertions(+), 11 deletions(-)
 rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
 rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
 rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
 rename etc/{release-manifest.scm => manifests/release.scm} (100%)
 rename etc/{source-manifest.scm => manifests/source.scm} (100%)
 rename etc/{ => manifests}/system-tests.scm (100%)
 rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
 rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)

diff --git a/Makefile.am b/Makefile.am
index 0cff32c607..8c763c68aa 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -686,7 +686,7 @@ endif !CAN_RUN_TESTS
 
 check-system: $(GOBJECTS)
 	$(AM_V_at)$(top_builddir)/pre-inst-env			\
-	  guix build -m $(top_srcdir)/etc/system-tests.scm -K
+	  guix build -m $(top_srcdir)/etc/manifests/system-tests.scm -K
 
 # Public keys used to sign substitutes.
 dist_pkgdata_DATA =				\
@@ -733,17 +733,17 @@ EXTRA_DIST +=						\
   build-aux/update-guix-package.scm			\
   build-aux/xgettext.scm				\
   doc/build.scm						\
-  etc/disarchive-manifest.scm				\
   etc/guix-install.sh					\
   etc/historical-authorizations				\
   etc/news.scm						\
-  etc/hurd-manifest.scm					\
-  etc/kernels-manifest.scm				\
-  etc/release-manifest.scm				\
-  etc/source-manifest.scm				\
-  etc/system-tests.scm					\
-  etc/time-travel-manifest.scm				\
-  etc/upgrade-manifest.scm				\
+  etc/manifests/disarchive.scm				\
+  etc/manifests/hurd.scm				\
+  etc/manifests/kernels.scm				\
+  etc/manifests/release.scm				\
+  etc/manifests/source.scm				\
+  etc/manifests/system-tests.scm			\
+  etc/manifests/time-travel.scm				\
+  etc/manifests/upgrade.scm				\
   scripts/guix.in					\
   tests/cve-sample.json					\
   tests/keys/civodul.pub				\
@@ -1194,7 +1194,7 @@ assert-no-store-file-names:
 # server so that '--display-missing' doesn't print two lists.
 assert-binaries-available: $(GOBJECTS)
 	$(AM_V_at)$(top_builddir)/pre-inst-env				\
-	  guix weather -m "$(top_srcdir)/etc/release-manifest.scm"	\
+	  guix weather -m "$(top_srcdir)/etc/manifests/release.scm"	\
 	                --substitute-urls="https://ci.guix.gnu.org"	\
 	                --display-missing
 
diff --git a/etc/disarchive-manifest.scm b/etc/manifests/disarchive.scm
similarity index 99%
rename from etc/disarchive-manifest.scm
rename to etc/manifests/disarchive.scm
index 3dbfa356df..a7f71414b6 100644
--- a/etc/disarchive-manifest.scm
+++ b/etc/manifests/disarchive.scm
@@ -24,7 +24,7 @@
              (guix base16)
              (gnu packages))
 
-(include "source-manifest.scm")
+(include "source.scm")
 
 (define (tarball-origin? origin)
   (match (origin-actual-file-name origin)
diff --git a/etc/hurd-manifest.scm b/etc/manifests/hurd.scm
similarity index 100%
rename from etc/hurd-manifest.scm
rename to etc/manifests/hurd.scm
diff --git a/etc/kernels-manifest.scm b/etc/manifests/kernels.scm
similarity index 100%
rename from etc/kernels-manifest.scm
rename to etc/manifests/kernels.scm
diff --git a/etc/release-manifest.scm b/etc/manifests/release.scm
similarity index 100%
rename from etc/release-manifest.scm
rename to etc/manifests/release.scm
diff --git a/etc/source-manifest.scm b/etc/manifests/source.scm
similarity index 100%
rename from etc/source-manifest.scm
rename to etc/manifests/source.scm
diff --git a/etc/system-tests.scm b/etc/manifests/system-tests.scm
similarity index 100%
rename from etc/system-tests.scm
rename to etc/manifests/system-tests.scm
diff --git a/etc/time-travel-manifest.scm b/etc/manifests/time-travel.scm
similarity index 100%
rename from etc/time-travel-manifest.scm
rename to etc/manifests/time-travel.scm
diff --git a/etc/upgrade-manifest.scm b/etc/manifests/upgrade.scm
similarity index 100%
rename from etc/upgrade-manifest.scm
rename to etc/manifests/upgrade.scm
-- 
2.46.0

[bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’.

[Ludovic Courtès]

On my laptop, wall-clock time for (all-packages) goes from 27s to 1s.

* gnu/packages.scm (all-packages): Use a hash table to remember visited
packages instead of calling ‘delete-duplicates’ on the final list.

Change-Id: I4aae804656b56ef2095993e91f0572a5891f419f
---
 gnu/packages.scm | 30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/gnu/packages.scm b/gnu/packages.scm
index 1af3b8d440..bdd5d21940 100644
--- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -258,18 +258,26 @@ (define all-packages
   (mlambda ()
     "Return the list of all public packages, including replacements and hidden
 packages, excluding superseded packages."
-    (delete-duplicates
-     (fold-packages (lambda (package result)
-                      (match (package-replacement package)
-                        ((? package? replacement)
-                         (cons* replacement package result))
-                        (#f
-                         (cons package result))))
-                    '()
+    ;; Note: 'fold-packages' never traverses the same package twice but
+    ;; replacements break that (they may or may not be visible to
+    ;; 'fold-packages'), hence this hash table to track visited packages.
+    (define visited (make-hash-table))
 
-                    ;; Dismiss deprecated packages but keep hidden packages.
-                    #:select? (negate package-superseded))
-     eq?)))
+    (fold-packages (lambda (package result)
+                     (if (hashq-ref visited package)
+                         result
+                         (begin
+                           (hashq-set! visited package #t)
+                           (match (package-replacement package)
+                             ((? package? replacement)
+                              (hashq-set! visited replacement #t)
+                              (cons* replacement package result))
+                             (#f
+                              (cons package result))))))
+                   '()
+
+                   ;; Dismiss deprecated packages but keep hidden packages.
+                   #:select? (negate package-superseded))))
 
 (define %package-cache-file
   ;; Location of the package cache.
-- 
2.46.0

[bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.

[Ludovic Courtès]

* guix/build-system/cargo.scm (crate-closure): Remove ‘first?’.

Change-Id: Ica1a661eb422a882ae3a1a54819b2dd7d68449de
---
 guix/build-system/cargo.scm | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 658a2e525e..0e9a4b1d23 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013-2016, 2019, 2021, 2024 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;; Copyright © 2016 David Craven <david@craven.ch>
@@ -227,24 +227,22 @@ (define (crate-closure inputs)
   (let loop ((inputs     inputs)
              (result     '())
              (propagated '())
-             (first?     #t)
              (seen       vlist-null))
     (match inputs
       (()
        (if (null? propagated)
            (reverse result)
-           (loop (reverse (concatenate propagated)) result '() #f seen)))
+           (loop (reverse (concatenate propagated)) result '() seen)))
       (((and input (label (? package? package))) rest ...)
-       (if (and (not first?) (seen? seen package))
-           (loop rest result propagated first? seen)
+       (if (seen? seen package)
+           (loop rest result propagated seen)
            (loop rest
                  (cons input result)
                  (cons (package-cargo-inputs package)
                        propagated)
-                 first?
                  (vhash-consq package package seen))))
       ((input rest ...)
-       (loop rest (cons input result) propagated first? seen)))))
+       (loop rest (cons input result) propagated seen)))))
 
 (define (expand-crate-sources cargo-inputs cargo-development-inputs)
   "Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
-- 
2.46.0

[bug#74654] [PATCH 4/5] guix build: Last argument of ‘dependents’ is optional.

[Ludovic Courtès]

* guix/scripts/build.scm (dependents): Make ‘max-depth’ optional.
Improve docstring.

Change-Id: Ibeffcf15297ecba3d4000b3adb4e3bc7853f6957
---
 guix/scripts/build.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 0080f1fe58..f0a637a2ef 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -564,8 +564,8 @@ (define %options
                  %standard-cross-build-options
                  %standard-native-build-options)))
 
-(define (dependents store packages max-depth)
-  "List all the things that would need to be rebuilt if PACKAGES are changed."
+(define* (dependents store packages #:optional (max-depth +inf.0))
+  "Return the list of dependents of all of PACKAGES up to distance MAX-DEPTH."
   ;; Using %BAG-NODE-TYPE is more accurate than using %PACKAGE-NODE-TYPE
   ;; because it includes implicit dependencies.
   (define (get-dependents packages edges)
-- 
2.46.0

[bug#74654] [PATCH 5/5] maint: Add ungrafting manifest.

[Ludovic Courtès]

* etc/manifests/ungraft.scm: New file.
* Makefile.am (EXTRA_DIST): Add it.

Change-Id: If4aa7649721b93d3297ecd967d5db4a6b71ec6eb
---
 Makefile.am               |  1 +
 etc/manifests/ungraft.scm | 49 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+)
 create mode 100644 etc/manifests/ungraft.scm

diff --git a/Makefile.am b/Makefile.am
index 8c763c68aa..d0d195636b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -743,6 +743,7 @@ EXTRA_DIST +=						\
   etc/manifests/source.scm				\
   etc/manifests/system-tests.scm			\
   etc/manifests/time-travel.scm				\
+  etc/manifests/ungraft.scm				\
   etc/manifests/upgrade.scm				\
   scripts/guix.in					\
   tests/cve-sample.json					\
diff --git a/etc/manifests/ungraft.scm b/etc/manifests/ungraft.scm
new file mode 100644
index 0000000000..3e42b98ece
--- /dev/null
+++ b/etc/manifests/ungraft.scm
@@ -0,0 +1,49 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+;; This manifest "ungrafts" all the currently grafted packages and returns
+;; said packages and all their dependents.
+
+(use-modules (guix diagnostics)
+             (guix i18n)
+             (guix packages)
+             (guix profiles)
+             (guix store)
+             ((guix scripts build) #:select (dependents))
+             ((gnu packages) #:select (all-packages))
+             (srfi srfi-1))
+
+(define (grafted-packages)
+  (info (G_ "enumerating grafted packages...~%"))
+  (let ((result (filter package-replacement (all-packages))))
+    (info (G_ "found ~d grafted packages:~{ ~a~}~%")
+          (length result) (map package-full-name result))
+    result))
+
+(manifest
+ (with-store store
+   (let* ((grafted (grafted-packages))
+          (ungraft-all (package-input-rewriting
+                        (map (lambda (package)
+                               `(,package . ,(package-replacement package)))
+                             grafted))))
+     (map (lambda (package)
+            (manifest-entry
+              (inherit (package->manifest-entry (ungraft-all package)))
+              (name (string-append (package-name package) "-ungrafted"))))
+          (dependents store grafted)))))
-- 
2.46.0

[bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2701 bytes --]

I test applied this on master and then went to build my rusty-packages
manifest. The time on './pre-inst-env guix build -m ... -n' went from 35
seconds to 20 seconds and the derivations didn't change.

On Mon, Dec 02, 2024 at 05:53:28PM +0100, Ludovic Courtès wrote:
> * guix/build-system/cargo.scm (crate-closure): Remove ‘first?’.
> 
> Change-Id: Ica1a661eb422a882ae3a1a54819b2dd7d68449de
> ---
>  guix/build-system/cargo.scm | 12 +++++-------
>  1 file changed, 5 insertions(+), 7 deletions(-)
> 
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 658a2e525e..0e9a4b1d23 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -1,5 +1,5 @@
>  ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2021 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2013-2016, 2019, 2021, 2024 Ludovic Courtès <ludo@gnu.org>
>  ;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
>  ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
>  ;;; Copyright © 2016 David Craven <david@craven.ch>
> @@ -227,24 +227,22 @@ (define (crate-closure inputs)
>    (let loop ((inputs     inputs)
>               (result     '())
>               (propagated '())
> -             (first?     #t)
>               (seen       vlist-null))
>      (match inputs
>        (()
>         (if (null? propagated)
>             (reverse result)
> -           (loop (reverse (concatenate propagated)) result '() #f seen)))
> +           (loop (reverse (concatenate propagated)) result '() seen)))
>        (((and input (label (? package? package))) rest ...)
> -       (if (and (not first?) (seen? seen package))
> -           (loop rest result propagated first? seen)
> +       (if (seen? seen package)
> +           (loop rest result propagated seen)
>             (loop rest
>                   (cons input result)
>                   (cons (package-cargo-inputs package)
>                         propagated)
> -                 first?
>                   (vhash-consq package package seen))))
>        ((input rest ...)
> -       (loop rest (cons input result) propagated first? seen)))))
> +       (loop rest (cons input result) propagated seen)))))
>  
>  (define (expand-crate-sources cargo-inputs cargo-development-inputs)
>    "Extract all transitive sources for CARGO-INPUTS and CARGO-DEVELOPMENT-INPUTS
> -- 
> 2.46.0
> 
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#74654] [PATCH 3/5] build-system/cargo: Simplify ‘crate-closure’.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> I test applied this on master and then went to build my rusty-packages
> manifest. The time on './pre-inst-env guix build -m ... -n' went from 35
> seconds to 20 seconds and the derivations didn't change.

Thanks for testing!  I didn’t expect this to have an impact on
performance, interesting.

Ludo’.

[bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’.

[David Elsing]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> On my laptop, wall-clock time for (all-packages) goes from 27s to 1s.
>
> * gnu/packages.scm (all-packages): Use a hash table to remember visited
> packages instead of calling ‘delete-duplicates’ on the final list.

Thanks for fixing this, commit 4b5dae8 also made
`guix graph -t reverse-package` much slower, because `delete-duplicates`
had not been used in the old `all-packages` procedure before. I wanted
to make a patch, but then saw your series already. :)

Cheers,
David

[bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory.

[Z572]

[-- Attachment #1: Type: text/plain, Size: 5384 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> * etc/disarchive-manifest.scm, etc/hurd-manifest.scm,
> etc/kernels-manifest.scm, etc/release-manifest.scm,
> etc/source-manifest.scm, etc/system-tests.scm,
> etc/time-travel-manifest.scm, etc/upgrade-manifest.scm: Move to…
> * etc/manifests: … here, and drop “-manifest” from file name.
> * Makefile.am (EXTRA_DIST, assert-binaries-available, check-system):
> Adjust accordingly.

Once these files are moved, it may be necessary to mention them in
etc/news.scm to inform others that they need to update their cuirass
specs.

>
> Change-Id: Iedee3d0cdd42e72ef8bbf654ea5d3b47dca95874
> ---
>  Makefile.am                                   | 20 +++++++++----------
>  .../disarchive.scm}                           |  2 +-
>  etc/{hurd-manifest.scm => manifests/hurd.scm} |  0
>  .../kernels.scm}                              |  0
>  .../release.scm}                              |  0
>  .../source.scm}                               |  0
>  etc/{ => manifests}/system-tests.scm          |  0
>  .../time-travel.scm}                          |  0
>  .../upgrade.scm}                              |  0
>  9 files changed, 11 insertions(+), 11 deletions(-)
>  rename etc/{disarchive-manifest.scm => manifests/disarchive.scm} (99%)
>  rename etc/{hurd-manifest.scm => manifests/hurd.scm} (100%)
>  rename etc/{kernels-manifest.scm => manifests/kernels.scm} (100%)
>  rename etc/{release-manifest.scm => manifests/release.scm} (100%)
>  rename etc/{source-manifest.scm => manifests/source.scm} (100%)
>  rename etc/{ => manifests}/system-tests.scm (100%)
>  rename etc/{time-travel-manifest.scm => manifests/time-travel.scm} (100%)
>  rename etc/{upgrade-manifest.scm => manifests/upgrade.scm} (100%)
>
> diff --git a/Makefile.am b/Makefile.am
> index 0cff32c607..8c763c68aa 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -686,7 +686,7 @@ endif !CAN_RUN_TESTS
>  
>  check-system: $(GOBJECTS)
>  	$(AM_V_at)$(top_builddir)/pre-inst-env			\
> -	  guix build -m $(top_srcdir)/etc/system-tests.scm -K
> +	  guix build -m $(top_srcdir)/etc/manifests/system-tests.scm -K
>  
>  # Public keys used to sign substitutes.
>  dist_pkgdata_DATA =				\
> @@ -733,17 +733,17 @@ EXTRA_DIST +=						\
>    build-aux/update-guix-package.scm			\
>    build-aux/xgettext.scm				\
>    doc/build.scm						\
> -  etc/disarchive-manifest.scm				\
>    etc/guix-install.sh					\
>    etc/historical-authorizations				\
>    etc/news.scm						\
> -  etc/hurd-manifest.scm					\
> -  etc/kernels-manifest.scm				\
> -  etc/release-manifest.scm				\
> -  etc/source-manifest.scm				\
> -  etc/system-tests.scm					\
> -  etc/time-travel-manifest.scm				\
> -  etc/upgrade-manifest.scm				\
> +  etc/manifests/disarchive.scm				\
> +  etc/manifests/hurd.scm				\
> +  etc/manifests/kernels.scm				\
> +  etc/manifests/release.scm				\
> +  etc/manifests/source.scm				\
> +  etc/manifests/system-tests.scm			\
> +  etc/manifests/time-travel.scm				\
> +  etc/manifests/upgrade.scm				\
>    scripts/guix.in					\
>    tests/cve-sample.json					\
>    tests/keys/civodul.pub				\
> @@ -1194,7 +1194,7 @@ assert-no-store-file-names:
>  # server so that '--display-missing' doesn't print two lists.
>  assert-binaries-available: $(GOBJECTS)
>  	$(AM_V_at)$(top_builddir)/pre-inst-env				\
> -	  guix weather -m "$(top_srcdir)/etc/release-manifest.scm"	\
> +	  guix weather -m "$(top_srcdir)/etc/manifests/release.scm"	\
>  	                --substitute-urls="https://ci.guix.gnu.org"	\
>  	                --display-missing
>  
> diff --git a/etc/disarchive-manifest.scm b/etc/manifests/disarchive.scm
> similarity index 99%
> rename from etc/disarchive-manifest.scm
> rename to etc/manifests/disarchive.scm
> index 3dbfa356df..a7f71414b6 100644
> --- a/etc/disarchive-manifest.scm
> +++ b/etc/manifests/disarchive.scm
> @@ -24,7 +24,7 @@
>               (guix base16)
>               (gnu packages))
>  
> -(include "source-manifest.scm")
> +(include "source.scm")
>  
>  (define (tarball-origin? origin)
>    (match (origin-actual-file-name origin)
> diff --git a/etc/hurd-manifest.scm b/etc/manifests/hurd.scm
> similarity index 100%
> rename from etc/hurd-manifest.scm
> rename to etc/manifests/hurd.scm
> diff --git a/etc/kernels-manifest.scm b/etc/manifests/kernels.scm
> similarity index 100%
> rename from etc/kernels-manifest.scm
> rename to etc/manifests/kernels.scm
> diff --git a/etc/release-manifest.scm b/etc/manifests/release.scm
> similarity index 100%
> rename from etc/release-manifest.scm
> rename to etc/manifests/release.scm
> diff --git a/etc/source-manifest.scm b/etc/manifests/source.scm
> similarity index 100%
> rename from etc/source-manifest.scm
> rename to etc/manifests/source.scm
> diff --git a/etc/system-tests.scm b/etc/manifests/system-tests.scm
> similarity index 100%
> rename from etc/system-tests.scm
> rename to etc/manifests/system-tests.scm
> diff --git a/etc/time-travel-manifest.scm b/etc/manifests/time-travel.scm
> similarity index 100%
> rename from etc/time-travel-manifest.scm
> rename to etc/manifests/time-travel.scm
> diff --git a/etc/upgrade-manifest.scm b/etc/manifests/upgrade.scm
> similarity index 100%
> rename from etc/upgrade-manifest.scm
> rename to etc/manifests/upgrade.scm

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#74654] [PATCH 1/5] etc: Move manifests to a separate directory.

[Ludovic Courtès]

Hi,

Z572 <zhengjunjie@iscas.ac.cn> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> * etc/disarchive-manifest.scm, etc/hurd-manifest.scm,
>> etc/kernels-manifest.scm, etc/release-manifest.scm,
>> etc/source-manifest.scm, etc/system-tests.scm,
>> etc/time-travel-manifest.scm, etc/upgrade-manifest.scm: Move to…
>> * etc/manifests: … here, and drop “-manifest” from file name.
>> * Makefile.am (EXTRA_DIST, assert-binaries-available, check-system):
>> Adjust accordingly.
>
> Once these files are moved, it may be necessary to mention them in
> etc/news.scm to inform others that they need to update their cuirass
> specs.

Do you think they are used outside the Guix project?  They’re not
documented and not meant for general consumption.

Ludo’.

[bug#74654] [PATCH 2/5] packages: Optimize ‘all-packages’.

[Ludovic Courtès]

Hello,

David Elsing <david.elsing@posteo.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> On my laptop, wall-clock time for (all-packages) goes from 27s to 1s.
>>
>> * gnu/packages.scm (all-packages): Use a hash table to remember visited
>> packages instead of calling ‘delete-duplicates’ on the final list.
>
> Thanks for fixing this, commit 4b5dae8 also made
> `guix graph -t reverse-package` much slower, because `delete-duplicates`
> had not been used in the old `all-packages` procedure before. I wanted
> to make a patch, but then saw your series already. :)

Yes, my bad!  I tried to take the common denominator of all the
‘all-packages’ procedures, and here’s what happens.  :-)

Ludo’.