[bug#29928] [PATCH 0/5] Optimize profile hooks

[bug#29928] [PATCH 0/5] Optimize profile hooks

[宋文武]

Hello, these patches make each profile hook run upon its specified interested
inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
install a package without info files won't trigger it.  Thus reduce the chance
and time to rerun them when your profile changed.


One drawback is 'guix package --dry-run' no longer report the derivations of
profile hooks, and the derivation of profile it reports is not the real one.
Addition files will be built when the profiles hooks are run.


Sou Bunnbu (宋文武) (5):
  gexp: Add 'eval-gexp'.
  profiles: info-dir-file: Don't consider unwanted manifest entries.
  guix package: Disable profile hooks on dry runs.
  profiles: Filter out unwanted manifest entries for profile hooks.
  profiles: Sort manifest inputs for profile hooks.

 guix/gexp.scm            |  16 ++++
 guix/profiles.scm        | 188 ++++++++++++++++++++++++++++++++---------------
 guix/scripts/package.scm |   2 +-
 3 files changed, 146 insertions(+), 60 deletions(-)

--
2.13.3

[bug#29926] [PATCH 1/5] gexp: Add 'eval-gexp'.

[宋文武]

* guix/gexp.scm (eval-gexp): New procedure.
---
 guix/gexp.scm | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/guix/gexp.scm b/guix/gexp.scm
index f005c4d29..9709c5ecb 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -76,6 +76,7 @@
             gexp->derivation
             gexp->file
             gexp->script
+            eval-gexp
             text-file*
             mixed-text-file
             file-union
@@ -1167,6 +1168,21 @@ and '%load-compiled-path' to honor EXP's imported modules."
                          #:local-build? #t
                          #:substitutable? #f)))))
 
+(define* (eval-gexp exp #:optional (name "computed-value"))
+  "Return as a monadic value the EXP (a gexp) evaluate to.  This is
+implemented by building a store file NAME that contains the textual
+representation of EXP's value, and then @code{read} from it."
+  (mlet* %store-monad
+      ((drv (gexp->derivation
+             name
+             (gexp (with-output-to-file (ungexp output)
+                     (lambda ()
+                       (write (eval (quote (ungexp exp)) (current-module))))))
+             #:local-build? #t
+             #:substitutable? #f))
+       (_   (built-derivations (list drv))))
+    (return (call-with-input-file (derivation->output-path drv) read))))
+
 (define* (text-file* name #:rest text)
   "Return as a monadic value a derivation that builds a text file containing
 all of TEXT.  TEXT may list, in addition to strings, objects of any type that
-- 
2.13.3

[bug#29927] [PATCH 2/5] profiles: info-dir-file: Don't consider unwanted manifest entries.

[宋文武]

* guix/profiles.scm (info-dir-file): Use 'eval-gexp' to filter out those
manifest inputs that doesn't have info manuals.
---
 guix/profiles.scm | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/guix/profiles.scm b/guix/profiles.scm
index 3c05543be..f6e455c96 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -684,7 +684,19 @@ MANIFEST."
   (define gzip                                    ;lazy reference
     (module-ref (resolve-interface '(gnu packages compression)) 'gzip))
 
-  (define build
+  ;; We only need to build the 'dir' file for inputs that does contain info
+  ;; manuals.
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/info")))
+        '#$(manifest-inputs manifest))))
+
+  ;; XXX: We have to pass paths of inputs instead of paths of info files,
+  ;; because 'gexp-inputs' only adds inputs for strings which satisfies
+  ;; 'direct-store-path?'.
+  (define (build inputs)
     (with-imported-modules '((guix build utils))
       #~(begin
           (use-modules (guix build utils)
@@ -708,12 +720,12 @@ MANIFEST."
 
           (mkdir-p (string-append #$output "/share/info"))
           (exit (every install-info
-                       (append-map info-files
-                                   '#$(manifest-inputs manifest)))))))
+                       (append-map info-files '#$inputs))))))
 
-  (gexp->derivation "info-dir" build
-                    #:local-build? #t
-                    #:substitutable? #f))
+  (mlet* %store-monad ((inputs interested))
+    (gexp->derivation "info-dir" (build inputs)
+                      #:local-build? #t
+                      #:substitutable? #f)))
 
 (define (ghc-package-cache-file manifest)
   "Return a derivation that builds the GHC 'package.cache' file for all the
-- 
2.13.3

[bug#29925] [PATCH 3/5] guix package: Disable profile hooks on dry runs.

[宋文武]

Profile hooks will access the content of manifest inputs before return the
profile derivation, disable them so we don't get inputs bulit on dry runs.

* guix/scripts/package.scm (build-and-use-profile): Disable profile hooks when
a dry-run parameter is given.
---
 guix/scripts/package.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index 617e102d9..ce0031551 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -203,7 +203,7 @@ specified in MANIFEST, a manifest object."
 
   (let* ((prof-drv (run-with-store store
                      (profile-derivation manifest
-                                         #:hooks (if bootstrap?
+                                         #:hooks (if (or bootstrap? dry-run?)
                                                      '()
                                                      %default-profile-hooks)
                                          #:locales? (not bootstrap?))))
-- 
2.13.3

[bug#29930] [PATCH 4/5] profiles: Filter out unwanted manifest entries for profile hooks.

[宋文武]

* guix/profiles.scm (manual-database, fonts-dir-file, ghc-package-cache-file)
(ca-certificate-bundle, gtk-icon-themes, gtk-im-modules)
(xdg-desktop-database, xdg-mime-database): Use 'eval-gexp' to filter out
unwanted manifest inputs.
---
 guix/profiles.scm | 164 ++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 111 insertions(+), 53 deletions(-)

diff --git a/guix/profiles.scm b/guix/profiles.scm
index f6e455c96..7d69d1a53 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -733,7 +733,15 @@ entries of MANIFEST, or #f if MANIFEST does not have any GHC packages."
   (define ghc                                     ;lazy reference
     (module-ref (resolve-interface '(gnu packages haskell)) 'ghc))
 
-  (define build
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/lib/ghc-"
+                                       #$(package-version ghc))))
+        '#$(manifest-inputs manifest))))
+
+  (define (build inputs)
     (with-imported-modules '((guix build utils))
       #~(begin
           (use-modules (guix build utils)
@@ -763,9 +771,7 @@ entries of MANIFEST, or #f if MANIFEST does not have any GHC packages."
 
           (system* (string-append #+ghc "/bin/ghc-pkg") "init" db-dir)
           (for-each copy-conf-file
-                    (append-map conf-files
-                                (delete-duplicates
-                                 '#$(manifest-inputs manifest))))
+                    (append-map conf-files '#$inputs))
           (let ((success
                  (zero?
                   (system* (string-append #+ghc "/bin/ghc-pkg") "recache"
@@ -773,11 +779,10 @@ entries of MANIFEST, or #f if MANIFEST does not have any GHC packages."
             (for-each delete-file (find-files db-dir "\\.conf$"))
             (exit success)))))
 
-  (with-monad %store-monad
+  (mlet* %store-monad ((inputs interested))
     ;; Don't depend on GHC when there's nothing to do.
-    (if (any (cut string-prefix? "ghc" <>)
-             (map manifest-entry-name (manifest-entries manifest)))
-        (gexp->derivation "ghc-package-cache" build
+    (if (not (null? inputs))
+        (gexp->derivation "ghc-package-cache" (build inputs)
                           #:local-build? #t
                           #:substitutable? #f)
         (return #f))))
@@ -789,10 +794,17 @@ MANIFEST.  Single-file bundles are required by programs such as Git and Lynx."
   ;; See <http://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html>
   ;; for a discussion.
 
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/etc/ssl/certs")))
+        '#$(manifest-inputs manifest))))
+
   (define glibc-utf8-locales                      ;lazy reference
     (module-ref (resolve-interface '(gnu packages base)) 'glibc-utf8-locales))
 
-  (define build
+  (define (build inputs)
     (with-imported-modules '((guix build utils))
       #~(begin
           (use-modules (guix build utils)
@@ -828,7 +840,7 @@ MANIFEST.  Single-file bundles are required by programs such as Git and Lynx."
                                  #+(package-version glibc-utf8-locales)))
           (setlocale LC_ALL "en_US.utf8")
 
-          (match (append-map ca-files '#$(manifest-inputs manifest))
+          (match (append-map ca-files '#$inputs)
             (()
              ;; Since there are no CA files, just create an empty directory.  Do
              ;; not create the etc/ssl/certs sub-directory, since that would
@@ -844,9 +856,10 @@ MANIFEST.  Single-file bundles are required by programs such as Git and Lynx."
                                                  "/ca-certificates.crt"))
                #t))))))
 
-  (gexp->derivation "ca-certificate-bundle" build
-                    #:local-build? #t
-                    #:substitutable? #f))
+  (mlet* %store-monad ((inputs interested))
+    (gexp->derivation "ca-certificate-bundle" (build inputs)
+                      #:local-build? #t
+                      #:substitutable? #f)))
 
 (define (gtk-icon-themes manifest)
   "Return a derivation that unions all icon themes from manifest entries and
@@ -854,7 +867,15 @@ creates the GTK+ 'icon-theme.cache' file for each theme."
   (define gtk+  ; lazy reference
     (module-ref (resolve-interface '(gnu packages gtk)) 'gtk+))
 
-  (mlet %store-monad ((%gtk+ (manifest-lookup-package manifest "gtk+"))
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/icons")))
+        '#$(manifest-inputs manifest))))
+
+  (mlet %store-monad ((inputs interested)
+                      (%gtk+ (manifest-lookup-package manifest "gtk+"))
                       ;; XXX: Can't use gtk-update-icon-cache corresponding
                       ;; to the gtk+ referenced by 'manifest'.  Because
                       ;; '%gtk+' can be either a package or store path, and
@@ -877,9 +898,8 @@ creates the GTK+ 'icon-theme.cache' file for each theme."
                          (ice-9 ftw))
 
             (let* ((destdir  (string-append #$output "/share/icons"))
-                   (icondirs (filter file-exists?
-                                     (map (cut string-append <> "/share/icons")
-                                          '#$(manifest-inputs manifest)))))
+                   (icondirs (map (cut string-append <> "/share/icons")
+                                  '#$inputs)))
 
               ;; Union all the icons.
               (mkdir-p (string-append #$output "/share"))
@@ -907,8 +927,18 @@ creates the GTK+ 'icon-theme.cache' file for each theme."
 (define (gtk-im-modules manifest)
   "Return a derivation that builds the cache files for input method modules
 for both major versions of GTK+."
-
-  (mlet %store-monad ((gtk+   (manifest-lookup-package manifest "gtk+" "3"))
+  (define interested
+    (eval-gexp
+     (with-imported-modules '((guix build utils))
+       #~(begin
+           (use-modules (guix build utils))
+           (filter
+            (lambda (input)
+              (not (null? (find-files input "^immodules$" #:directories? #t))))
+            '#$(manifest-inputs manifest))))))
+
+  (mlet %store-monad ((inputs interested)
+                      (gtk+   (manifest-lookup-package manifest "gtk+" "3"))
                       (gtk+-2 (manifest-lookup-package manifest "gtk+" "2")))
 
     (define (build gtk gtk-version query)
@@ -932,7 +962,7 @@ for both major versions of GTK+."
                      (moddirs (cons (string-append #$gtk prefix "/immodules")
                                     (filter file-exists?
                                             (map (cut string-append <> prefix "/immodules")
-                                                 '#$(manifest-inputs manifest)))))
+                                                 '#$inputs))))
                      (modules (append-map (cut find-files <> "\\.so$")
                                           moddirs)))
 
@@ -980,11 +1010,19 @@ for both major versions of GTK+."
   "Return a derivation that builds the @file{mimeinfo.cache} database from
 desktop files.  It's used to query what applications can handle a given
 MIME type."
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/applications")))
+        '#$(manifest-inputs manifest))))
+
   (define desktop-file-utils            ; lazy reference
     (module-ref (resolve-interface '(gnu packages freedesktop))
                 'desktop-file-utils))
 
-  (mlet %store-monad ((glib
+  (mlet %store-monad ((inputs interested)
+                      (glib
                        (manifest-lookup-package
                         manifest "glib")))
     (define build
@@ -995,10 +1033,9 @@ MIME type."
                          (guix build utils)
                          (guix build union))
             (let* ((destdir (string-append #$output "/share/applications"))
-                   (appdirs (filter file-exists?
-                                    (map (cut string-append <>
-                                              "/share/applications")
-                                         '#$(manifest-inputs manifest))))
+                   (appdirs (map (cut string-append <>
+                                      "/share/applications")
+                                 '#$inputs))
                    (update-desktop-database (string-append
                                              #+desktop-file-utils
                                              "/bin/update-desktop-database")))
@@ -1017,10 +1054,18 @@ MIME type."
 (define (xdg-mime-database manifest)
   "Return a derivation that builds the @file{mime.cache} database from manifest
 entries.  It's used to query the MIME type of a given file."
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/mime/packages")))
+        '#$(manifest-inputs manifest))))
+
   (define shared-mime-info  ; lazy reference
     (module-ref (resolve-interface '(gnu packages gnome)) 'shared-mime-info))
 
-  (mlet %store-monad ((glib
+  (mlet %store-monad ((inputs interested)
+                      (glib
                        (manifest-lookup-package
                         manifest "glib")))
     (define build
@@ -1032,11 +1077,10 @@ entries.  It's used to query the MIME type of a given file."
                          (guix build union))
             (let* ((datadir (string-append #$output "/share"))
                    (destdir (string-append datadir "/mime"))
-                   (pkgdirs (filter file-exists?
-                                    (map (cut string-append <>
-                                              "/share/mime/packages")
-                                         (cons #+shared-mime-info
-                                               '#$(manifest-inputs manifest)))))
+                   (pkgdirs (map (cut string-append <>
+                                      "/share/mime/packages")
+                                 (cons #+shared-mime-info
+                                       '#$inputs)))
                    (update-mime-database (string-append
                                           #+shared-mime-info
                                           "/bin/update-mime-database")))
@@ -1059,21 +1103,27 @@ entries.  It's used to query the MIME type of a given file."
 (define (fonts-dir-file manifest)
   "Return a derivation that builds the @file{fonts.dir} and @file{fonts.scale}
 files for the fonts of the @var{manifest} entries."
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/fonts")))
+        '#$(manifest-inputs manifest))))
+
   (define mkfontscale
     (module-ref (resolve-interface '(gnu packages xorg)) 'mkfontscale))
 
   (define mkfontdir
     (module-ref (resolve-interface '(gnu packages xorg)) 'mkfontdir))
 
-  (define build
+  (define (build inputs)
     #~(begin
         (use-modules (srfi srfi-26)
                      (guix build utils)
                      (guix build union))
-        (let ((fonts-dirs (filter file-exists?
-                                  (map (cut string-append <>
-                                            "/share/fonts")
-                                       '#$(manifest-inputs manifest)))))
+        (let ((fonts-dirs (map (cut string-append <>
+                                    "/share/fonts")
+                               '#$inputs)))
           (mkdir #$output)
           (if (null? fonts-dirs)
               (exit #t)
@@ -1116,16 +1166,24 @@ files for the fonts of the @var{manifest} entries."
                                   (delete-file fonts-dir-file))))
                             directories)))))))
 
-  (gexp->derivation "fonts-dir" build
-                    #:modules '((guix build utils)
-                                (guix build union)
-                                (srfi srfi-26))
-                    #:local-build? #t
-                    #:substitutable? #f))
+  (mlet* %store-monad ((inputs interested))
+    (gexp->derivation "fonts-dir" (build inputs)
+                      #:modules '((guix build utils)
+                                  (guix build union)
+                                  (srfi srfi-26))
+                      #:local-build? #t
+                      #:substitutable? #f)))
 
 (define (manual-database manifest)
   "Return a derivation that builds the manual page database (\"mandb\") for
 the entries in MANIFEST."
+  (define interested
+    (eval-gexp
+     #~(filter
+        (lambda (input)
+          (file-exists? (string-append input "/share/man")))
+        '#$(manifest-inputs manifest))))
+
   (define gdbm-ffi
     (module-ref (resolve-interface '(gnu packages guile))
                 'guile-gdbm-ffi))
@@ -1148,7 +1206,7 @@ the entries in MANIFEST."
                   (source-module-closure `((guix build utils)
                                            (guix man-db))))))
 
-  (define build
+  (define (build inputs)
     (with-imported-modules modules
       #~(begin
           (add-to-load-path (string-append #$gdbm-ffi "/share/guile/site/"
@@ -1162,10 +1220,8 @@ the entries in MANIFEST."
           (define (compute-entries)
             (append-map (lambda (directory)
                           (let ((man (string-append directory "/share/man")))
-                            (if (directory-exists? man)
-                                (mandb-entries man)
-                                '())))
-                        '#$(manifest-inputs manifest)))
+                            (mandb-entries man)))
+                        '#$inputs))
 
           (define man-directory
             (string-append #$output "/share/man"))
@@ -1186,14 +1242,16 @@ the entries in MANIFEST."
                        (* (time-nanosecond duration) (expt 10 -9))))
             (force-output)))))
 
-  (gexp->derivation "manual-database" build
+  (mlet* %store-monad ((inputs interested))
+    (gexp->derivation
+     "manual-databased" (build inputs)
 
-                    ;; Work around GDBM 1.13 issue whereby uninitialized bytes
-                    ;; get written to disk:
-                    ;; <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=29654#23>.
-                    #:env-vars `(("MALLOC_PERTURB_" . "1"))
+     ;; Work around GDBM 1.13 issue whereby uninitialized bytes get written to
+     ;; disk:
+     ;; <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=29654#23>.
+     #:env-vars `(("MALLOC_PERTURB_" . "1"))
 
-                    #:local-build? #t))
+     #:local-build? #t)))
 
 (define %default-profile-hooks
   ;; This is the list of derivation-returning procedures that are called by
-- 
2.13.3

[bug#29929] [PATCH 5/5] profiles: Sort manifest inputs for profile hooks.

[宋文武]

* guix/profiles.scm (info-dir-file, manual-database, fonts-dir-file)
(ghc-package-cache-file, ca-certificate-bundle, gtk-icon-themes)
(gtk-im-modules, xdg-desktop-database, xdg-mime-database): Sort the result of
'manifest-inputs'.
---
 guix/profiles.scm | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/guix/profiles.scm b/guix/profiles.scm
index 7d69d1a53..5da4807ad 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -691,7 +691,7 @@ MANIFEST."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/info")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   ;; XXX: We have to pass paths of inputs instead of paths of info files,
   ;; because 'gexp-inputs' only adds inputs for strings which satisfies
@@ -739,7 +739,7 @@ entries of MANIFEST, or #f if MANIFEST does not have any GHC packages."
         (lambda (input)
           (file-exists? (string-append input "/lib/ghc-"
                                        #$(package-version ghc))))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define (build inputs)
     (with-imported-modules '((guix build utils))
@@ -799,7 +799,7 @@ MANIFEST.  Single-file bundles are required by programs such as Git and Lynx."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/etc/ssl/certs")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define glibc-utf8-locales                      ;lazy reference
     (module-ref (resolve-interface '(gnu packages base)) 'glibc-utf8-locales))
@@ -872,7 +872,7 @@ creates the GTK+ 'icon-theme.cache' file for each theme."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/icons")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (mlet %store-monad ((inputs interested)
                       (%gtk+ (manifest-lookup-package manifest "gtk+"))
@@ -935,7 +935,7 @@ for both major versions of GTK+."
            (filter
             (lambda (input)
               (not (null? (find-files input "^immodules$" #:directories? #t))))
-            '#$(manifest-inputs manifest))))))
+            (sort '#$(manifest-inputs manifest) string<))))))
 
   (mlet %store-monad ((inputs interested)
                       (gtk+   (manifest-lookup-package manifest "gtk+" "3"))
@@ -1015,7 +1015,7 @@ MIME type."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/applications")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define desktop-file-utils            ; lazy reference
     (module-ref (resolve-interface '(gnu packages freedesktop))
@@ -1059,7 +1059,7 @@ entries.  It's used to query the MIME type of a given file."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/mime/packages")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define shared-mime-info  ; lazy reference
     (module-ref (resolve-interface '(gnu packages gnome)) 'shared-mime-info))
@@ -1108,7 +1108,7 @@ files for the fonts of the @var{manifest} entries."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/fonts")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define mkfontscale
     (module-ref (resolve-interface '(gnu packages xorg)) 'mkfontscale))
@@ -1182,7 +1182,7 @@ the entries in MANIFEST."
      #~(filter
         (lambda (input)
           (file-exists? (string-append input "/share/man")))
-        '#$(manifest-inputs manifest))))
+        (sort '#$(manifest-inputs manifest) string<))))
 
   (define gdbm-ffi
     (module-ref (resolve-interface '(gnu packages guile))
-- 
2.13.3

[bug#29925] [PATCH 3/5] guix package: Disable profile hooks on dry runs.

[Danny Milosavljevic]

LGTM!

[bug#29929] [PATCH 5/5] profiles: Sort manifest inputs for profile hooks.

[Danny Milosavljevic]

LGTM!

[bug#29928] [PATCH 0/5] Optimize profile hooks

[Ludovic Courtès]

Hello!

宋文武 <iyzsong@member.fsf.org> skribis:

> Hello, these patches make each profile hook run upon its specified interested
> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
> install a package without info files won't trigger it.  Thus reduce the chance
> and time to rerun them when your profile changed.

That’s an area in need of improvements, thanks for looking into it!

> One drawback is 'guix package --dry-run' no longer report the derivations of
> profile hooks, and the derivation of profile it reports is not the real one.
> Addition files will be built when the profiles hooks are run.

FWIW I’m not entirely convinced by the approach.  As discussed earlier,
I’d like to experiment with a notion of “build rounds”: the first round
would build a profile without any hooks, the second round would,
depending on what the profile contains, rebuild it with certain hooks.

I don’t have anything to show yet, though…

Ludo’.

[bug#29928] [PATCH 0/5] Optimize profile hooks

[宋文武]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> 宋文武 <iyzsong@member.fsf.org> skribis:
>
>> Hello, these patches make each profile hook run upon its specified interested
>> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
>> install a package without info files won't trigger it.  Thus reduce the chance
>> and time to rerun them when your profile changed.
>
> That’s an area in need of improvements, thanks for looking into it!
>
>> One drawback is 'guix package --dry-run' no longer report the derivations of
>> profile hooks, and the derivation of profile it reports is not the real one.
>> Addition files will be built when the profiles hooks are run.
>
> FWIW I’m not entirely convinced by the approach.

Well.. these patches modify package hooks to:

1. build all manifest inputs first.

2. filter manifest inputs to get interested ones.

3. run hook with its interested inputs.

I think reducing the inputs of hook from the whole manifest to its
interested ones is the only way to avoid unneeded reruns.


> As discussed earlier, I’d like to experiment with a notion of “build
> rounds”: the first round would build a profile without any hooks, the
> second round would, depending on what the profile contains, rebuild it
> with certain hooks.

I think "build rounds" would improve the UI/UX, and by changing inputs
from manifest inputs to a built profile, it would simply the current
implementations of profile hooks, but it won't avoid unneeded reruns
compare to the filtered interested inputs way.

Is my understanding correct?

[bug#29928] [PATCH 0/5] Optimize profile hooks

[Ludovic Courtès]

Hi,

iyzsong@member.fsf.org (宋文武) skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>> profile hooks, and the derivation of profile it reports is not the real one.
>>> Addition files will be built when the profiles hooks are run.
>>
>> FWIW I’m not entirely convinced by the approach.
>
> Well.. these patches modify package hooks to:
>
> 1. build all manifest inputs first.
>
> 2. filter manifest inputs to get interested ones.
>
> 3. run hook with its interested inputs.
>
> I think reducing the inputs of hook from the whole manifest to its
> interested ones is the only way to avoid unneeded reruns.

Agreed.

>> As discussed earlier, I’d like to experiment with a notion of “build
>> rounds”: the first round would build a profile without any hooks, the
>> second round would, depending on what the profile contains, rebuild it
>> with certain hooks.
>
> I think "build rounds" would improve the UI/UX, and by changing inputs
> from manifest inputs to a built profile, it would simply the current
> implementations of profile hooks, but it won't avoid unneeded reruns
> compare to the filtered interested inputs way.
>
> Is my understanding correct?

A “build round” is something that computes derivations based on the
output of previously-built derivations.  So it’s just another way to
structure steps 1/2/3 above; it should not reduce expressivity.

Build rounds would allow us to ensure that ‘build-derivations’ is not
called right in the middle of Guix, and is instead always under the
control of the “top level”.

Ludo’.

[bug#29928] [PATCH 0/5] Optimize profile hooks

[宋文武]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> iyzsong@member.fsf.org (宋文武) skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>>> profile hooks, and the derivation of profile it reports is not the real one.
>>>> Addition files will be built when the profiles hooks are run.
>>>
>>> FWIW I’m not entirely convinced by the approach.
>>
>> Well.. these patches modify package hooks to:
>>
>> 1. build all manifest inputs first.
>>
>> 2. filter manifest inputs to get interested ones.
>>
>> 3. run hook with its interested inputs.
>>
>> I think reducing the inputs of hook from the whole manifest to its
>> interested ones is the only way to avoid unneeded reruns.
>
> Agreed.
>
>>> As discussed earlier, I’d like to experiment with a notion of “build
>>> rounds”: the first round would build a profile without any hooks, the
>>> second round would, depending on what the profile contains, rebuild it
>>> with certain hooks.
>>
>> I think "build rounds" would improve the UI/UX, and by changing inputs
>> from manifest inputs to a built profile, it would simply the current
>> implementations of profile hooks, but it won't avoid unneeded reruns
>> compare to the filtered interested inputs way.
>>
>> Is my understanding correct?
>
> A “build round” is something that computes derivations based on the
> output of previously-built derivations.  So it’s just another way to
> structure steps 1/2/3 above; it should not reduce expressivity.
>
> Build rounds would allow us to ensure that ‘build-derivations’ is not
> called right in the middle of Guix, and is instead always under the
> control of the “top level”.
>

I see, looking forward it, thanks!

[bug#29928] [PATCH 0/5] Optimize profile hooks

[Leo Prikler]

merge 29928 29926
merge 29928 29927
thanks

This series (29928 29926 29927 29925 29929 29930) has by now slept on
the mailing list for more than three years.
Should we still try to merge it?  If so, could you send an updated
version, that is not spread across six threads?

Thanks,
Leo

bug#29925: [PATCH 0/5] Optimize profile hooks

[宋文武]

Leo Prikler <leo.prikler@student.tugraz.at> writes:

> merge 29928 29926
> merge 29928 29927
> thanks
>
> This series (29928 29926 29927 29925 29929 29930) has by now slept on
> the mailing list for more than three years.
> Should we still try to merge it?  If so, could you send an updated
> version, that is not spread across six threads?

Hello, thoes patches are obsoleted, should be rewrite to use
`with-build-handler` (I haven't figure it out how yet).

Look like they were all merged as #29925, I think we can close it for
now, thank you!