unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#53210: installer: referring to N-1 guix is problematic.
@ 2022-01-12 16:23 Mathieu Othacehe
  2022-01-12 17:41 ` Ludovic Courtès
  0 siblings, 1 reply; 15+ messages in thread
From: Mathieu Othacehe @ 2022-01-12 16:23 UTC (permalink / raw)
  To: 53210


Hello,

If I download the latest installer and use it to install a new system,
the "guix system init" command will install the guix package defined in
the (gnu packages package-management) module.

That means that if the installer is built with a guix package at the
version N in that module, the guix installed by the installer will be at
version N-1.

Besides the fact it is quite disturbing there are at least two major
issues:

* If the guix package is broken, it needs to build updated twice. I
  recently added a comment about that:

--8<---------------cut here---------------start------------->8---
;; If you are updating this package because it fails to build, you need to
;; actually update it *twice*, as the installer is pointing to the N-1 guix
;; package revision.
--8<---------------cut here---------------end--------------->8---

* When the guix package is updated twice in two separate but near
  commits, Cuirass that is checkouting Guix every 5 minutes might miss
  the intermediate version of the guix package. Hence, the installation
  is taking a while because the guix package is expensive to build.

Any idea on how to improve this inception problem?

Thanks,

Mathieu




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-01-12 16:23 bug#53210: installer: referring to N-1 guix is problematic Mathieu Othacehe
@ 2022-01-12 17:41 ` Ludovic Courtès
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
  0 siblings, 1 reply; 15+ messages in thread
From: Ludovic Courtès @ 2022-01-12 17:41 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: 53210

Hi!

Mathieu Othacehe <othacehe@gnu.org> skribis:

> Any idea on how to improve this inception problem?

We can arrange for ‘installation-os’ to use Guix from the current
commit, as built by (guix self), instead of using the ‘guix’ package.
(For the record, (gnu ci) already does that in ‘system-test-jobs’ to
make installation tests faster.)

I’ll take a look hopefully in the coming days and send a patch here.

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout
  2022-01-12 17:41 ` Ludovic Courtès
@ 2022-02-14  9:29   ` Josselin Poiret via Bug reports for GNU Guix
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code Josselin Poiret via Bug reports for GNU Guix
                       ` (5 more replies)
  0 siblings, 6 replies; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-02-14  9:29 UTC (permalink / raw)
  To: Ludovic Courtès, Mathieu Othacehe; +Cc: Josselin Poiret, 53210

Hello everyone,

I've been working on this for a bit, trying to find the best approach.

First, let me try to summarize the different ways in which Guix can be
run, and how it is built:

* From a `guix pull` profile.  This is the most common.  The `guix
  pull` profile's manifest file holds specific metadata describing the
  channels that were used to build the generation, which can be
  fetched by `(current-channels)` from (guix describe).

* From a built Guix inside the store, outside a `guix pull` profile.
  This is what happens when the user hasn't `guix pull`ed yet after a
  Guix System installation, for example.  Here, the only available
  metadata is the one embedded in (guix config) by the build process,
  and (current-channels) returns that one.

* From a Git checkout.  Here, there is no build metadata at all.

Furthermore, there are two ways to build Guix: either through the
`guix` package definition, which uses the Makefile, or through
`build-aux/build-self.scm` which uses `guix/self.scm`.

Here, I use the second method, since it's how the current guix is most
likely built.  This means that, with proper care, all the derivations
already have their output in the store.

This is only a WIP as there are some unresolved things: I've disabled
authentication, since I'm currently testing my own unsigned patches.
Should we indiscriminately enable it?  This would cause some issues
for people that have their own 'guix channel.  Maybe we could enable
authentication if the channel URL is the default one?  Also, when I
tried to replace the guix in the installer by (current-guix), it ended
up building the Guix derivation multiple times, even though the
package is memoized and the package->derivation process should be
cached.

There's a bit of rift between what the ci uses (renamed here to
channel-profile-build-system) and what current-guix uses, since the
former builds a profile as a package, whereas the latter simply builds
the derivation for the 'guix channel.  This avoids creating profile
collisions when the package is added to the system profile, but it
also means we don't get the package cache generation.

Best,

Josselin Poiret (4):
  gnu: ci: Move generic channel building code.
  gnu: Rename channel-build-system and channel-source->package.
  gnu: current-guix: Support when running outside a checkout.
  gnu: package-management: Memoize current-guix.

 etc/system-tests.scm                |  3 +-
 gnu/ci.scm                          | 40 +------------------
 gnu/packages/package-management.scm | 60 +++++++++++++++++++++++------
 guix/channels.scm                   | 51 ++++++++++++++++++++++++
 4 files changed, 102 insertions(+), 52 deletions(-)

-- 
2.34.0





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code.
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14  9:29     ` Josselin Poiret via Bug reports for GNU Guix
  2022-02-14 17:12       ` Ludovic Courtès
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 2/4] gnu: Rename channel-build-system and channel-source->package Josselin Poiret via Bug reports for GNU Guix
                       ` (4 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-02-14  9:29 UTC (permalink / raw)
  To: Ludovic Courtès, Mathieu Othacehe; +Cc: Josselin Poiret, 53210

* gnu/ci.scm (channel-build-system): Move to guix/channels.scm.
(channel-source->package): Move to
gnu/packages/package-management.scm.
* guix/channels.scm (channel-build-system): Moved from gnu/ci.scm.
* gnu/packages/package-management.scm (channel-source->package): Moved
from gnu/ci.scm.
* etc/system-tests.scm: Use module (gnu packages package-management)
instead of (gnu ci).
---
 etc/system-tests.scm                |  1 -
 gnu/ci.scm                          | 38 -----------------------------
 gnu/packages/package-management.scm | 16 ++++++++++++
 guix/channels.scm                   | 27 ++++++++++++++++++++
 4 files changed, 43 insertions(+), 39 deletions(-)

diff --git a/etc/system-tests.scm b/etc/system-tests.scm
index 1085deed24..ba0c106553 100644
--- a/etc/system-tests.scm
+++ b/etc/system-tests.scm
@@ -18,7 +18,6 @@
 
 (use-modules (gnu tests)
              (gnu packages package-management)
-             ((gnu ci) #:select (channel-source->package))
              ((guix git-download) #:select (git-predicate))
              ((guix utils) #:select (current-source-directory))
              (git)
diff --git a/gnu/ci.scm b/gnu/ci.scm
index 35fd583f75..be19bda413 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -32,7 +32,6 @@ (define-module (gnu ci)
   #:use-module (guix channels)
   #:use-module (guix config)
   #:use-module (guix derivations)
-  #:use-module (guix build-system)
   #:use-module (guix monads)
   #:use-module (guix gexp)
   #:use-module (guix ui)
@@ -72,7 +71,6 @@ (define-module (gnu ci)
 
             %core-packages
             %cross-targets
-            channel-source->package
 
             arguments->systems
             cuirass-jobs))
@@ -300,42 +298,6 @@ (define MiB
               '()))
       '()))
 
-(define channel-build-system
-  ;; Build system used to "convert" a channel instance to a package.
-  (let* ((build (lambda* (name inputs
-                               #:key source commit system
-                               #:allow-other-keys)
-                  (mlet* %store-monad ((source (if (string? source)
-                                                   (return source)
-                                                   (lower-object source)))
-                                       (instance
-                                        -> (checkout->channel-instance
-                                            source #:commit commit)))
-                    (channel-instances->derivation (list instance)))))
-         (lower (lambda* (name #:key system source commit
-                               #:allow-other-keys)
-                  (bag
-                    (name name)
-                    (system system)
-                    (build build)
-                    (arguments `(#:source ,source
-                                 #:commit ,commit))))))
-    (build-system (name 'channel)
-                  (description "Turn a channel instance into a package.")
-                  (lower lower))))
-
-(define* (channel-source->package source #:key commit)
-  "Return a package for the given channel SOURCE, a lowerable object."
-  (package
-    (inherit guix)
-    (version (string-append (package-version guix) "+"))
-    (build-system channel-build-system)
-    (arguments `(#:source ,source
-                 #:commit ,commit))
-    (inputs '())
-    (native-inputs '())
-    (propagated-inputs '())))
-
 (define* (system-test-jobs store system
                            #:key source commit)
   "Return a list of jobs for the system tests."
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 03cc9a6612..edef91ff7e 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -114,6 +114,9 @@ (define-module (gnu packages package-management)
   #:use-module (guix build-system meson)
   #:use-module (guix build-system python)
   #:use-module (guix build-system trivial)
+  ;; This will be loaded by build-self.scm, but guile-git is unavailable, so
+  ;; lazily load instead.
+  #:autoload (guix channels) (channel-build-system guix-channel?)
   #:use-module (guix download)
   #:use-module (guix gexp)
   #:use-module (guix git-download)
@@ -572,6 +575,19 @@ (define (wrong-extension? file)
     (_
      #t)))
 
+(define-public channel-source->package
+  (lambda* (source #:key commit)
+    "Return a package for the given channel SOURCE, a lowerable object."
+    (package
+      (inherit guix)
+      (version (string-append (package-version guix) "+"))
+      (build-system channel-build-system)
+      (arguments `(#:source ,source
+                   #:commit ,commit))
+      (inputs '())
+      (native-inputs '())
+      (propagated-inputs '()))))
+
 (define-public current-guix-package
   ;; This parameter allows callers to override the package that 'current-guix'
   ;; returns.  This is useful when 'current-guix' cannot compute it by itself,
diff --git a/guix/channels.scm b/guix/channels.scm
index 5f47834c10..d637d5863a 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -27,6 +27,7 @@ (define-module (guix channels)
                 #:select (openpgp-public-key-fingerprint
                           openpgp-format-fingerprint))
   #:use-module (guix base16)
+  #:use-module (guix build-system)
   #:use-module (guix records)
   #:use-module (guix gexp)
   #:use-module (guix modules)
@@ -93,6 +94,8 @@ (define-module (guix channels)
             channel-instances->derivation
             ensure-forward-channel-update
 
+            channel-build-system
+
             profile-channels
             manifest-entry-channel
             sexp->channel
@@ -952,6 +955,30 @@ (define* (latest-channel-derivation #:optional (channels %default-channels)
                                                   validate-pull)))
     (channel-instances->derivation instances)))
 
+(define channel-build-system
+  ;; Build system used to "convert" a channel instance to a package.
+  (let* ((build (lambda* (name inputs
+                               #:key source commit system
+                               #:allow-other-keys)
+                  (mlet* %store-monad ((source (if (string? source)
+                                                   (return source)
+                                                   (lower-object source)))
+                                       (instance
+                                        -> (checkout->channel-instance
+                                            source #:commit commit)))
+                    (channel-instances->derivation (list instance)))))
+         (lower (lambda* (name #:key system source commit
+                               #:allow-other-keys)
+                  (bag
+                    (name name)
+                    (system system)
+                    (build build)
+                    (arguments `(#:source ,source
+                                 #:commit ,commit))))))
+    (build-system (name 'channel)
+                  (description "Turn a channel instance into a package.")
+                  (lower lower))))
+
 (define* (sexp->channel sexp #:optional (name 'channel))
   "Read SEXP, a provenance sexp as created by 'channel-instance->sexp'; use
 NAME as the channel name if SEXP does not specify it.  Return #f if the sexp
-- 
2.34.0





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 2/4] gnu: Rename channel-build-system and channel-source->package.
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14  9:29     ` Josselin Poiret via Bug reports for GNU Guix
  2022-07-18 19:35       ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout Josselin Poiret via Bug reports for GNU Guix
                       ` (3 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-02-14  9:29 UTC (permalink / raw)
  To: Ludovic Courtès, Mathieu Othacehe; +Cc: Josselin Poiret, 53210

* etc/system-tests.scm (tests-for-current-guix):
* gnu/ci.scm (system-test-jobs):
* gnu/packages/package-management.scm (channel-source->package,
current-guix):
* guix/channels.scm (channel-build-system):
Rename channel-build-system to channel-profile-build-system and
channel-source->package to channel-source->profile-package.
---
 etc/system-tests.scm                | 2 +-
 gnu/ci.scm                          | 2 +-
 gnu/packages/package-management.scm | 4 ++--
 guix/channels.scm                   | 7 ++++---
 4 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/etc/system-tests.scm b/etc/system-tests.scm
index ba0c106553..ae5e604e10 100644
--- a/etc/system-tests.scm
+++ b/etc/system-tests.scm
@@ -48,7 +48,7 @@ (define (tests-for-current-guix source commit)
   ;;
   ;;   make check-system TESTS=installed-os
   (parameterize ((current-guix-package
-                  (channel-source->package source #:commit commit)))
+                  (channel-source->profile-package source #:commit commit)))
     (match (getenv "TESTS")
       (#f
        (all-system-tests))
diff --git a/gnu/ci.scm b/gnu/ci.scm
index be19bda413..1848015194 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -319,7 +319,7 @@ (define (->job test)
       ;; expensive.  It also makes sure we get a valid Guix package when this
       ;; code is not running from a checkout.
       (parameterize ((current-guix-package
-                      (channel-source->package source #:commit commit)))
+                      (channel-source->profile-package source #:commit commit)))
         (map ->job (all-system-tests)))
       '()))
 
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index edef91ff7e..35913e6153 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -575,13 +575,13 @@ (define (wrong-extension? file)
     (_
      #t)))
 
-(define-public channel-source->package
+(define-public channel-source->profile-package
   (lambda* (source #:key commit)
     "Return a package for the given channel SOURCE, a lowerable object."
     (package
       (inherit guix)
       (version (string-append (package-version guix) "+"))
-      (build-system channel-build-system)
+      (build-system channel-profile-build-system)
       (arguments `(#:source ,source
                    #:commit ,commit))
       (inputs '())
diff --git a/guix/channels.scm b/guix/channels.scm
index d637d5863a..01f63d9631 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -94,7 +94,7 @@ (define-module (guix channels)
             channel-instances->derivation
             ensure-forward-channel-update
 
-            channel-build-system
+            channel-profile-build-system
 
             profile-channels
             manifest-entry-channel
@@ -955,8 +955,9 @@ (define* (latest-channel-derivation #:optional (channels %default-channels)
                                                   validate-pull)))
     (channel-instances->derivation instances)))
 
-(define channel-build-system
-  ;; Build system used to "convert" a channel instance to a package.
+(define channel-profile-build-system
+  ;; Build system used to "convert" a channel instance to a profile, in
+  ;; package form.
   (let* ((build (lambda* (name inputs
                                #:key source commit system
                                #:allow-other-keys)
-- 
2.34.0





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout.
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code Josselin Poiret via Bug reports for GNU Guix
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 2/4] gnu: Rename channel-build-system and channel-source->package Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14  9:29     ` Josselin Poiret via Bug reports for GNU Guix
  2022-02-14 17:15       ` Ludovic Courtès
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 4/4] gnu: package-management: Memoize current-guix Josselin Poiret via Bug reports for GNU Guix
                       ` (2 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-02-14  9:29 UTC (permalink / raw)
  To: Ludovic Courtès, Mathieu Othacehe; +Cc: Josselin Poiret, 53210

* guix/channels.scm (channel-build-system): Add build system that
turns a channel record into a package.
* gnu/packages/package-management.scm (current-guix): Use
channel-build-system.
---
 gnu/packages/package-management.scm | 45 +++++++++++++++++++++--------
 guix/channels.scm                   | 23 +++++++++++++++
 2 files changed, 56 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 35913e6153..fe906fd440 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -116,10 +116,14 @@ (define-module (gnu packages package-management)
   #:use-module (guix build-system trivial)
   ;; This will be loaded by build-self.scm, but guile-git is unavailable, so
   ;; lazily load instead.
-  #:autoload (guix channels) (channel-build-system guix-channel?)
+  #:autoload (guix channels) (channel-profile-build-system
+                              channel-build-system
+                              guix-channel?)
+  #:use-module (guix describe)
   #:use-module (guix download)
   #:use-module (guix gexp)
   #:use-module (guix git-download)
+  #:autoload (guix git) (git-checkout)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
   #:use-module (guix utils)
@@ -588,6 +592,18 @@ (define-public channel-source->profile-package
       (native-inputs '())
       (propagated-inputs '()))))
 
+(define-public channel->package
+  (lambda (channel)
+    "Return a package for the given CHANNEL."
+    (package
+      (inherit guix)
+      (version (string-append (package-version guix) "+"))
+      (build-system channel-build-system)
+      (arguments `(#:channel ,channel))
+      (inputs '())
+      (native-inputs '())
+      (propagated-inputs '()))))
+
 (define-public current-guix-package
   ;; This parameter allows callers to override the package that 'current-guix'
   ;; returns.  This is useful when 'current-guix' cannot compute it by itself,
@@ -595,22 +611,27 @@ (define-public current-guix-package
   (make-parameter #f))
 
 (define-public current-guix
-  (let* ((repository-root (delay (canonicalize-path
-                                  (string-append (current-source-directory)
-                                                 "/../.."))))
-         (select? (delay (or (git-predicate (force repository-root))
-                             source-file?))))
-    (lambda ()
-      "Return a package representing Guix built from the current source tree.
-This works by adding the current source tree to the store (after filtering it
-out) and returning a package that uses that as its 'source'."
+  (lambda ()
+    "Return a package representing Guix built from the currently used one.
+This works by either looking up profile or build metadata, and building from
+the current Guix channel.  If that metadata is missing, assume we are running
+from a Git checkout, so add the current source tree to the store (after
+filtering it out) and return a package that uses that as its 'source'."
+    (let* ((guix-channel (find guix-channel? (current-channels)))
+           (repository-root (canonicalize-path
+                             (string-append (current-source-directory)
+                                            "/../..")))
+           (select? (or (git-predicate  repository-root)
+                        source-file?)))
       (or (current-guix-package)
+          (and guix-channel
+               (channel->package guix-channel))
           (package
             (inherit guix)
             (version (string-append (package-version guix) "+"))
-            (source (local-file (force repository-root) "guix-current"
+            (source (local-file repository-root "guix-current"
                                 #:recursive? #t
-                                #:select? (force select?))))))))
+                                #:select? select?)))))))
 
 (define-public guix-icons
   (package
diff --git a/guix/channels.scm b/guix/channels.scm
index 01f63d9631..c930fd2ae7 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -94,6 +94,7 @@ (define-module (guix channels)
             channel-instances->derivation
             ensure-forward-channel-update
 
+            channel-build-system
             channel-profile-build-system
 
             profile-channels
@@ -955,6 +956,28 @@ (define* (latest-channel-derivation #:optional (channels %default-channels)
                                                   validate-pull)))
     (channel-instances->derivation instances)))
 
+(define channel-build-system
+  ;; Build system used to "convert" a channel to a package.
+  (let* ((build (lambda* (name inputs
+                               #:key channel system
+                               #:allow-other-keys)
+                  (mlet* %store-monad ((instance
+                                        ((store-lift latest-channel-instance)
+                                         channel
+                                         #:authenticate? #f
+                                         #:validate-pull (lambda (. rest) #t))))
+                    (build-from-source instance #:system system))))
+         (lower (lambda* (name #:key system channel
+                               #:allow-other-keys)
+                  (bag
+                    (name name)
+                    (system system)
+                    (build build)
+                    (arguments `(#:channel ,channel))))))
+    (build-system (name 'channel)
+                  (description "Turn a channel into a package.")
+                  (lower lower))))
+
 (define channel-profile-build-system
   ;; Build system used to "convert" a channel instance to a profile, in
   ;; package form.
-- 
2.34.0





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 4/4] gnu: package-management: Memoize current-guix.
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
                       ` (2 preceding siblings ...)
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14  9:29     ` Josselin Poiret via Bug reports for GNU Guix
  2022-02-14 17:10     ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Ludovic Courtès
  2022-08-09 13:26     ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
  5 siblings, 0 replies; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-02-14  9:29 UTC (permalink / raw)
  To: Ludovic Courtès, Mathieu Othacehe; +Cc: Josselin Poiret, 53210

* gnu/packages/package-management.scm (current-guix): Memoize.
---
 gnu/packages/package-management.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index fe906fd440..a7b98bbb1a 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -125,6 +125,7 @@ (define-module (gnu packages package-management)
   #:use-module (guix git-download)
   #:autoload (guix git) (git-checkout)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix memoization)
   #:use-module (guix packages)
   #:use-module (guix utils)
   #:use-module (ice-9 match)
@@ -611,7 +612,7 @@ (define-public current-guix-package
   (make-parameter #f))
 
 (define-public current-guix
-  (lambda ()
+  (mlambda ()
     "Return a package representing Guix built from the currently used one.
 This works by either looking up profile or build metadata, and building from
 the current Guix channel.  If that metadata is missing, assume we are running
-- 
2.34.0





^ permalink raw reply related	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
                       ` (3 preceding siblings ...)
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 4/4] gnu: package-management: Memoize current-guix Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14 17:10     ` Ludovic Courtès
  2022-08-09 13:26     ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
  5 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-02-14 17:10 UTC (permalink / raw)
  To: Josselin Poiret; +Cc: Mathieu Othacehe, 53210

Hello,

Thanks for working on it!

Josselin Poiret <dev@jpoiret.xyz> skribis:

> This is only a WIP as there are some unresolved things: I've disabled
> authentication, since I'm currently testing my own unsigned patches.
> Should we indiscriminately enable it?  This would cause some issues
> for people that have their own 'guix channel.  Maybe we could enable
> authentication if the channel URL is the default one?

How about adding a keyword argument to ‘channel-build-system’,
defaulting to true?

> Also, when I tried to replace the guix in the installer by
> (current-guix), it ended up building the Guix derivation multiple
> times, even though the package is memoized and the package->derivation
> process should be cached.

‘current-guix’ returns a fresh package object every time it’s called:

 (define-public current-guix
   (lambda ()
     …
     (package …)))

This defeats the object-to-derivation cache, which relies on object
identity (eq?).  So yes, you definitely have to turn it into ‘mlambda’.

Why that’s no enough, I don’t know.

> There's a bit of rift between what the ci uses (renamed here to
> channel-profile-build-system) and what current-guix uses, since the
> former builds a profile as a package, whereas the latter simply builds
> the derivation for the 'guix channel.  This avoids creating profile
> collisions when the package is added to the system profile, but it
> also means we don't get the package cache generation.

As discussed on IRC, I think the file-level collisions are fine: unless
I’m mistaken, it’s the ‘manifest’ file of the real profile that “wins”,
so it doesn’t really matter that the ‘guix’ package also provides a
‘manifest’ file.

What’s more problematic though is that it means that
“/run/current-system/profile/bin/guix describe” would now fail (I think?
Can you confirm?) since it gets its provenance data from the profile
it’s in.

WDYT?

Ludo’.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code.
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14 17:12       ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-02-14 17:12 UTC (permalink / raw)
  To: Josselin Poiret; +Cc: Mathieu Othacehe, 53210

Josselin Poiret <dev@jpoiret.xyz> skribis:

> * gnu/ci.scm (channel-build-system): Move to guix/channels.scm.
> (channel-source->package): Move to
> gnu/packages/package-management.scm.
> * guix/channels.scm (channel-build-system): Moved from gnu/ci.scm.
> * gnu/packages/package-management.scm (channel-source->package): Moved
> from gnu/ci.scm.
> * etc/system-tests.scm: Use module (gnu packages package-management)
> instead of (gnu ci).

How about making it (guix build-system channel), to be consistent with
other build systems?




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout.
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout Josselin Poiret via Bug reports for GNU Guix
@ 2022-02-14 17:15       ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-02-14 17:15 UTC (permalink / raw)
  To: Josselin Poiret; +Cc: Mathieu Othacehe, 53210

Josselin Poiret <dev@jpoiret.xyz> skribis:

> * guix/channels.scm (channel-build-system): Add build system that
> turns a channel record into a package.
> * gnu/packages/package-management.scm (current-guix): Use
> channel-build-system.

[...]

>  (define-public current-guix
> -  (let* ((repository-root (delay (canonicalize-path
> -                                  (string-append (current-source-directory)
> -                                                 "/../.."))))
> -         (select? (delay (or (git-predicate (force repository-root))
> -                             source-file?))))
> -    (lambda ()
> -      "Return a package representing Guix built from the current source tree.
> -This works by adding the current source tree to the store (after filtering it
> -out) and returning a package that uses that as its 'source'."
> +  (lambda ()
> +    "Return a package representing Guix built from the currently used one.
> +This works by either looking up profile or build metadata, and building from
> +the current Guix channel.  If that metadata is missing, assume we are running
> +from a Git checkout, so add the current source tree to the store (after
> +filtering it out) and return a package that uses that as its 'source'."
> +    (let* ((guix-channel (find guix-channel? (current-channels)))
> +           (repository-root (canonicalize-path
> +                             (string-append (current-source-directory)
> +                                            "/../..")))
> +           (select? (or (git-predicate  repository-root)
> +                        source-file?)))

We should keep these two variables as promises in the closure, to make
sure we don’t recompute them over and over… unless ‘lambda’ is changed
to ‘mlambda’ in the same commit, which is probably the best course of
action.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-02-14  9:29     ` bug#53210: [WIP PATCH 2/4] gnu: Rename channel-build-system and channel-source->package Josselin Poiret via Bug reports for GNU Guix
@ 2022-07-18 19:35       ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-07-18 19:35 UTC (permalink / raw)
  To: Josselin Poiret; +Cc: Mathieu Othacehe, 53210

Josselin Poiret <dev@jpoiret.xyz> skribis:

> -(define channel-build-system
> -  ;; Build system used to "convert" a channel instance to a package.
> +(define channel-profile-build-system
> +  ;; Build system used to "convert" a channel instance to a profile, in
> +  ;; package form.

In hindsight I’m not convinced about this rename; I don’t think adding
“profile” brings much.  WDYT?

Ludo’.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
                       ` (4 preceding siblings ...)
  2022-02-14 17:10     ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Ludovic Courtès
@ 2022-08-09 13:26     ` Ludovic Courtès
  2022-08-09 19:17       ` Josselin Poiret via Bug reports for GNU Guix
  2022-08-10 10:13       ` Mathieu Othacehe
  5 siblings, 2 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-08-09 13:26 UTC (permalink / raw)
  To: Josselin Poiret; +Cc: Mathieu Othacehe, 53210

Hi Josselin & all,

I ended up pushing the following patches:

  fdafd40432 maint: Use a pretty version string in ISO and VM images.
  95a03aa5c5 system: install: Always use 'current-guix'.
  57f1892d36 gnu: guix: Default 'current-guix' is built using the current channels.
  64a070717c channels: Add 'repository->guix-channel'.
  cf60a0a906 build-system/channel: Accept a channel or instance as the source.
  5bce4c8242 build-system: Add 'channel-build-system'.

The basics are similar to what you had posted.  It adds a useful default
for ‘current-guix’: a package built from the ‘guix’ channel as returned
by ‘guix describe’.  In turn, ‘%installation-os’ in (gnu system install)
is changed to use (current-guix) instead of the ‘guix’ package, meaning
that:

  guix system image gnu/system/install.scm

and:

  ./pre-inst-env guix system image gnu/system/install.scm

both produce an image that contains the current Guix.

That allows us to remove the second ‘update-guix-package.scm’ + ‘git
commit’ invocation in ‘make release’, which should make the whole
process faster.

Note that the default for installed systems remains the ‘guix’ package,
not (current-guix).  The main reason is that the “Computing derivation”
step when using (current-guix) is too high to do that by default.  We
could mitigate that by caching the result of that step more
systematically, but I think that can come later.

Let me know if you have comments!

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-08-09 13:26     ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
@ 2022-08-09 19:17       ` Josselin Poiret via Bug reports for GNU Guix
  2022-08-10 10:13       ` Mathieu Othacehe
  1 sibling, 0 replies; 15+ messages in thread
From: Josselin Poiret via Bug reports for GNU Guix @ 2022-08-09 19:17 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Mathieu Othacehe, 53210

Hi Ludo,

Sorry for the delay, I was on holidays for the past few weeks.

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Josselin & all,
>
> I ended up pushing the following patches:
>
>   fdafd40432 maint: Use a pretty version string in ISO and VM images.
>   95a03aa5c5 system: install: Always use 'current-guix'.
>   57f1892d36 gnu: guix: Default 'current-guix' is built using the current channels.
>   64a070717c channels: Add 'repository->guix-channel'.
>   cf60a0a906 build-system/channel: Accept a channel or instance as the source.
>   5bce4c8242 build-system: Add 'channel-build-system'.

Great, thanks for adopting my code and fixing it up!

> Let me know if you have comments!
>
> Thanks,
> Ludo’.

-- 
Josselin Poiret




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-08-09 13:26     ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
  2022-08-09 19:17       ` Josselin Poiret via Bug reports for GNU Guix
@ 2022-08-10 10:13       ` Mathieu Othacehe
  2022-08-11 14:03         ` Ludovic Courtès
  1 sibling, 1 reply; 15+ messages in thread
From: Mathieu Othacehe @ 2022-08-10 10:13 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Josselin Poiret, 53210


Hey,

> Let me know if you have comments!

Thanks for taking care of this!

Looks like we have a small regression on 'system-tests and 'guix
specifications:

https://ci.guix.gnu.org/eval/528053/log/raw
https://ci.guix.gnu.org/eval/528056/log/raw

I think this is because channel-source->package is given a raw directory
as source in (gnu ci) while this procedure expects either a channel or a
lowerable object.

Thanks,

Mathieu




^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#53210: installer: referring to N-1 guix is problematic.
  2022-08-10 10:13       ` Mathieu Othacehe
@ 2022-08-11 14:03         ` Ludovic Courtès
  0 siblings, 0 replies; 15+ messages in thread
From: Ludovic Courtès @ 2022-08-11 14:03 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: Josselin Poiret, 53210

Hello,

Mathieu Othacehe <othacehe@gnu.org> skribis:

>> Let me know if you have comments!
>
> Thanks for taking care of this!
>
> Looks like we have a small regression on 'system-tests and 'guix
> specifications:
>
> https://ci.guix.gnu.org/eval/528053/log/raw
> https://ci.guix.gnu.org/eval/528056/log/raw
>
> I think this is because channel-source->package is given a raw directory
> as source in (gnu ci) while this procedure expects either a channel or a
> lowerable object.

Indeed.  This should be fixed by
a81706494753ad84754cbb7583ccc783452decc0.

Thanks!

Ludo'.




^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2022-08-11 14:04 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-12 16:23 bug#53210: installer: referring to N-1 guix is problematic Mathieu Othacehe
2022-01-12 17:41 ` Ludovic Courtès
2022-02-14  9:29   ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Josselin Poiret via Bug reports for GNU Guix
2022-02-14  9:29     ` bug#53210: [WIP PATCH 1/4] gnu: ci: Move generic channel building code Josselin Poiret via Bug reports for GNU Guix
2022-02-14 17:12       ` Ludovic Courtès
2022-02-14  9:29     ` bug#53210: [WIP PATCH 2/4] gnu: Rename channel-build-system and channel-source->package Josselin Poiret via Bug reports for GNU Guix
2022-07-18 19:35       ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
2022-02-14  9:29     ` bug#53210: [WIP PATCH 3/4] gnu: current-guix: Support when running outside a checkout Josselin Poiret via Bug reports for GNU Guix
2022-02-14 17:15       ` Ludovic Courtès
2022-02-14  9:29     ` bug#53210: [WIP PATCH 4/4] gnu: package-management: Memoize current-guix Josselin Poiret via Bug reports for GNU Guix
2022-02-14 17:10     ` bug#53210: [WIP PATCH 0/4] Make current-guix work when run outside a Git checkout Ludovic Courtès
2022-08-09 13:26     ` bug#53210: installer: referring to N-1 guix is problematic Ludovic Courtès
2022-08-09 19:17       ` Josselin Poiret via Bug reports for GNU Guix
2022-08-10 10:13       ` Mathieu Othacehe
2022-08-11 14:03         ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).