[bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Ludovic Courtès]

Hello,

This change makes things like:

  guix build --with-input=guile=guile-next guix -n --no-grafts

more useful and tractable.

Low-level rewrites are still possible for packages not marked
as hidden in 'commencement.scm', such as glibc:

  guix build --with-latest=glibc hello -n

Thoughts?

Ludo'.

Ludovic Courtès (2):
  packages: Use SRFI-71 instead of SRFI-11.
  packages: 'package-input-rewriting/spec' ignores hidden packages.

 doc/guix.texi      | 21 ++++++++++++---------
 guix/packages.scm  | 14 ++++++++------
 tests/packages.scm | 20 +++++++++++++++++++-
 3 files changed, 39 insertions(+), 16 deletions(-)


base-commit: 0a37921d851b94aef6900214098dc5bd62b46e26
-- 
2.39.1

[bug#62047] [PATCH 1/2] packages: Use SRFI-71 instead of SRFI-11.

[Ludovic Courtès]

* guix/packages.scm (package-input-rewriting/spec): Use SRFI-71 'let'.
---
 guix/packages.scm | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..cd61878bcc 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2014, 2015, 2017, 2018, 2019 Mark H Weaver <mhw@netris.org>
 ;;; Copyright © 2015 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2016 Alex Kost <alezost@gmail.com>
@@ -51,10 +51,10 @@ (define-module (guix packages)
   #:use-module (ice-9 regex)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9 gnu)
-  #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
+  #:use-module (srfi srfi-71)
   #:use-module (rnrs bytevectors)
   #:use-module (web uri)
   #:autoload   (texinfo) (texi-fragment->stexi)
@@ -1534,8 +1534,7 @@ (define table
     (fold (lambda (replacement table)
             (match replacement
               ((spec . proc)
-               (let-values (((name version)
-                             (package-name->name+version spec)))
+               (let ((name version (package-name->name+version spec)))
                  (vhash-cons name (list version proc) table)))))
           vlist-null
           replacements))
-- 
2.39.1

[bug#62047] [PATCH 2/2] packages: 'package-input-rewriting/spec' ignores hidden packages.

[Ludovic Courtès]

The primary motivation is to support things like:

  guix build guix --with-input=guile=guile-next

without triggering a rebuild of (@@ (gnu packages commencement)
guile-final) and similar things.

It is also consistent with package name resolution on the command line:
a package that cannot be named cannot be replaced.

* guix/packages.scm (package-input-rewriting/spec)[rewrite]: When P is
hidden, return it as-is.
* tests/packages.scm ("package-input-rewriting/spec, hidden package"):
New test.
* doc/guix.texi (Defining Package Variants): Update.
(Package Transformation Options): Update '--with-input' example.
---
 doc/guix.texi      | 21 ++++++++++++---------
 guix/packages.scm  |  7 +++++--
 tests/packages.scm | 20 +++++++++++++++++++-
 3 files changed, 36 insertions(+), 12 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 6671ba9305..6803512435 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -8331,10 +8331,13 @@ be replaced by name rather than by identity.
 @deffn {Scheme Procedure} package-input-rewriting/spec @var{replacements} [#:deep? #t]
 Return a procedure that, given a package, applies the given
 @var{replacements} to all the package graph, including implicit inputs
-unless @var{deep?} is false.  @var{replacements} is a list of
-spec/procedures pair; each spec is a package specification such as
-@code{"gcc"} or @code{"guile@@2"}, and each procedure takes a matching
-package and returns a replacement for that package.
+unless @var{deep?} is false.
+
+@var{replacements} is a list of spec/procedures pair; each spec is a
+package specification such as @code{"gcc"} or @code{"guile@@2"}, and
+each procedure takes a matching package and returns a replacement for
+that package.  Matching packages that have the @code{hidden?} property
+set are not replaced.
 @end deffn
 
 The example above could be rewritten this way:
@@ -12664,18 +12667,18 @@ or @code{guile@@1.8}.
 
 For instance, the following command builds Guix, but replaces its
 dependency on the current stable version of Guile with a dependency on
-the legacy version of Guile, @code{guile@@2.0}:
+the legacy version of Guile, @code{guile@@2.2}:
 
 @example
-guix build --with-input=guile=guile@@2.0 guix
+guix build --with-input=guile=guile@@2.2 guix
 @end example
 
 This is a recursive, deep replacement.  So in this example, both
 @code{guix} and its dependency @code{guile-json} (which also depends on
-@code{guile}) get rebuilt against @code{guile@@2.0}.
+@code{guile}) get rebuilt against @code{guile@@2.2}.
 
-This is implemented using the @code{package-input-rewriting} Scheme
-procedure (@pxref{Defining Packages, @code{package-input-rewriting}}).
+This is implemented using the @code{package-input-rewriting/spec} Scheme
+procedure (@pxref{Defining Packages, @code{package-input-rewriting/spec}}).
 
 @item --with-graft=@var{package}=@var{replacement}
 This is similar to @option{--with-input} but with an important difference:
diff --git a/guix/packages.scm b/guix/packages.scm
index cd61878bcc..11c066b292 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1527,9 +1527,11 @@ (define (cut? p)
 (define* (package-input-rewriting/spec replacements #:key (deep? #t))
   "Return a procedure that, given a package, applies the given REPLACEMENTS to
 all the package graph, including implicit inputs unless DEEP? is false.
+
 REPLACEMENTS is a list of spec/procedures pair; each spec is a package
 specification such as \"gcc\" or \"guile@2\", and each procedure takes a
-matching package and returns a replacement for that package."
+matching package and returns a replacement for that package.  Matching
+packages that have the 'hidden?' property set are not replaced."
   (define table
     (fold (lambda (replacement table)
             (match replacement
@@ -1557,7 +1559,8 @@ (define replacement-property
     (gensym " package-replacement"))
 
   (define (rewrite p)
-    (if (assq-ref (package-properties p) replacement-property)
+    (if (or (assq-ref (package-properties p) replacement-property)
+            (hidden-package? p))
         p
         (match (find-replacement p)
           (#f p)
diff --git a/tests/packages.scm b/tests/packages.scm
index f58c47817b..446be6ba52 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
@@ -1577,6 +1577,24 @@ (define right-system?
     (match (delete-duplicates pythons eq?)
       ((p) (eq? p (rewrite python))))))
 
+(test-assert "package-input-rewriting/spec, hidden package"
+  ;; Hidden packages are not subject to rewriting.
+  (let* ((python  (hidden-package python))
+         (p0      (dummy-package "chbouib"
+                    (build-system trivial-build-system)
+                    (inputs (list python))))
+         (rewrite (package-input-rewriting/spec
+                   `(("python" . ,(const sed)))
+                   #:deep? #t))
+         (p1      (rewrite p0))
+         (bag1    (package->bag p1))
+         (pythons (filter-map (match-lambda
+                                (("python" python) python)
+                                (_ #f))
+                              (bag-transitive-inputs bag1))))
+    (match (delete-duplicates pythons eq?)
+      ((p) (eq? p python)))))
+
 (test-equal "package-input-rewriting/spec, graft"
   (derivation-file-name (package-derivation %store sed))
 
-- 
2.39.1

[bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Josselin Poiret via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> Hello,
>
> This change makes things like:
>
>   guix build --with-input=guile=guile-next guix -n --no-grafts
>
> more useful and tractable.
>
> Low-level rewrites are still possible for packages not marked
> as hidden in 'commencement.scm', such as glibc:
>
>   guix build --with-latest=glibc hello -n
>
> Thoughts?
>
> Ludo'.

LGTM, and tested fine.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

[bug#62047] [PATCH 1/2] packages: Use SRFI-71 instead of SRFI-11.

[Simon Tournier]

Hi Ludo,

On mer., 08 mars 2023 at 13:03, Ludovic Courtès <ludo@gnu.org> wrote:

> -               (let-values (((name version)
> -                             (package-name->name+version spec)))
> +               (let ((name version (package-name->name+version spec)))

Out of curiosity, what is the difference?  Aside SRFI-71 instead of
SRFI-11?  Or why do you prefer SRFI-71 over -11?

Cheers,
simon

[bug#62047] [PATCH 1/2] packages: Use SRFI-71 instead of SRFI-11.

[Ludovic Courtès]

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On mer., 08 mars 2023 at 13:03, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> -               (let-values (((name version)
>> -                             (package-name->name+version spec)))
>> +               (let ((name version (package-name->name+version spec)))
>
> Out of curiosity, what is the difference?  Aside SRFI-71 instead of
> SRFI-11?  Or why do you prefer SRFI-71 over -11?

There’s no semantic difference, but SRFI-71 looks clearer, doesn’t it?
:-)

See
<https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00234.html>
for the original proposal.

Ludo’.

[bug#62047] [PATCH 1/2] packages: Use SRFI-71 instead of SRFI-11.

[Simon Tournier]

Hi,

On Fri, 10 Mar 2023 at 17:46, Ludovic Courtès <ludo@gnu.org> wrote:

> See
> <https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00234.html>
> for the original proposal.

Thanks!  I had overlooked this.

Cheers,
simon

Re: [bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Greg Hogan]

On Wed, Mar 8, 2023 at 7:03 AM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hello,
>
> This change makes things like:
>
>   guix build --with-input=guile=guile-next guix -n --no-grafts
>
> more useful and tractable.
>
> Low-level rewrites are still possible for packages not marked
> as hidden in 'commencement.scm', such as glibc:
>
>   guix build --with-latest=glibc hello -n
>
> Thoughts?
>
> Ludo'.
>
> Ludovic Courtès (2):
>   packages: Use SRFI-71 instead of SRFI-11.
>   packages: 'package-input-rewriting/spec' ignores hidden packages.
>
>  doc/guix.texi      | 21 ++++++++++++---------
>  guix/packages.scm  | 14 ++++++++------
>  tests/packages.scm | 20 +++++++++++++++++++-
>  3 files changed, 39 insertions(+), 16 deletions(-)
>
>
> base-commit: 0a37921d851b94aef6900214098dc5bd62b46e26
> --
> 2.39.1

(resending to guix-devel since the bug has been archived)

Ludo',

This has broken, for example, building clang with a newer version of
gcc using package-input-rewriting/spec. What do you think of adding a
hidden? property to enable the old behavior?

Alternatively, why are gcc, binutils, and libc used as build-time
dependencies rather than gcc-toolchain? gcc-toolchain could be
rewritten as a non-hidden package and use of the toolchain would
seemingly better support profiles created with
package->development-manifest.

Greg

Re: [bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

Hi Greg,

Greg Hogan <code@greghogan.com> writes:

> This has broken, for example, building clang with a newer version of
> gcc using package-input-rewriting/spec. What do you think of adding a
> hidden? property to enable the old behavior?

Since you're mentioning package-input-rewriting/spec, I assume you're
doing this in Guile?  If so, package-input-rewriting doesn't have that
same behavior, and still replaces hidden packages!

HTH,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: [bug#62047] [PATCH 0/2] '--with-input' & co. no longer replace hidden packages

[Ludovic Courtès]

Hi Greg,

(It’s been two months but I completely missed this message.)

Greg Hogan <code@greghogan.com> skribis:

> On Wed, Mar 8, 2023 at 7:03 AM Ludovic Courtès <ludo@gnu.org> wrote:
>>
>> Hello,
>>
>> This change makes things like:
>>
>>   guix build --with-input=guile=guile-next guix -n --no-grafts
>>
>> more useful and tractable.
>>
>> Low-level rewrites are still possible for packages not marked
>> as hidden in 'commencement.scm', such as glibc:
>>
>>   guix build --with-latest=glibc hello -n
>>
>> Thoughts?
>>
>> Ludo'.
>>
>> Ludovic Courtès (2):
>>   packages: Use SRFI-71 instead of SRFI-11.
>>   packages: 'package-input-rewriting/spec' ignores hidden packages.

[...]

> This has broken, for example, building clang with a newer version of
> gcc using package-input-rewriting/spec.

As in:

   guix build clang --with-c-toolchain=clang=gcc-toolchain@12

?  Or some other command?

> What do you think of adding a hidden? property to enable the old
> behavior?

Maybe, but I’m not sure I fully understand the problem.

> Alternatively, why are gcc, binutils, and libc used as build-time
> dependencies rather than gcc-toolchain? gcc-toolchain could be
> rewritten as a non-hidden package and use of the toolchain would
> seemingly better support profiles created with
> package->development-manifest.

We could replace gcc, binutils, etc. with just ‘gcc-toolchain’; this
will need testing of course, but I cannot think of any issue it would
cause.  (That’s a ‘core-updates’ change though.)

Thanks,
Ludo’.