[RFC] package-with-features

[RFC] package-with-features

[(]

Heya Guix,

This comment by oriansj on IRC:

  <oriansj> I am thinking in terms of gentoo builds and making
  it easy to avoid some packages from being downloaded or built
  like pulseaudio (I like alsa better) and trim down the dependencies
  to only those that are absolutely directly required.

made me wonder how we could incorporate such a feature into
Guix.

# Why it's Needed

Sure, it's already possible to create package variants:

  (define-public foo
    (package
      (name "foo")
      (version "1.2.3")
      (source (origin
               ...))
      (build-system gnu-build-system)
      (arguments
       (list #:configure-flags
             #~(list "--enable-pulseaudio")))
      (inputs (list pulseaudio ...))
      ...))

  (define-public foo/without-pulseaudio
    (package
      (inherit foo)
      (name "foo-without-pulseaudio")
      (arguments
       (substitute-keyword-arguments (package-arguments foo)
         ((#:configure-flags old-flags)
          #~(list #$@flags
                  "--disable-pulseaudio"))))
      (inputs
       (modify-inputs (package-inputs foo)
         (delete pulseaudio)))))

This is currently manageable, but what if there's another optional
feature, say, pipewire support?

  (define-public foo/without-pipewire
    (package
      (inherit foo)
      (name "foo-without-pulseaudio")
      (arguments
       (substitute-keyword-arguments (package-arguments foo)
         ((#:configure-flags old-flags)
          #~(list #$@flags
                  "--disable-pipewire"))))
      (inputs
       (modify-inputs (package-inputs foo)
         (delete pipewire)))))

And it's entirely reasonable to want to disable both and just use
Ye Olde ALSA...

  (define-public foo/without-pipewire-or-pulse
    (package
      (inherit foo/without-pipewire)
      (name "foo-without-pulseaudio-or-pulse")
      (arguments
       (substitute-keyword-arguments
           (package-arguments foo/without-pipewire)
         ((#:configure-flags old-flags)
          #~(list #$@flags
                  "--disable-pulseaudio"))))
      (inputs
       (modify-inputs (package-inputs foo/without-pipewire)
         (delete pulseaudio)))))

We only have two features, pulseaudio and pipewire, and it's already
getting A Wee Bit Silly. Even worse if we have 3 features:

  /without-pipewire
  /without-pulse
  /without-jack
  /without-pipewire-or-pulse
  /without-pipewire-or-jack
  /without-pulse-or-jack
  /without-pipewire-pulse-or-jack

And now there are seven variants, and eight ``foo'' packages in total.
We can do better, surely? Here's my proposal: a new ``features'' field
for ``package'' that accepts a list of records like this:

  (define-public foo
    (package
      ...
      (features
       (list (feature
              (name "jack")
              (default? #f)
              (description "JACK audio backend"))
             (feature
              (name "pipewire")
              (default? #t)
              (description "Pipewire audio backend"))
             (feature
              (name "pulseaudio")
              (default? #t)
              (description "PulseAudio audio backend"))))
      ...))

And then we'd simply have a ``feature?'' (or some other name) procedure
that we can use in a package definition.

  (define-public foo
    (package
      ...
      (arguments
       (list #:configure-flags
             #~(list (if #$(feature? "jack")
                         "--enable-jack"
                         "--disable-jack")
                     (if #$(feature? "pipewire")
                         "--enable-pipewire"
                         "--disable-pipewire")
                     (if #$(feature? "pulseaudio")
                         "--enable-pulseaudio"
                         "--disable-pulseaudio"))))
      (inputs
       (append (list alsa-lib)
               (if (feature? "jack")
                   (list jack)
                   '())
               (if (feature? "pipewire")
                   (list pipewire)
                   '())
               (if (feature? "pulseaudio")
                   (list pulseaudio)
                   '())))
      ...))

# Features and the CLI

When you ``guix show'' this package, it would display something like
this:

  name: foo
  version: 0.2.7
  outputs:
  + out: everything
  features:
  + jack (disabled): JACK audio backend
  + pipewire (enabled): Pipewire audio backend
  + pulseaudio (enabled): PulseAudio audio backend
  systems: x86_64-linux
  dependencies: alsa-lib@... jack[jack]@...
  + pipewire[pipewire]@... pulseaudio[pulseaudio]@...
  location: ...
  homepage: ...
  license: ...
  synopsis: ...
  description: ...

Note those square brackets; this is how we specify features in package
specifications, as they are treated as normal characters in POSIX-like
shells and Fish (probably most others, too).

This installs foo without the pulseaudio feature.

  guix install foo[!pulseaudio]

This installs foo with the jack feature and without the pipewire feature.

  guix install foo[jack,!pipewire]

This installs foo with all optional features disabled.

  guix install foo[!all]

And this installs foo with all optional features enabled.

  guix install foo[all]

(The all feature is special, much like out is with outputs.)

Now, what if we have bar, which depends on foo, and we want to disable
pulseaudio for bar's foo dependency?

  guix install bar[foo[!pulseaudio]]

Or we want to disable it for all foos in the transitive dependencies
of bar?

  guix install bar[@foo[!pulseaudio]]

# Feature APIs

Obviously there's FEATURE? and the FEATURE record-type operators:

  (feature ...)
  (feature-name feature)
  (feature-default? feature)
  (feature-description feature)
  (feature? feature-name)

There's also THIS-PACKAGE-FEATURE, which returns the feature record
for a given name string:

  (this-package-feature feature-name)

There's PACKAGE-WITH-FEATURES:

  (package-with-features FEATURES PACKAGE)

And PACKAGE-INPUT-FEATURE-REWRITING:

  (package-input-feature-rewriting REPLACEMENTS)

For example:

  ((package-input-feature-rewriting
    `((,foo "!pipewire" "jack")))
   bar)

Maybe we could also have a macro for PACKAGE-INPUT-FEATURE-REWRITING
(and another for PACKAGE-INPUT-REWRITING, for orthogonality)?

  (with-package-features baz
    (foo => "!pulseaudio" "jack")
    (bar => "vulkan"))

  (with-package-replacements (list aparte irssi)
    (openssl => libressl))

SPECIFICATION->PACKAGE and SPECIFICATION->PACKAGE+OUTPUT would have
#:SPECIFY-FEATURES? flags:

  (specification->package "foo[!pipewire]" #:specify-features? #t)

# Unanswered Questions

- Should CI try to build at least some non-default feature permutations?
  (Probably not.)
- Might there be a better syntax for features in package specs?
- Is this actually a good idea? :P
- Anything else...?

    -- (

Re: [RFC] package-with-features

[zimoun]

Hi,

On Thu, 24 Nov 2022 at 20:26, "(" <paren@disroot.org> wrote:

> This comment by oriansj on IRC:
>
>   <oriansj> I am thinking in terms of gentoo builds and making
>   it easy to avoid some packages from being downloaded or built
>   like pulseaudio (I like alsa better) and trim down the dependencies
>   to only those that are absolutely directly required.
>
> made me wonder how we could incorporate such a feature into
> Guix.

Maybe, you could be interested by past discussions:

Parameterized packages (May 2019)
id:87woitz1xx.fsf@gnu.org
https://yhetil.org/guix/87woitz1xx.fsf@gnu.org/

A plan for parameterized packages (Nov 2020)
id:87eeku8trb.fsf@gnu.org
https://yhetil.org/guix/87eeku8trb.fsf@gnu.org


> # Unanswered Questions
>
> - Should CI try to build at least some non-default feature permutations?
>   (Probably not.)

From my understanding, one of the issue of the unmanageable number of
combinations.  Well, I am doubtful we (the project) would be able to
guarantee that this feature combination builds or even works.

Maybe Mathieu or Chris can comment about the CI, but to my knowledge,
the build farms are already quite busy.  Without speaking it would
require to store such resulting substitutes; which means less space for
others and so a poorer experience with “guix time-machine“ for older
Guix revisions.


> - Might there be a better syntax for features in package specs?

Maybe, the ’outputs’ mechanism could be used.  The TODO list file in the
Guix repository contains:

--8<---------------cut here---------------start------------->8---
** extend ‘propagated-build-inputs’ with support for multiple outputs

#+BEGIN_SRC scheme
  (outputs '("out" "include"))
  (propagated-build-inputs
    `(((("i1" ,p1 "o1")
        ("i2" ,p2))
       => "include")
      ("i3" ,p3)))
#+END_SRC
--8<---------------cut here---------------end--------------->8---

where the idea seems to have conditional inputs depending on the
outputs.  Somehow, this mechanism would ease to build feature variants.


Cheers,
simon

Re: [RFC] package-with-features

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> Maybe, you could be interested by past discussions:
>
> Parameterized packages (May 2019)
> id:87woitz1xx.fsf@gnu.org
> https://yhetil.org/guix/87woitz1xx.fsf@gnu.org/
>
> A plan for parameterized packages (Nov 2020)
> id:87eeku8trb.fsf@gnu.org
> https://yhetil.org/guix/87eeku8trb.fsf@gnu.org

I got sidetracked but haven’t given up on this…

Ludo’.

Re: [RFC] package-with-features

[(]

On Thu Nov 24, 2022 at 8:26 PM GMT, ( wrote:
> Now, what if we have bar, which depends on foo, and we want to disable
> pulseaudio for bar's foo dependency?
>
>   guix install bar[foo[!pulseaudio]]
>
> Or we want to disable it for all foos in the transitive dependencies
> of bar?
>
>   guix install bar[@foo[!pulseaudio]]

On second thought, I don't think this is necessary... We can just
use ``--with-input=foo=foo[!pulseaudio]''.


    -- (