Discussion on Parameterized Packages

Discussion on Parameterized Packages

[Sarthak Shah]

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

Hello Guix!
I'll be working on bringing Parameterized Packages to Guix for GSoC 2023
under the guidance of Gábor and Pjotr. I've been a Guix user for a few
years now as it works great for Common Lisp and Scheme projects, and I've
always wanted to contribute to it as it has one of the best codebases I've
seen. Parameterized Packages will serve as an awesome feature that
leverages Guix's dedication to ensuring that all packages can be compiled
from source.
Parameterized Packages will introduce functionality similar to Gentoo's USE
flags, making it possible to change compile-time options for packages. This
will provide users with a lot more freedom over what features they'd like
to include or exclude from packages, and also aid with reducing the size of
binaries.
I have provided a detailed outline of parameterized packages and the
proposed user interface for interacting with them (for both users and
maintainers) in this post on my blog:
https://blog.lispy.tech/2023/05/parameterized-packages.html
I would really appreciate feedback on
(1) parameters you'd like to see in Guix
(2) the user interface for searching/installing/packaging with parameters
A lot of these details are still in a draft phase and thus any suggestions
that could help polish them would be awesome!
Here's a list of parameters I think would be cool to have:
graphics-related: x11, gtk, qt, motif etc.
media-related: png, jpeg, mp4, flac etc.
playback-related: pulseaudio, alsa, jack etc.
locales: en_us, ja_jp etc.
(possibly) optimizations: jit, o3, ofast, lto, graphite etc.
fundamental: unicode, debug, examples etc.

Please let me know what you think!
Happy Hacking,
Sarthak.

[-- Attachment #2: Type: text/html, Size: 2041 bytes --]

Re: Discussion on Parameterized Packages

[Csepp]

Sarthak Shah <shahsarthakw@gmail.com> writes:

> Hello Guix!
> I'll be working on bringing Parameterized Packages to Guix for GSoC 2023 under the guidance of Gábor and Pjotr. I've been a Guix user for a
> few years now as it works great for Common Lisp and Scheme projects, and I've always wanted to contribute to it as it has one of the best
> codebases I've seen. Parameterized Packages will serve as an awesome feature that leverages Guix's dedication to ensuring that all packages
> can be compiled from source.
> Parameterized Packages will introduce functionality similar to Gentoo's USE flags, making it possible to change compile-time options for
> packages. This will provide users with a lot more freedom over what features they'd like to include or exclude from packages, and also aid
> with reducing the size of binaries.
> I have provided a detailed outline of parameterized packages and the proposed user interface for interacting with them (for both users and
> maintainers) in this post on my blog: https://blog.lispy.tech/2023/05/parameterized-packages.html
> I would really appreciate feedback on
> (1) parameters you'd like to see in Guix

A hardening flag would be very welcome, I think Guix definitely needs to
be much more serious about this if we want it to be used widely for
hosting internet facing services.
NixOS already has hardening by deafult as far as I know.

Also I think a lot of flags you mentioned would be better expressed as
outputs, both to avoid combinatorial explosion for testing, and so
people don't have to re-build everything.  Or at least they should be
implemented with outputs underneath.

I think Alpine solve this very well, they split packages up by
functionality, and also make it very easy to install certain subpackages
of all packages by just installing a virtual package, like "doc".

The idea of using flags to reduce package sizes is good, but let's not
do it at the expense of CPU hours.

TLDR: we should avoid big recompiles when changing flags.

Re: Discussion on Parameterized Packages

[Liliana Marie Prikler]

Am Donnerstag, dem 11.05.2023 um 20:38 +0530 schrieb Sarthak Shah:
> Hello Guix!
> I'll be working on bringing Parameterized Packages to Guix for GSoC
> 2023 under the guidance of Gábor and Pjotr. I've been a Guix user for
> a few years now as it works great for Common Lisp and Scheme
> projects, and I've always wanted to contribute to it as it has one of
> the best codebases I've seen. Parameterized Packages will serve as an
> awesome feature that leverages Guix's dedication to ensuring that all
> packages can be compiled from source.
> Parameterized Packages will introduce functionality similar to
> Gentoo's USE flags, making it possible to change compile-time options
> for packages. This will provide users with a lot more freedom over
> what features they'd like to include or exclude from packages, and
> also aid with reducing the size of binaries.
> I have provided a detailed outline of parameterized packages and the
> proposed user interface for interacting with them (for both users and
> maintainers) in this post on my blog:
> https://blog.lispy.tech/2023/05/parameterized-packages.html
> I would really appreciate feedback on
> (1) parameters you'd like to see in Guix
Having read your blog and skimmed ahead a little, preferably none with
the way things are currently phrased.  That doesn't mean I am against
parameters, but I think it's important to understand them as a package-
local transformations.  Failure to do so is imho one of the biggest
flaws in Gentoo's USE flags.

First of all, let's address the elephant in the room: tunable?  While
implemented as a package property, that's actually half a parameter. 
The tuned package specifies which microarchitecture to optimize the
package for.  I think long-term, tunable? should be turned into a
parameter.

Secondly, let's have a look at Emacs.
  (package
     (name "emacs")
     (parameters 
       (optional jit #t)
       (optional png #t)
       (optional alsa #t)
       (one-of window-system gtk pure-gtk motif none))
     ...)
looks much nicer than the proposed mix of meaningful names with
symbols.  Note, that (one-of ...) takes the first value as implicit
default.

Now you could argue that specifying 20 optional parameters like this
would be a pain in the butt and you prefer your own optional notation.
But have you considered the following syntax as an alternative:
  (optionals/enabled jit png alsa)
  (optional treesit (some-supported-system-check))
  (optionals/disabled xwidgets wide-int ...)

> (2) the user interface for searching/installing/packaging with
> parameters
As with outputs, available parameters ought to be shown in the output
of `guix show', preferably one parameter per line with an explanation
what that parameter does.  Note that this isn't supported by either
mockup however, as just like with outputs a place for the documentation
is missing.

For the `guix package' CLI, I'd like to have two options: 
  --with-parameter=PACKAGE=PARAMETER[=VALUE] enables a parameter 
    or assigns a specific value to a one-of parameter
  --without-parameter=PACKAGE=PARAMETER disables a parameter
If you want to be fancy, you can do 
  --with-parameters=PACKAGE=PSPEC[,PSPEC...]
where PSPEC is one of
  "not PARAMETER" to disable PARAMETER,
  "PARAMETER" to enable it,
  "PARAMETER=VALUE" to set a specific one-of value
which internally gets lowered to the same specification as equivalent
--with-parameter and --without-parameter arguments. 

To support "global" parameters, you could allow PACKAGE to be a regular
expression or if that's too much work, simply allow "*" as a special
package matching all packages.  Again, note that I'm not very fond of
global parameters.

For the operating-system side of things, I'd prefer it if we didn't
gain yet another field to care about.  I think modify-services should
work well in combination with package transformations, wherein you
simply 
  (define our-package-parameters
    '((with-parameter "emacs=window-system=none")
      (without-parameter "emacs=jit")
      ...))
ahead of time and then use 
  (options->transformation our-package-parameters)
in the right location.


Cheers

Re: Discussion on Parameterized Packages

[david larsson]

On 2023-05-11 17:08, Sarthak Shah wrote:
> Hello Guix!
> I'll be working on bringing Parameterized Packages to Guix for GSoC
> 2023 under the guidance of Gábor and Pjotr. I've been a Guix user for
> a few years now as it works great for Common Lisp and Scheme projects,
> and I've always wanted to contribute to it as it has one of the best
> codebases I've seen. Parameterized Packages will serve as an awesome
> feature that leverages Guix's dedication to ensuring that all packages
> can be compiled from source.
> Parameterized Packages will introduce functionality similar to
> Gentoo's USE flags, making it possible to change compile-time options
> for packages. This will provide users with a lot more freedom over
> what features they'd like to include or exclude from packages, and
> also aid with reducing the size of binaries.
> I have provided a detailed outline of parameterized packages and the
> proposed user interface for interacting with them (for both users and
> maintainers) in this post on my blog:
> https://blog.lispy.tech/2023/05/parameterized-packages.html
> 
> I would really appreciate feedback on
> 
> (1) parameters you'd like to see in Guix
> 
> (2) the user interface for searching/installing/packaging with
> parameters

Im not sure this belongs as a parameter per se, but I would like to see 
something like a stable "parameter", that only uses stable features and 
stable versions of packages when applied on an operating system level 
(which was suggested as being a possible level to apply at in the blog 
post), thus:

   "guix build python-SOMEPACKAGE --with-parameter=stable"

would recursively use only the stable version of the package, including 
stable version of the dependencies.

If this were a thing, Guix could avoid an LTS version, and just run 
GuixLTS via package-parameters - which would be, uhm fun, IMO :-)

Maybe something to put into package-properties 
(https://github.com/guix-mirror/guix/blob/270db2a56bc50bcab5739de2c9393644ab65ac6c/guix/packages.scm#L611)

Best regards,
David

Re: Discussion on Parameterized Packages

[david larsson]

On 2023-05-13 20:26, david larsson wrote:

[..]

> 
> If this were a thing, Guix could avoid an LTS version, and just run
> GuixLTS via package-parameters - which would be, uhm fun, IMO :-)

I do not mean LTS. I only meant a "stable" version, sorry for any 
confusion caused.

Best regards,
David

Re: Discussion on Parameterized Packages

[Simon Tournier]

Hi,

On jeu., 11 mai 2023 at 20:38, Sarthak Shah <shahsarthakw@gmail.com> wrote:

> https://blog.lispy.tech/2023/05/parameterized-packages.html
>
> I would really appreciate feedback on
> (1) parameters you'd like to see in Guix
> (2) the user interface for searching/installing/packaging with
> parameters

Just a quick remark.  You are proposing something like:

--8<---------------cut here---------------start------------->8---
 1  (define-public emacs  
 2    (package  
 3      (parameters (and  
 4        (optional jit^ png^ alsa^)  
 5        (one-of motif gtk^ x11!*)))  
 6      (parameter-transforms  
 7        ((x11!)  
 8         (changes-to-be-made-to-the-package)))))  
--8<---------------cut here---------------end--------------->8---

or other variants.  Well, I am a bit afraid by the maintenance of such
packages.  The combinatorial complexity will be exploding and it will be
harder to update such packages, IMHO.

Instead, I would go with something similar as ’package/inherit’.  For
instance, something like that:

--8<---------------cut here---------------start------------->8---
(define-public emacs-params
  (package/parametrized emacs
    (parameters (and
                 (optional jit^ png^ alsa^)
                 (one-of motif gtk^ x11!*)))
    (parameter-transforms
     ((x11!)
      (changes-to-be-made-to-the-package)))))
--8<---------------cut here---------------end--------------->8---

Well, from my point of view, these parametrized packages could go to
specific modules (or channels).  And keeping them separate would avoid
nightmares about maintenance – it is already enough complex without
parameters. :-)

Cheers,
simon