unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* A plan for parameterized packages
@ 2020-11-15 16:33 Ludovic Courtès
  2020-11-15 17:30 ` Nicolò Balzarotti
                   ` (3 more replies)
  0 siblings, 4 replies; 24+ messages in thread
From: Ludovic Courtès @ 2020-11-15 16:33 UTC (permalink / raw)
  To: guix-devel


[-- Attachment #1.1: Type: text/plain, Size: 4426 bytes --]

Hello Guix!

For some time we’ve discussed ways to achieve “parameterized
packages”—packages where one can from the command line or from Scheme
configure optional build-time features, similar to Gentoo “USE flags”.

I still have mixed feeling about this feature: on one hand it can bring
much welcome flexibility, but on the other hand it can also lead us to
the wild west of untested package configurations and combinatorial
explosion.  It could also be argued that we achieve something similar
today by simply defining package variants when we have to:

  https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html

That said, this message is about a possible implementation of package
parameters, so here we go.  :-)

To me the requirements for package parameters are:

  1. it must be possible to discover them and choose them from the UI;

  2. they must contain on-line internationalized documentation such that
     the UI can list a package’s parameters and their type;

  3. the chosen parameters when installing a package in a profile must
     be preserved;

  4. it must be possible to enumerate all the possible values of a
     parameter, and thus to build the Cartesian product of all the
     possible parameter combinations of a package (or of a package
     graph!), so we can test those combinations as much as possible.

This leads to the patches below.  The last one gives an example use for
Bitlbee: it adds a “libpurple” parameter that allows users to choose
whether or not to enable the optional libpurple dependency:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=true -n
The following derivation would be built:
   /gnu/store/mkknqgjsa93ajcl5d2krngizb11j1b0q-bitlbee-3.6.drv
$ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=false -n
/gnu/store/c2ckg51ffwgs6jni3l549k06w3jd3b7a-bitlbee-3.6
$ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=wat? -n
guix build: error: wrong value
$ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libviolet=true -n
gnu/packages/messaging.scm:283:5: error: libviolet: no such package parameter
$ ./pre-inst-env guix show bitlbee
name: bitlbee
version: 3.6
outputs: out
parameters: libpurple
systems: x86_64-linux i686-linux
dependencies: check@0.12.0 glib@2.62.6 gnutls@3.6.12 libotr@4.1.1 perl@5.30.2 pkg-config@0.29.2 python@3.8.2
location: gnu/packages/messaging.scm:243:2
homepage: https://www.bitlbee.org/
license: GPL 2+, FreeBSD
synopsis: IRC to instant messaging gateway  
description: BitlBee brings IM (instant messaging) to IRC clients, for people who have an IRC client running all
+ the time and don't want to run an additional IM client.  BitlBee currently supports XMPP/Jabber (including Google
+ Talk), MSN Messenger, Yahoo! Messenger, AIM and ICQ, and the Twitter microblogging network (plus all other Twitter
+ API compatible services like identi.ca and status.net).

$ ./pre-inst-env guix install bitlbee --with-parameter=bitlbee=libpurple=true -p /tmp/test-bitlbee
The following package will be installed:
   bitlbee 3.6

The following derivation will be built:
   /gnu/store/clvs5521v5ybdw1z1yh97z2ky1dxm4d9-bitlbee-3.6.drv

[...]

$ cat /tmp/test-bitlbee/manifest
;; This file was automatically generated and is for internal use only.
;; It cannot be passed to the '--manifest' option.

(manifest
  (version 3)
  (packages
    (("bitlbee"
      "3.6"
      "out"
      "/gnu/store/d67r9k5hwfm5hkd1d3pbhg49fcc2jj4q-bitlbee-3.6"
      (propagated-inputs ())
      (search-paths ())
      (properties
        (transformations
          (with-parameter . "bitlbee=libpurple=true")))))))
--8<---------------cut here---------------end--------------->8---

That’s it!

We would need more things, like a ‘guix parameters’ command to show the
available parameters of a package and an ‘--all-parameter-values’ option
to ‘guix build’ to build all the variants of a given package.

An important question: do we have examples of packages for which we’d
like to have parameters?  I’d grepped for “inherit” and that yields a
few potential candidates, but also maybe a few potential non-candidates.
Would this be a good fit for them?

Thoughts?

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0003-ui-package-recutils-emits-parameters-field.patch --]
[-- Type: text/x-patch, Size: 1414 bytes --]

From 9155411f2e8e78922e1e46d92068ac8f652ff0a5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Sun, 15 Nov 2020 17:01:14 +0100
Subject: [PATCH 3/4] ui: 'package->recutils' emits "parameters" field.

* guix/ui.scm (package->recutils): Add "parameters" recutils field.
---
 guix/ui.scm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/guix/ui.scm b/guix/ui.scm
index 4e686297e8..2485400cc9 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -60,6 +60,7 @@
                 #:hide (package-name->name+version
                         ;; Avoid "overrides core binding" warning.
                         delete))
+  #:autoload   (guix parameters) (package-parameters package-parameter-name)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-11)
@@ -1529,6 +1530,10 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate."
   (format port "name: ~a~%" (package-name p))
   (format port "version: ~a~%" (package-version p))
   (format port "outputs: ~a~%" (string-join (package-outputs p)))
+  (match (package-parameters p)
+    (() #t)
+    (lst (format port "parameters:~{ ~a~}~%"
+                 (map package-parameter-name lst))))
   (format port "systems: ~a~%"
           (string-join (package-transitive-supported-systems p)))
   (format port "dependencies: ~a~%"
-- 
2.29.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0004-gnu-bitlbee-Add-libpurple-parameter.patch --]
[-- Type: text/x-patch, Size: 3823 bytes --]

From 49d7746ada4d4674acbbfd2606ad9bff4f6207eb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Sun, 15 Nov 2020 17:04:18 +0100
Subject: [PATCH 4/4] gnu: bitlbee: Add "libpurple" parameter.

* gnu/packages/messaging.scm (bitlbee)[inputs]: Add optional PIDGIN
input.
[properties]: New field.
(bitlbee-purple): Mark as deprecated.
---
 gnu/packages/messaging.scm | 43 +++++++++++++++++++-------------------
 1 file changed, 22 insertions(+), 21 deletions(-)

diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm
index b462504894..2f0f44d10d 100644
--- a/gnu/packages/messaging.scm
+++ b/gnu/packages/messaging.scm
@@ -120,6 +120,7 @@
   #:use-module (guix build-system trivial)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix parameters)
   #:use-module (guix hg-download)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
@@ -256,7 +257,8 @@ end-to-end encryption.")
               ("libotr" ,libotr)
               ("gnutls" ,gnutls)
               ("python" ,python)
-              ("perl" ,perl)))
+              ("perl" ,perl)
+              ,@(optionally 'libpurple? `("purple" ,pidgin))))
     (arguments
      `(#:phases
        (modify-phases %standard-phases
@@ -275,7 +277,21 @@ end-to-end encryption.")
              (invoke "./configure"
                      (string-append "--prefix="
                                     (assoc-ref outputs "out"))
-                     "--otr=1"))))))
+                     "--otr=1"
+                     ,@(optionally 'libpurple? "--purple=1")))))
+
+       ;; XXX: Tests fail to link, and ./configure says that it's "supported
+       ;; on a best-effort basis" anyway.
+       #:tests? ,(not (assq-ref (package-properties this-package)
+                                'libpurple?))))
+    (properties
+     `((parameters
+        ,(package-parameter (name "libpurple")
+                            (description
+                             "Whether to enable libpurple (Pidgin)
+support.")
+                            (property 'libpurple?)
+                            (type boolean)))))
     (synopsis "IRC to instant messaging gateway")
     (description "BitlBee brings IM (instant messaging) to IRC clients, for
 people who have an IRC client running all the time and don't want to run an
@@ -289,25 +305,10 @@ identi.ca and status.net).")
 (define-public bitlbee-purple
   ;; This variant uses libpurple, which provides support for more protocols at
   ;; the expense of a much bigger closure.
-  (package/inherit bitlbee
-    (name "bitlbee-purple")
-    (synopsis "IRC to instant messaging gateway (using Pidgin's libpurple)")
-    (inputs `(("purple" ,pidgin)
-              ,@(package-inputs bitlbee)))
-    (arguments
-     (substitute-keyword-arguments (package-arguments bitlbee)
-       ((#:phases phases '%standard-phases)
-        `(modify-phases ,phases
-           (replace 'configure                    ;add "--purple=1"
-             (lambda* (#:key outputs #:allow-other-keys)
-               (invoke "./configure"
-                       (string-append "--prefix="
-                                      (assoc-ref outputs "out"))
-                       "--otr=1" "--purple=1")))))
-       ((#:tests? _ #t)
-        ;; XXX: Tests fail to link, and ./configure says that it's "supported
-        ;; on a best-effort basis" anyway.
-        #f)))))
+  (deprecated-package "bitlbee-purple"
+                      (package/inherit bitlbee
+                        (properties `((libpurple? . #t)
+                                      ,@(package-properties bitlbee))))))
 
 (define-public bitlbee-discord
   (package
-- 
2.29.2


[-- Attachment #1.4: 0001-DRAFT-Add-guix-parameters.patch --]
[-- Type: text/x-patch, Size: 6318 bytes --]

From f42b68499c4e2a9bd368fe6a516932f5afa7a189 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Sun, 15 Nov 2020 16:58:52 +0100
Subject: [PATCH 1/4] DRAFT Add (guix parameters).

DRAFT: Missing tests & doc.

* guix/parameters.scm: New file.
* Makefile.am (MODULES): Add it.
---
 Makefile.am         |   1 +
 guix/parameters.scm | 131 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 guix/parameters.scm

diff --git a/Makefile.am b/Makefile.am
index e7053ee4f4..72f955360d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -235,6 +235,7 @@ MODULES =					\
   guix/build/make-bootstrap.scm			\
   guix/search-paths.scm				\
   guix/packages.scm				\
+  guix/parameters.scm				\
   guix/import/cabal.scm				\
   guix/import/cpan.scm				\
   guix/import/cran.scm				\
diff --git a/guix/parameters.scm b/guix/parameters.scm
new file mode 100644
index 0000000000..e4f8240aa4
--- /dev/null
+++ b/guix/parameters.scm
@@ -0,0 +1,131 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix parameters)
+  #:use-module (guix packages)
+  #:use-module (guix records)
+  #:use-module (guix diagnostics)
+  #:use-module (guix i18n)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:use-module (ice-9 match)
+  #:export (package-parameter
+            package-parameter?
+            package-parameter-name
+            package-parameter-property
+            package-parameter-type
+            package-parameter-description
+
+            boolean
+            optionally
+
+            package-parameters
+            lookup-package-parameter
+            package-parameter-value
+            set-package-parameter-value))
+
+;;; Commentary:
+;;;
+;;; This module provides a way to express high-level "package parameters",
+;;; which allow users to customize how packages are built.  Parameters are an
+;;; interface that package developers define, where each parameter has a name
+;;; and type.  The user interface then converts parameter values from string
+;;; to Scheme values and records them in the package properties.
+;;;
+;;; Package parameters are discoverable; their description is
+;;; internationalized.  The possible values of a parameter can be enumerated,
+;;; and thus the Cartesian product of all possible parameter values for a
+;;; package can be enumerated as well.
+;;;
+;;; Code:
+
+;; Package parameter interface.
+(define-record-type* <package-parameter> package-parameter
+  make-package-parameter
+  package-parameter?
+  (name          package-parameter-name)
+  (property      package-parameter-property (default (string->symbol name)))
+  (type          package-parameter-type)
+  (description   package-parameter-description))
+
+;; Type of a package parameter.
+(define-record-type* <parameter-type> parameter-type
+  make-parameter-type
+  parameter-type?
+  (name          parameter-type-name)              ;debugging purposes only!
+  (string->value parameter-type-string->value)
+  (value->string parameter-type-value->string)
+  (universe      parameter-type-universe))
+
+(define boolean
+  ;; The Boolean parameter type.
+  (parameter-type (name 'boolean)
+                  (universe '(#true #false))
+                  (value->string
+                   (match-lambda
+                     (#f "false")
+                     (#t "true")))
+                  (string->value
+                   (lambda (str)
+                     (cond ((string-ci=? str "true")
+                            #t)
+                           ((string-ci=? str "false")
+                            #f)
+                           (else
+                            (raise (condition
+                                    (&message (message "wrong value"))))))))))
+
+(define (package-parameters package)
+  (or (assq-ref (package-properties package) 'parameters)
+      '()))
+
+(define (package-parameter-value package parameter)
+  (assq-ref (package-properties package)
+            (package-parameter-property parameter)))
+
+(define (lookup-package-parameter package name)
+  (find (lambda (parameter)
+          (string=? (package-parameter-name parameter) name))
+        (package-parameters package)))
+
+(define (set-package-parameter-value package name value)
+  (let ((parameter (lookup-package-parameter package name))
+        (location  (package-field-location package 'properties)))
+    (unless parameter
+      (raise (apply make-compound-condition
+                    (formatted-message
+                     (G_ "~a: no such package parameter")
+                     name)
+                    (if location
+                        (list (condition
+                               (&error-location (location location))))
+                        '()))))
+    (let* ((property (package-parameter-property parameter))
+           (type     (package-parameter-type parameter))
+           (value    ((parameter-type-string->value type) value)))
+      (package/inherit package
+        (properties
+         (alist-cons property value
+                     (alist-delete property (package-properties package)
+                                   eq?)))))))
+
+(define-syntax-rule (optionally property exp)
+  (if (assq-ref (package-properties this-package) property)
+      (list exp)
+      '()))
-- 
2.29.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.5: 0002-DRAFT-transformations-Add-with-parameter.patch --]
[-- Type: text/x-patch, Size: 3849 bytes --]

From 5d6d81e4c3e37de53fe7f62ff7ef94da8b2df033 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Sun, 15 Nov 2020 16:59:48 +0100
Subject: [PATCH 2/4] DRAFT transformations: Add '--with-parameter'.

DRAFT: Missing tests & doc.

* guix/transformations.scm (evaluate-parameter-specs)
(transform-package-parameters): New procedures.
(%transformations, %transformation-options): Add 'with-parameter'.
---
 guix/transformations.scm | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/guix/transformations.scm b/guix/transformations.scm
index 30142dd059..0f83eb470d 100644
--- a/guix/transformations.scm
+++ b/guix/transformations.scm
@@ -25,6 +25,7 @@
   #:autoload   (guix download) (download-to-store)
   #:autoload   (guix git-download) (git-reference? git-reference-url)
   #:autoload   (guix git) (git-checkout git-checkout? git-checkout-url)
+  #:autoload   (guix parameters) (set-package-parameter-value)
   #:use-module (guix utils)
   #:use-module (guix memoization)
   #:use-module (guix gexp)
@@ -324,6 +325,41 @@ a checkout of the Git repository at the given URL."
         (rewrite obj)
         obj)))
 
+(define (evaluate-parameter-specs specs proc)
+  "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a
+list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called
+to return the replacement package.  Raise an error if an element of SPECS uses
+invalid syntax, or if a package it refers to could not be found."
+  (map (lambda (spec)
+         (match (string-tokenize spec %not-equal)
+           ((spec name value)
+            (define (replace old)
+              (proc old name value))
+
+            (cons spec replace))
+           (_
+            (raise
+             (formatted-message
+              (G_ "invalid package parameter specification: ~s")
+              spec)))))
+       specs))
+
+(define (transform-package-parameters replacement-specs)
+  "Return a procedure that, when passed a package, replaces its direct
+dependencies according to REPLACEMENT-SPECS.  REPLACEMENT-SPECS is a list of
+strings like \"guile-next=stable-3.0\" meaning that packages are built using
+'guile-next' from the latest commit on its 'stable-3.0' branch."
+  (define (replace old name value)
+    (set-package-parameter-value old name value))
+
+  (let* ((replacements (evaluate-parameter-specs replacement-specs
+                                                 replace))
+         (rewrite      (package-input-rewriting/spec replacements)))
+    (lambda (obj)
+      (if (package? obj)
+          (rewrite obj)
+          obj))))
+
 (define (package-dependents/spec top bottom)
   "Return the list of dependents of BOTTOM, a spec string, that are also
 dependencies of TOP, a package."
@@ -467,6 +503,7 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field."
     (with-branch . ,transform-package-source-branch)
     (with-commit . ,transform-package-source-commit)
     (with-git-url . ,transform-package-source-git-url)
+    (with-parameter . ,transform-package-parameters)
     (with-c-toolchain . ,transform-package-toolchain)
     (with-debug-info . ,transform-package-with-debug-info)
     (without-tests . ,transform-package-tests)))
@@ -503,6 +540,8 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field."
                   (parser 'with-commit))
           (option '("with-git-url") #t #f
                   (parser 'with-git-url))
+          (option '("with-parameter") #t #f
+                  (parser 'with-parameter))
           (option '("with-c-toolchain") #t #f
                   (parser 'with-c-toolchain))
           (option '("with-debug-info") #t #f
-- 
2.29.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

^ permalink raw reply related	[flat|nested] 24+ messages in thread
end of thread, other threads:[~2020-11-20 19:45 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-11-15 16:33 A plan for parameterized packages Ludovic Courtès
2020-11-15 17:30 ` Nicolò Balzarotti
2020-11-15 17:40   ` Nicolò Balzarotti
2020-11-15 17:44   ` Pierre Neidhardt
2020-11-15 18:09     ` zimoun
2020-11-16 11:50     ` Ludovic Courtès
2020-11-16 12:03       ` Pierre Neidhardt
2020-11-16 14:05         ` zimoun
2020-11-15 17:37 ` zimoun
2020-11-16 11:54   ` Ludovic Courtès
2020-11-15 18:51 ` Taylan Kammer
2020-11-15 20:46 ` Danny Milosavljevic
2020-11-15 21:16   ` zimoun
2020-11-16 11:25     ` Make mutiple packages from outputs (Was: A plan for parameterized packages) 宋文武
2020-11-16 14:53       ` Make mutiple packages from outputs Ludovic Courtès
2020-11-16 15:10       ` Make mutiple packages from outputs (Was: A plan for parameterized packages) zimoun
2020-11-15 21:24   ` A plan for parameterized packages raingloom
2020-11-16  1:54     ` Ryan Prior
2020-11-16  5:38       ` Clozure size zimoun
2020-11-18  1:30     ` A plan for parameterized packages Denis 'GNUtoo' Carikli
2020-11-20 11:39       ` Ludovic Courtès
2020-11-20 14:38         ` zimoun
2020-11-20 19:44         ` Christopher Baines
2020-11-16 14:51   ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).