[-- Attachment #1.1: Type: text/plain, Size: 4426 bytes --] Hello Guix! For some time we’ve discussed ways to achieve “parameterized packages”—packages where one can from the command line or from Scheme configure optional build-time features, similar to Gentoo “USE flags”. I still have mixed feeling about this feature: on one hand it can bring much welcome flexibility, but on the other hand it can also lead us to the wild west of untested package configurations and combinatorial explosion. It could also be argued that we achieve something similar today by simply defining package variants when we have to: https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html That said, this message is about a possible implementation of package parameters, so here we go. :-) To me the requirements for package parameters are: 1. it must be possible to discover them and choose them from the UI; 2. they must contain on-line internationalized documentation such that the UI can list a package’s parameters and their type; 3. the chosen parameters when installing a package in a profile must be preserved; 4. it must be possible to enumerate all the possible values of a parameter, and thus to build the Cartesian product of all the possible parameter combinations of a package (or of a package graph!), so we can test those combinations as much as possible. This leads to the patches below. The last one gives an example use for Bitlbee: it adds a “libpurple” parameter that allows users to choose whether or not to enable the optional libpurple dependency: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=true -n The following derivation would be built: /gnu/store/mkknqgjsa93ajcl5d2krngizb11j1b0q-bitlbee-3.6.drv $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=false -n /gnu/store/c2ckg51ffwgs6jni3l549k06w3jd3b7a-bitlbee-3.6 $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=wat? -n guix build: error: wrong value $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libviolet=true -n gnu/packages/messaging.scm:283:5: error: libviolet: no such package parameter $ ./pre-inst-env guix show bitlbee name: bitlbee version: 3.6 outputs: out parameters: libpurple systems: x86_64-linux i686-linux dependencies: check@0.12.0 glib@2.62.6 gnutls@3.6.12 libotr@4.1.1 perl@5.30.2 pkg-config@0.29.2 python@3.8.2 location: gnu/packages/messaging.scm:243:2 homepage: https://www.bitlbee.org/ license: GPL 2+, FreeBSD synopsis: IRC to instant messaging gateway description: BitlBee brings IM (instant messaging) to IRC clients, for people who have an IRC client running all + the time and don't want to run an additional IM client. BitlBee currently supports XMPP/Jabber (including Google + Talk), MSN Messenger, Yahoo! Messenger, AIM and ICQ, and the Twitter microblogging network (plus all other Twitter + API compatible services like identi.ca and status.net). $ ./pre-inst-env guix install bitlbee --with-parameter=bitlbee=libpurple=true -p /tmp/test-bitlbee The following package will be installed: bitlbee 3.6 The following derivation will be built: /gnu/store/clvs5521v5ybdw1z1yh97z2ky1dxm4d9-bitlbee-3.6.drv [...] $ cat /tmp/test-bitlbee/manifest ;; This file was automatically generated and is for internal use only. ;; It cannot be passed to the '--manifest' option. (manifest (version 3) (packages (("bitlbee" "3.6" "out" "/gnu/store/d67r9k5hwfm5hkd1d3pbhg49fcc2jj4q-bitlbee-3.6" (propagated-inputs ()) (search-paths ()) (properties (transformations (with-parameter . "bitlbee=libpurple=true"))))))) --8<---------------cut here---------------end--------------->8--- That’s it! We would need more things, like a ‘guix parameters’ command to show the available parameters of a package and an ‘--all-parameter-values’ option to ‘guix build’ to build all the variants of a given package. An important question: do we have examples of packages for which we’d like to have parameters? I’d grepped for “inherit” and that yields a few potential candidates, but also maybe a few potential non-candidates. Would this be a good fit for them? Thoughts? Ludo’. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.2: 0003-ui-package-recutils-emits-parameters-field.patch --] [-- Type: text/x-patch, Size: 1414 bytes --] From 9155411f2e8e78922e1e46d92068ac8f652ff0a5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> Date: Sun, 15 Nov 2020 17:01:14 +0100 Subject: [PATCH 3/4] ui: 'package->recutils' emits "parameters" field. * guix/ui.scm (package->recutils): Add "parameters" recutils field. --- guix/ui.scm | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/guix/ui.scm b/guix/ui.scm index 4e686297e8..2485400cc9 100644 --- a/guix/ui.scm +++ b/guix/ui.scm @@ -60,6 +60,7 @@ #:hide (package-name->name+version ;; Avoid "overrides core binding" warning. delete)) + #:autoload (guix parameters) (package-parameters package-parameter-name) #:use-module (srfi srfi-1) #:use-module (srfi srfi-9 gnu) #:use-module (srfi srfi-11) @@ -1529,6 +1530,10 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate." (format port "name: ~a~%" (package-name p)) (format port "version: ~a~%" (package-version p)) (format port "outputs: ~a~%" (string-join (package-outputs p))) + (match (package-parameters p) + (() #t) + (lst (format port "parameters:~{ ~a~}~%" + (map package-parameter-name lst)))) (format port "systems: ~a~%" (string-join (package-transitive-supported-systems p))) (format port "dependencies: ~a~%" -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.3: 0004-gnu-bitlbee-Add-libpurple-parameter.patch --] [-- Type: text/x-patch, Size: 3823 bytes --] From 49d7746ada4d4674acbbfd2606ad9bff4f6207eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> Date: Sun, 15 Nov 2020 17:04:18 +0100 Subject: [PATCH 4/4] gnu: bitlbee: Add "libpurple" parameter. * gnu/packages/messaging.scm (bitlbee)[inputs]: Add optional PIDGIN input. [properties]: New field. (bitlbee-purple): Mark as deprecated. --- gnu/packages/messaging.scm | 43 +++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm index b462504894..2f0f44d10d 100644 --- a/gnu/packages/messaging.scm +++ b/gnu/packages/messaging.scm @@ -120,6 +120,7 @@ #:use-module (guix build-system trivial) #:use-module (guix download) #:use-module (guix git-download) + #:use-module (guix parameters) #:use-module (guix hg-download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) @@ -256,7 +257,8 @@ end-to-end encryption.") ("libotr" ,libotr) ("gnutls" ,gnutls) ("python" ,python) - ("perl" ,perl))) + ("perl" ,perl) + ,@(optionally 'libpurple? `("purple" ,pidgin)))) (arguments `(#:phases (modify-phases %standard-phases @@ -275,7 +277,21 @@ end-to-end encryption.") (invoke "./configure" (string-append "--prefix=" (assoc-ref outputs "out")) - "--otr=1")))))) + "--otr=1" + ,@(optionally 'libpurple? "--purple=1"))))) + + ;; XXX: Tests fail to link, and ./configure says that it's "supported + ;; on a best-effort basis" anyway. + #:tests? ,(not (assq-ref (package-properties this-package) + 'libpurple?)))) + (properties + `((parameters + ,(package-parameter (name "libpurple") + (description + "Whether to enable libpurple (Pidgin) +support.") + (property 'libpurple?) + (type boolean))))) (synopsis "IRC to instant messaging gateway") (description "BitlBee brings IM (instant messaging) to IRC clients, for people who have an IRC client running all the time and don't want to run an @@ -289,25 +305,10 @@ identi.ca and status.net).") (define-public bitlbee-purple ;; This variant uses libpurple, which provides support for more protocols at ;; the expense of a much bigger closure. - (package/inherit bitlbee - (name "bitlbee-purple") - (synopsis "IRC to instant messaging gateway (using Pidgin's libpurple)") - (inputs `(("purple" ,pidgin) - ,@(package-inputs bitlbee))) - (arguments - (substitute-keyword-arguments (package-arguments bitlbee) - ((#:phases phases '%standard-phases) - `(modify-phases ,phases - (replace 'configure ;add "--purple=1" - (lambda* (#:key outputs #:allow-other-keys) - (invoke "./configure" - (string-append "--prefix=" - (assoc-ref outputs "out")) - "--otr=1" "--purple=1"))))) - ((#:tests? _ #t) - ;; XXX: Tests fail to link, and ./configure says that it's "supported - ;; on a best-effort basis" anyway. - #f))))) + (deprecated-package "bitlbee-purple" + (package/inherit bitlbee + (properties `((libpurple? . #t) + ,@(package-properties bitlbee)))))) (define-public bitlbee-discord (package -- 2.29.2 [-- Attachment #1.4: 0001-DRAFT-Add-guix-parameters.patch --] [-- Type: text/x-patch, Size: 6318 bytes --] From f42b68499c4e2a9bd368fe6a516932f5afa7a189 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> Date: Sun, 15 Nov 2020 16:58:52 +0100 Subject: [PATCH 1/4] DRAFT Add (guix parameters). DRAFT: Missing tests & doc. * guix/parameters.scm: New file. * Makefile.am (MODULES): Add it. --- Makefile.am | 1 + guix/parameters.scm | 131 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) create mode 100644 guix/parameters.scm diff --git a/Makefile.am b/Makefile.am index e7053ee4f4..72f955360d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -235,6 +235,7 @@ MODULES = \ guix/build/make-bootstrap.scm \ guix/search-paths.scm \ guix/packages.scm \ + guix/parameters.scm \ guix/import/cabal.scm \ guix/import/cpan.scm \ guix/import/cran.scm \ diff --git a/guix/parameters.scm b/guix/parameters.scm new file mode 100644 index 0000000000..e4f8240aa4 --- /dev/null +++ b/guix/parameters.scm @@ -0,0 +1,131 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix parameters) + #:use-module (guix packages) + #:use-module (guix records) + #:use-module (guix diagnostics) + #:use-module (guix i18n) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:use-module (ice-9 match) + #:export (package-parameter + package-parameter? + package-parameter-name + package-parameter-property + package-parameter-type + package-parameter-description + + boolean + optionally + + package-parameters + lookup-package-parameter + package-parameter-value + set-package-parameter-value)) + +;;; Commentary: +;;; +;;; This module provides a way to express high-level "package parameters", +;;; which allow users to customize how packages are built. Parameters are an +;;; interface that package developers define, where each parameter has a name +;;; and type. The user interface then converts parameter values from string +;;; to Scheme values and records them in the package properties. +;;; +;;; Package parameters are discoverable; their description is +;;; internationalized. The possible values of a parameter can be enumerated, +;;; and thus the Cartesian product of all possible parameter values for a +;;; package can be enumerated as well. +;;; +;;; Code: + +;; Package parameter interface. +(define-record-type* <package-parameter> package-parameter + make-package-parameter + package-parameter? + (name package-parameter-name) + (property package-parameter-property (default (string->symbol name))) + (type package-parameter-type) + (description package-parameter-description)) + +;; Type of a package parameter. +(define-record-type* <parameter-type> parameter-type + make-parameter-type + parameter-type? + (name parameter-type-name) ;debugging purposes only! + (string->value parameter-type-string->value) + (value->string parameter-type-value->string) + (universe parameter-type-universe)) + +(define boolean + ;; The Boolean parameter type. + (parameter-type (name 'boolean) + (universe '(#true #false)) + (value->string + (match-lambda + (#f "false") + (#t "true"))) + (string->value + (lambda (str) + (cond ((string-ci=? str "true") + #t) + ((string-ci=? str "false") + #f) + (else + (raise (condition + (&message (message "wrong value")))))))))) + +(define (package-parameters package) + (or (assq-ref (package-properties package) 'parameters) + '())) + +(define (package-parameter-value package parameter) + (assq-ref (package-properties package) + (package-parameter-property parameter))) + +(define (lookup-package-parameter package name) + (find (lambda (parameter) + (string=? (package-parameter-name parameter) name)) + (package-parameters package))) + +(define (set-package-parameter-value package name value) + (let ((parameter (lookup-package-parameter package name)) + (location (package-field-location package 'properties))) + (unless parameter + (raise (apply make-compound-condition + (formatted-message + (G_ "~a: no such package parameter") + name) + (if location + (list (condition + (&error-location (location location)))) + '())))) + (let* ((property (package-parameter-property parameter)) + (type (package-parameter-type parameter)) + (value ((parameter-type-string->value type) value))) + (package/inherit package + (properties + (alist-cons property value + (alist-delete property (package-properties package) + eq?))))))) + +(define-syntax-rule (optionally property exp) + (if (assq-ref (package-properties this-package) property) + (list exp) + '())) -- 2.29.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.5: 0002-DRAFT-transformations-Add-with-parameter.patch --] [-- Type: text/x-patch, Size: 3849 bytes --] From 5d6d81e4c3e37de53fe7f62ff7ef94da8b2df033 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> Date: Sun, 15 Nov 2020 16:59:48 +0100 Subject: [PATCH 2/4] DRAFT transformations: Add '--with-parameter'. DRAFT: Missing tests & doc. * guix/transformations.scm (evaluate-parameter-specs) (transform-package-parameters): New procedures. (%transformations, %transformation-options): Add 'with-parameter'. --- guix/transformations.scm | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/guix/transformations.scm b/guix/transformations.scm index 30142dd059..0f83eb470d 100644 --- a/guix/transformations.scm +++ b/guix/transformations.scm @@ -25,6 +25,7 @@ #:autoload (guix download) (download-to-store) #:autoload (guix git-download) (git-reference? git-reference-url) #:autoload (guix git) (git-checkout git-checkout? git-checkout-url) + #:autoload (guix parameters) (set-package-parameter-value) #:use-module (guix utils) #:use-module (guix memoization) #:use-module (guix gexp) @@ -324,6 +325,41 @@ a checkout of the Git repository at the given URL." (rewrite obj) obj))) +(define (evaluate-parameter-specs specs proc) + "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a +list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called +to return the replacement package. Raise an error if an element of SPECS uses +invalid syntax, or if a package it refers to could not be found." + (map (lambda (spec) + (match (string-tokenize spec %not-equal) + ((spec name value) + (define (replace old) + (proc old name value)) + + (cons spec replace)) + (_ + (raise + (formatted-message + (G_ "invalid package parameter specification: ~s") + spec))))) + specs)) + +(define (transform-package-parameters replacement-specs) + "Return a procedure that, when passed a package, replaces its direct +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of +strings like \"guile-next=stable-3.0\" meaning that packages are built using +'guile-next' from the latest commit on its 'stable-3.0' branch." + (define (replace old name value) + (set-package-parameter-value old name value)) + + (let* ((replacements (evaluate-parameter-specs replacement-specs + replace)) + (rewrite (package-input-rewriting/spec replacements))) + (lambda (obj) + (if (package? obj) + (rewrite obj) + obj)))) + (define (package-dependents/spec top bottom) "Return the list of dependents of BOTTOM, a spec string, that are also dependencies of TOP, a package." @@ -467,6 +503,7 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field." (with-branch . ,transform-package-source-branch) (with-commit . ,transform-package-source-commit) (with-git-url . ,transform-package-source-git-url) + (with-parameter . ,transform-package-parameters) (with-c-toolchain . ,transform-package-toolchain) (with-debug-info . ,transform-package-with-debug-info) (without-tests . ,transform-package-tests))) @@ -503,6 +540,8 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field." (parser 'with-commit)) (option '("with-git-url") #t #f (parser 'with-git-url)) + (option '("with-parameter") #t #f + (parser 'with-parameter)) (option '("with-c-toolchain") #t #f (parser 'with-c-toolchain)) (option '("with-debug-info") #t #f -- 2.29.2 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 853 bytes --]
Neat! Thanks for working on it :) > An important question: do we have examples of packages for which we’d > like to have parameters? I’d grepped for “inherit” and that yields a > few potential candidates, but also maybe a few potential non-candidates. > Would this be a good fit for them? What about evince? eps/ps/dvi rendering is disabled, according to this thread [1] for security reasons, so it makes sense to have it disabled by default. But I'd like to be able to enable it easily. WDYT? Nicolò [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39674 Ludovic Courtès <ludo@gnu.org> writes: > Hello Guix! > > For some time we’ve discussed ways to achieve “parameterized > packages”—packages where one can from the command line or from Scheme > configure optional build-time features, similar to Gentoo “USE flags”. > > I still have mixed feeling about this feature: on one hand it can bring > much welcome flexibility, but on the other hand it can also lead us to > the wild west of untested package configurations and combinatorial > explosion. It could also be argued that we achieve something similar > today by simply defining package variants when we have to: > > https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html > > That said, this message is about a possible implementation of package > parameters, so here we go. :-) > > To me the requirements for package parameters are: > > 1. it must be possible to discover them and choose them from the UI; > > 2. they must contain on-line internationalized documentation such that > the UI can list a package’s parameters and their type; > > 3. the chosen parameters when installing a package in a profile must > be preserved; > > 4. it must be possible to enumerate all the possible values of a > parameter, and thus to build the Cartesian product of all the > possible parameter combinations of a package (or of a package > graph!), so we can test those combinations as much as possible. > > This leads to the patches below. The last one gives an example use for > Bitlbee: it adds a “libpurple” parameter that allows users to choose > whether or not to enable the optional libpurple dependency: > > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=true -n > The following derivation would be built: > /gnu/store/mkknqgjsa93ajcl5d2krngizb11j1b0q-bitlbee-3.6.drv > $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=false -n > /gnu/store/c2ckg51ffwgs6jni3l549k06w3jd3b7a-bitlbee-3.6 > $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=wat? -n > guix build: error: wrong value > $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libviolet=true -n > gnu/packages/messaging.scm:283:5: error: libviolet: no such package parameter > $ ./pre-inst-env guix show bitlbee > name: bitlbee > version: 3.6 > outputs: out > parameters: libpurple > systems: x86_64-linux i686-linux > dependencies: check@0.12.0 glib@2.62.6 gnutls@3.6.12 libotr@4.1.1 perl@5.30.2 pkg-config@0.29.2 python@3.8.2 > location: gnu/packages/messaging.scm:243:2 > homepage: https://www.bitlbee.org/ > license: GPL 2+, FreeBSD > synopsis: IRC to instant messaging gateway > description: BitlBee brings IM (instant messaging) to IRC clients, for people who have an IRC client running all > + the time and don't want to run an additional IM client. BitlBee currently supports XMPP/Jabber (including Google > + Talk), MSN Messenger, Yahoo! Messenger, AIM and ICQ, and the Twitter microblogging network (plus all other Twitter > + API compatible services like identi.ca and status.net). > > $ ./pre-inst-env guix install bitlbee --with-parameter=bitlbee=libpurple=true -p /tmp/test-bitlbee > The following package will be installed: > bitlbee 3.6 > > The following derivation will be built: > /gnu/store/clvs5521v5ybdw1z1yh97z2ky1dxm4d9-bitlbee-3.6.drv > > [...] > > $ cat /tmp/test-bitlbee/manifest > ;; This file was automatically generated and is for internal use only. > ;; It cannot be passed to the '--manifest' option. > > (manifest > (version 3) > (packages > (("bitlbee" > "3.6" > "out" > "/gnu/store/d67r9k5hwfm5hkd1d3pbhg49fcc2jj4q-bitlbee-3.6" > (propagated-inputs ()) > (search-paths ()) > (properties > (transformations > (with-parameter . "bitlbee=libpurple=true"))))))) > --8<---------------cut here---------------end--------------->8--- > > That’s it! > > We would need more things, like a ‘guix parameters’ command to show the > available parameters of a package and an ‘--all-parameter-values’ option > to ‘guix build’ to build all the variants of a given package. > > An important question: do we have examples of packages for which we’d > like to have parameters? I’d grepped for “inherit” and that yields a > few potential candidates, but also maybe a few potential non-candidates. > Would this be a good fit for them? > > Thoughts? > > Ludo’. > > From 9155411f2e8e78922e1e46d92068ac8f652ff0a5 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> > Date: Sun, 15 Nov 2020 17:01:14 +0100 > Subject: [PATCH 3/4] ui: 'package->recutils' emits "parameters" field. > > * guix/ui.scm (package->recutils): Add "parameters" recutils field. > --- > guix/ui.scm | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/guix/ui.scm b/guix/ui.scm > index 4e686297e8..2485400cc9 100644 > --- a/guix/ui.scm > +++ b/guix/ui.scm > @@ -60,6 +60,7 @@ > #:hide (package-name->name+version > ;; Avoid "overrides core binding" warning. > delete)) > + #:autoload (guix parameters) (package-parameters package-parameter-name) > #:use-module (srfi srfi-1) > #:use-module (srfi srfi-9 gnu) > #:use-module (srfi srfi-11) > @@ -1529,6 +1530,10 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate." > (format port "name: ~a~%" (package-name p)) > (format port "version: ~a~%" (package-version p)) > (format port "outputs: ~a~%" (string-join (package-outputs p))) > + (match (package-parameters p) > + (() #t) > + (lst (format port "parameters:~{ ~a~}~%" > + (map package-parameter-name lst)))) > (format port "systems: ~a~%" > (string-join (package-transitive-supported-systems p))) > (format port "dependencies: ~a~%" > -- > 2.29.2 > > From 49d7746ada4d4674acbbfd2606ad9bff4f6207eb Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> > Date: Sun, 15 Nov 2020 17:04:18 +0100 > Subject: [PATCH 4/4] gnu: bitlbee: Add "libpurple" parameter. > > * gnu/packages/messaging.scm (bitlbee)[inputs]: Add optional PIDGIN > input. > [properties]: New field. > (bitlbee-purple): Mark as deprecated. > --- > gnu/packages/messaging.scm | 43 +++++++++++++++++++------------------- > 1 file changed, 22 insertions(+), 21 deletions(-) > > diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm > index b462504894..2f0f44d10d 100644 > --- a/gnu/packages/messaging.scm > +++ b/gnu/packages/messaging.scm > @@ -120,6 +120,7 @@ > #:use-module (guix build-system trivial) > #:use-module (guix download) > #:use-module (guix git-download) > + #:use-module (guix parameters) > #:use-module (guix hg-download) > #:use-module ((guix licenses) #:prefix license:) > #:use-module (guix packages) > @@ -256,7 +257,8 @@ end-to-end encryption.") > ("libotr" ,libotr) > ("gnutls" ,gnutls) > ("python" ,python) > - ("perl" ,perl))) > + ("perl" ,perl) > + ,@(optionally 'libpurple? `("purple" ,pidgin)))) > (arguments > `(#:phases > (modify-phases %standard-phases > @@ -275,7 +277,21 @@ end-to-end encryption.") > (invoke "./configure" > (string-append "--prefix=" > (assoc-ref outputs "out")) > - "--otr=1")))))) > + "--otr=1" > + ,@(optionally 'libpurple? "--purple=1"))))) > + > + ;; XXX: Tests fail to link, and ./configure says that it's "supported > + ;; on a best-effort basis" anyway. > + #:tests? ,(not (assq-ref (package-properties this-package) > + 'libpurple?)))) > + (properties > + `((parameters > + ,(package-parameter (name "libpurple") > + (description > + "Whether to enable libpurple (Pidgin) > +support.") > + (property 'libpurple?) > + (type boolean))))) > (synopsis "IRC to instant messaging gateway") > (description "BitlBee brings IM (instant messaging) to IRC clients, for > people who have an IRC client running all the time and don't want to run an > @@ -289,25 +305,10 @@ identi.ca and status.net).") > (define-public bitlbee-purple > ;; This variant uses libpurple, which provides support for more protocols at > ;; the expense of a much bigger closure. > - (package/inherit bitlbee > - (name "bitlbee-purple") > - (synopsis "IRC to instant messaging gateway (using Pidgin's libpurple)") > - (inputs `(("purple" ,pidgin) > - ,@(package-inputs bitlbee))) > - (arguments > - (substitute-keyword-arguments (package-arguments bitlbee) > - ((#:phases phases '%standard-phases) > - `(modify-phases ,phases > - (replace 'configure ;add "--purple=1" > - (lambda* (#:key outputs #:allow-other-keys) > - (invoke "./configure" > - (string-append "--prefix=" > - (assoc-ref outputs "out")) > - "--otr=1" "--purple=1"))))) > - ((#:tests? _ #t) > - ;; XXX: Tests fail to link, and ./configure says that it's "supported > - ;; on a best-effort basis" anyway. > - #f))))) > + (deprecated-package "bitlbee-purple" > + (package/inherit bitlbee > + (properties `((libpurple? . #t) > + ,@(package-properties bitlbee)))))) > > (define-public bitlbee-discord > (package > -- > 2.29.2 > > From f42b68499c4e2a9bd368fe6a516932f5afa7a189 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> > Date: Sun, 15 Nov 2020 16:58:52 +0100 > Subject: [PATCH 1/4] DRAFT Add (guix parameters). > > DRAFT: Missing tests & doc. > > * guix/parameters.scm: New file. > * Makefile.am (MODULES): Add it. > --- > Makefile.am | 1 + > guix/parameters.scm | 131 ++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 132 insertions(+) > create mode 100644 guix/parameters.scm > > diff --git a/Makefile.am b/Makefile.am > index e7053ee4f4..72f955360d 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -235,6 +235,7 @@ MODULES = \ > guix/build/make-bootstrap.scm \ > guix/search-paths.scm \ > guix/packages.scm \ > + guix/parameters.scm \ > guix/import/cabal.scm \ > guix/import/cpan.scm \ > guix/import/cran.scm \ > diff --git a/guix/parameters.scm b/guix/parameters.scm > new file mode 100644 > index 0000000000..e4f8240aa4 > --- /dev/null > +++ b/guix/parameters.scm > @@ -0,0 +1,131 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org> > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. > + > +(define-module (guix parameters) > + #:use-module (guix packages) > + #:use-module (guix records) > + #:use-module (guix diagnostics) > + #:use-module (guix i18n) > + #:use-module (srfi srfi-1) > + #:use-module (srfi srfi-34) > + #:use-module (srfi srfi-35) > + #:use-module (ice-9 match) > + #:export (package-parameter > + package-parameter? > + package-parameter-name > + package-parameter-property > + package-parameter-type > + package-parameter-description > + > + boolean > + optionally > + > + package-parameters > + lookup-package-parameter > + package-parameter-value > + set-package-parameter-value)) > + > +;;; Commentary: > +;;; > +;;; This module provides a way to express high-level "package parameters", > +;;; which allow users to customize how packages are built. Parameters are an > +;;; interface that package developers define, where each parameter has a name > +;;; and type. The user interface then converts parameter values from string > +;;; to Scheme values and records them in the package properties. > +;;; > +;;; Package parameters are discoverable; their description is > +;;; internationalized. The possible values of a parameter can be enumerated, > +;;; and thus the Cartesian product of all possible parameter values for a > +;;; package can be enumerated as well. > +;;; > +;;; Code: > + > +;; Package parameter interface. > +(define-record-type* <package-parameter> package-parameter > + make-package-parameter > + package-parameter? > + (name package-parameter-name) > + (property package-parameter-property (default (string->symbol name))) > + (type package-parameter-type) > + (description package-parameter-description)) > + > +;; Type of a package parameter. > +(define-record-type* <parameter-type> parameter-type > + make-parameter-type > + parameter-type? > + (name parameter-type-name) ;debugging purposes only! > + (string->value parameter-type-string->value) > + (value->string parameter-type-value->string) > + (universe parameter-type-universe)) > + > +(define boolean > + ;; The Boolean parameter type. > + (parameter-type (name 'boolean) > + (universe '(#true #false)) > + (value->string > + (match-lambda > + (#f "false") > + (#t "true"))) > + (string->value > + (lambda (str) > + (cond ((string-ci=? str "true") > + #t) > + ((string-ci=? str "false") > + #f) > + (else > + (raise (condition > + (&message (message "wrong value")))))))))) > + > +(define (package-parameters package) > + (or (assq-ref (package-properties package) 'parameters) > + '())) > + > +(define (package-parameter-value package parameter) > + (assq-ref (package-properties package) > + (package-parameter-property parameter))) > + > +(define (lookup-package-parameter package name) > + (find (lambda (parameter) > + (string=? (package-parameter-name parameter) name)) > + (package-parameters package))) > + > +(define (set-package-parameter-value package name value) > + (let ((parameter (lookup-package-parameter package name)) > + (location (package-field-location package 'properties))) > + (unless parameter > + (raise (apply make-compound-condition > + (formatted-message > + (G_ "~a: no such package parameter") > + name) > + (if location > + (list (condition > + (&error-location (location location)))) > + '())))) > + (let* ((property (package-parameter-property parameter)) > + (type (package-parameter-type parameter)) > + (value ((parameter-type-string->value type) value))) > + (package/inherit package > + (properties > + (alist-cons property value > + (alist-delete property (package-properties package) > + eq?))))))) > + > +(define-syntax-rule (optionally property exp) > + (if (assq-ref (package-properties this-package) property) > + (list exp) > + '())) > -- > 2.29.2 > > From 5d6d81e4c3e37de53fe7f62ff7ef94da8b2df033 Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> > Date: Sun, 15 Nov 2020 16:59:48 +0100 > Subject: [PATCH 2/4] DRAFT transformations: Add '--with-parameter'. > > DRAFT: Missing tests & doc. > > * guix/transformations.scm (evaluate-parameter-specs) > (transform-package-parameters): New procedures. > (%transformations, %transformation-options): Add 'with-parameter'. > --- > guix/transformations.scm | 39 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 39 insertions(+) > > diff --git a/guix/transformations.scm b/guix/transformations.scm > index 30142dd059..0f83eb470d 100644 > --- a/guix/transformations.scm > +++ b/guix/transformations.scm > @@ -25,6 +25,7 @@ > #:autoload (guix download) (download-to-store) > #:autoload (guix git-download) (git-reference? git-reference-url) > #:autoload (guix git) (git-checkout git-checkout? git-checkout-url) > + #:autoload (guix parameters) (set-package-parameter-value) > #:use-module (guix utils) > #:use-module (guix memoization) > #:use-module (guix gexp) > @@ -324,6 +325,41 @@ a checkout of the Git repository at the given URL." > (rewrite obj) > obj))) > > +(define (evaluate-parameter-specs specs proc) > + "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a > +list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called > +to return the replacement package. Raise an error if an element of SPECS uses > +invalid syntax, or if a package it refers to could not be found." > + (map (lambda (spec) > + (match (string-tokenize spec %not-equal) > + ((spec name value) > + (define (replace old) > + (proc old name value)) > + > + (cons spec replace)) > + (_ > + (raise > + (formatted-message > + (G_ "invalid package parameter specification: ~s") > + spec))))) > + specs)) > + > +(define (transform-package-parameters replacement-specs) > + "Return a procedure that, when passed a package, replaces its direct > +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of > +strings like \"guile-next=stable-3.0\" meaning that packages are built using > +'guile-next' from the latest commit on its 'stable-3.0' branch." > + (define (replace old name value) > + (set-package-parameter-value old name value)) > + > + (let* ((replacements (evaluate-parameter-specs replacement-specs > + replace)) > + (rewrite (package-input-rewriting/spec replacements))) > + (lambda (obj) > + (if (package? obj) > + (rewrite obj) > + obj)))) > + > (define (package-dependents/spec top bottom) > "Return the list of dependents of BOTTOM, a spec string, that are also > dependencies of TOP, a package." > @@ -467,6 +503,7 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field." > (with-branch . ,transform-package-source-branch) > (with-commit . ,transform-package-source-commit) > (with-git-url . ,transform-package-source-git-url) > + (with-parameter . ,transform-package-parameters) > (with-c-toolchain . ,transform-package-toolchain) > (with-debug-info . ,transform-package-with-debug-info) > (without-tests . ,transform-package-tests))) > @@ -503,6 +540,8 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field." > (parser 'with-commit)) > (option '("with-git-url") #t #f > (parser 'with-git-url)) > + (option '("with-parameter") #t #f > + (parser 'with-parameter)) > (option '("with-c-toolchain") #t #f > (parser 'with-c-toolchain)) > (option '("with-debug-info") #t #f > -- > 2.29.2
Hi Lduo, On Sun, 15 Nov 2020 at 17:33, Ludovic Courtès <ludo@gnu.org> wrote: > That said, this message is about a possible implementation of package > parameters, so here we go. :-) Cool! > To me the requirements for package parameters are: > > 1. it must be possible to discover them and choose them from the UI; > > 2. they must contain on-line internationalized documentation such that > the UI can list a package’s parameters and their type; Except ’boolean’, which kind of type do you have in mind? Aside that you did not find examples of packages requiring parameters. ;-) The answer leads to your point #4. > 3. the chosen parameters when installing a package in a profile must > be preserved; You mean track the parameters with ’properties’ in <profile>/manifest, right? > 4. it must be possible to enumerate all the possible values of a > parameter, and thus to build the Cartesian product of all the > possible parameter combinations of a package (or of a package > graph!), so we can test those combinations as much as possible. The values of the option are therefore known at package time, right? However, this implies restricted possibility for the type, right? > Subject: [PATCH 1/4] DRAFT Add (guix parameters). > > DRAFT: Missing tests & doc. > > * guix/parameters.scm: New file. > * Makefile.am (MODULES): Add it. > --- [...] > diff --git a/guix/parameters.scm b/guix/parameters.scm > + > +;; Type of a package parameter. > +(define-record-type* <parameter-type> parameter-type > + make-parameter-type > + parameter-type? > + (name parameter-type-name) ;debugging purposes only! > + (string->value parameter-type-string->value) > + (value->string parameter-type-value->string) > + (universe parameter-type-universe)) > + > +(define boolean > + ;; The Boolean parameter type. > + (parameter-type (name 'boolean) > + (universe '(#true #false)) > + (value->string > + (match-lambda > + (#f "false") > + (#t "true"))) > + (string->value > + (lambda (str) > + (cond ((string-ci=? str "true") > + #t) > + ((string-ci=? str "false") > + #f) > + (else > + (raise (condition > + (&message (message "wrong value")))))))))) The types will be “hard-coded“ here, right? Boolean being the simplest example and imagination just needs to be released, right? :-) > Subject: [PATCH 2/4] DRAFT transformations: Add '--with-parameter'. > > DRAFT: Missing tests & doc. > > * guix/transformations.scm (evaluate-parameter-specs) > (transform-package-parameters): New procedures. > (%transformations, %transformation-options): Add 'with-parameter'. > --- > guix/transformations.scm | 39 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 39 insertions(+) > > diff --git a/guix/transformations.scm b/guix/transformations.scm [...] > +(define (evaluate-parameter-specs specs proc) > + "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a > +list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called > +to return the replacement package. Raise an error if an element of SPECS uses > +invalid syntax, or if a package it refers to could not be found." > + (map (lambda (spec) > + (match (string-tokenize spec %not-equal) > + ((spec name value) > + (define (replace old) > + (proc old name value)) > + > + (cons spec replace)) > + (_ > + (raise > + (formatted-message > + (G_ "invalid package parameter specification: ~s") > + spec))))) > + specs)) Here ’proc’ could be anything, right? But then… > +(define (transform-package-parameters replacement-specs) > + "Return a procedure that, when passed a package, replaces its direct > +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of > +strings like \"guile-next=stable-3.0\" meaning that packages are built using > +'guile-next' from the latest commit on its 'stable-3.0' branch." > + (define (replace old name value) > + (set-package-parameter-value old name value)) > + > + (let* ((replacements (evaluate-parameter-specs replacement-specs > + replace)) > + (rewrite (package-input-rewriting/spec replacements))) > + (lambda (obj) > + (if (package? obj) > + (rewrite obj) > + obj)))) … it is ’set-package-parameter-value’. It is not clear in my mind. Does this constrain the hypothetical types? Cheers, simon
More in general, grepping for `--enable` or `--with` flags shows
a lot of potential packages (but also many false positives).
Nicolò Balzarotti <anothersms@gmail.com> writes:
> Neat! Thanks for working on it :)
>
>> An important question: do we have examples of packages for which we’d
>> like to have parameters? I’d grepped for “inherit” and that yields a
>> few potential candidates, but also maybe a few potential non-candidates.
>> Would this be a good fit for them?
>
> What about evince? eps/ps/dvi rendering is disabled, according to this
> thread [1] for security reasons, so it makes sense to have it disabled
> by default. But I'd like to be able to enable it easily.
>
> WDYT?
>
> Nicolò
>
> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39674
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello Guix!
>>
>> For some time we’ve discussed ways to achieve “parameterized
>> packages”—packages where one can from the command line or from Scheme
>> configure optional build-time features, similar to Gentoo “USE flags”.
>>
>> I still have mixed feeling about this feature: on one hand it can bring
>> much welcome flexibility, but on the other hand it can also lead us to
>> the wild west of untested package configurations and combinatorial
>> explosion. It could also be argued that we achieve something similar
>> today by simply defining package variants when we have to:
>>
>> https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html
>>
>> That said, this message is about a possible implementation of package
>> parameters, so here we go. :-)
>>
>> To me the requirements for package parameters are:
>>
>> 1. it must be possible to discover them and choose them from the UI;
>>
>> 2. they must contain on-line internationalized documentation such that
>> the UI can list a package’s parameters and their type;
>>
>> 3. the chosen parameters when installing a package in a profile must
>> be preserved;
>>
>> 4. it must be possible to enumerate all the possible values of a
>> parameter, and thus to build the Cartesian product of all the
>> possible parameter combinations of a package (or of a package
>> graph!), so we can test those combinations as much as possible.
>>
>> This leads to the patches below. The last one gives an example use for
>> Bitlbee: it adds a “libpurple” parameter that allows users to choose
>> whether or not to enable the optional libpurple dependency:
>>
>> --8<---------------cut here---------------start------------->8---
>> $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=true -n
>> The following derivation would be built:
>> /gnu/store/mkknqgjsa93ajcl5d2krngizb11j1b0q-bitlbee-3.6.drv
>> $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=false -n
>> /gnu/store/c2ckg51ffwgs6jni3l549k06w3jd3b7a-bitlbee-3.6
>> $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libpurple=wat? -n
>> guix build: error: wrong value
>> $ ./pre-inst-env guix build bitlbee --with-parameter=bitlbee=libviolet=true -n
>> gnu/packages/messaging.scm:283:5: error: libviolet: no such package parameter
>> $ ./pre-inst-env guix show bitlbee
>> name: bitlbee
>> version: 3.6
>> outputs: out
>> parameters: libpurple
>> systems: x86_64-linux i686-linux
>> dependencies: check@0.12.0 glib@2.62.6 gnutls@3.6.12 libotr@4.1.1 perl@5.30.2 pkg-config@0.29.2 python@3.8.2
>> location: gnu/packages/messaging.scm:243:2
>> homepage: https://www.bitlbee.org/
>> license: GPL 2+, FreeBSD
>> synopsis: IRC to instant messaging gateway
>> description: BitlBee brings IM (instant messaging) to IRC clients, for people who have an IRC client running all
>> + the time and don't want to run an additional IM client. BitlBee currently supports XMPP/Jabber (including Google
>> + Talk), MSN Messenger, Yahoo! Messenger, AIM and ICQ, and the Twitter microblogging network (plus all other Twitter
>> + API compatible services like identi.ca and status.net).
>>
>> $ ./pre-inst-env guix install bitlbee --with-parameter=bitlbee=libpurple=true -p /tmp/test-bitlbee
>> The following package will be installed:
>> bitlbee 3.6
>>
>> The following derivation will be built:
>> /gnu/store/clvs5521v5ybdw1z1yh97z2ky1dxm4d9-bitlbee-3.6.drv
>>
>> [...]
>>
>> $ cat /tmp/test-bitlbee/manifest
>> ;; This file was automatically generated and is for internal use only.
>> ;; It cannot be passed to the '--manifest' option.
>>
>> (manifest
>> (version 3)
>> (packages
>> (("bitlbee"
>> "3.6"
>> "out"
>> "/gnu/store/d67r9k5hwfm5hkd1d3pbhg49fcc2jj4q-bitlbee-3.6"
>> (propagated-inputs ())
>> (search-paths ())
>> (properties
>> (transformations
>> (with-parameter . "bitlbee=libpurple=true")))))))
>> --8<---------------cut here---------------end--------------->8---
>>
>> That’s it!
>>
>> We would need more things, like a ‘guix parameters’ command to show the
>> available parameters of a package and an ‘--all-parameter-values’ option
>> to ‘guix build’ to build all the variants of a given package.
>>
>> An important question: do we have examples of packages for which we’d
>> like to have parameters? I’d grepped for “inherit” and that yields a
>> few potential candidates, but also maybe a few potential non-candidates.
>> Would this be a good fit for them?
>>
>> Thoughts?
>>
>> Ludo’.
>>
>> From 9155411f2e8e78922e1e46d92068ac8f652ff0a5 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Sun, 15 Nov 2020 17:01:14 +0100
>> Subject: [PATCH 3/4] ui: 'package->recutils' emits "parameters" field.
>>
>> * guix/ui.scm (package->recutils): Add "parameters" recutils field.
>> ---
>> guix/ui.scm | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/guix/ui.scm b/guix/ui.scm
>> index 4e686297e8..2485400cc9 100644
>> --- a/guix/ui.scm
>> +++ b/guix/ui.scm
>> @@ -60,6 +60,7 @@
>> #:hide (package-name->name+version
>> ;; Avoid "overrides core binding" warning.
>> delete))
>> + #:autoload (guix parameters) (package-parameters package-parameter-name)
>> #:use-module (srfi srfi-1)
>> #:use-module (srfi srfi-9 gnu)
>> #:use-module (srfi srfi-11)
>> @@ -1529,6 +1530,10 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate."
>> (format port "name: ~a~%" (package-name p))
>> (format port "version: ~a~%" (package-version p))
>> (format port "outputs: ~a~%" (string-join (package-outputs p)))
>> + (match (package-parameters p)
>> + (() #t)
>> + (lst (format port "parameters:~{ ~a~}~%"
>> + (map package-parameter-name lst))))
>> (format port "systems: ~a~%"
>> (string-join (package-transitive-supported-systems p)))
>> (format port "dependencies: ~a~%"
>> --
>> 2.29.2
>>
>> From 49d7746ada4d4674acbbfd2606ad9bff4f6207eb Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Sun, 15 Nov 2020 17:04:18 +0100
>> Subject: [PATCH 4/4] gnu: bitlbee: Add "libpurple" parameter.
>>
>> * gnu/packages/messaging.scm (bitlbee)[inputs]: Add optional PIDGIN
>> input.
>> [properties]: New field.
>> (bitlbee-purple): Mark as deprecated.
>> ---
>> gnu/packages/messaging.scm | 43 +++++++++++++++++++-------------------
>> 1 file changed, 22 insertions(+), 21 deletions(-)
>>
>> diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm
>> index b462504894..2f0f44d10d 100644
>> --- a/gnu/packages/messaging.scm
>> +++ b/gnu/packages/messaging.scm
>> @@ -120,6 +120,7 @@
>> #:use-module (guix build-system trivial)
>> #:use-module (guix download)
>> #:use-module (guix git-download)
>> + #:use-module (guix parameters)
>> #:use-module (guix hg-download)
>> #:use-module ((guix licenses) #:prefix license:)
>> #:use-module (guix packages)
>> @@ -256,7 +257,8 @@ end-to-end encryption.")
>> ("libotr" ,libotr)
>> ("gnutls" ,gnutls)
>> ("python" ,python)
>> - ("perl" ,perl)))
>> + ("perl" ,perl)
>> + ,@(optionally 'libpurple? `("purple" ,pidgin))))
>> (arguments
>> `(#:phases
>> (modify-phases %standard-phases
>> @@ -275,7 +277,21 @@ end-to-end encryption.")
>> (invoke "./configure"
>> (string-append "--prefix="
>> (assoc-ref outputs "out"))
>> - "--otr=1"))))))
>> + "--otr=1"
>> + ,@(optionally 'libpurple? "--purple=1")))))
>> +
>> + ;; XXX: Tests fail to link, and ./configure says that it's "supported
>> + ;; on a best-effort basis" anyway.
>> + #:tests? ,(not (assq-ref (package-properties this-package)
>> + 'libpurple?))))
>> + (properties
>> + `((parameters
>> + ,(package-parameter (name "libpurple")
>> + (description
>> + "Whether to enable libpurple (Pidgin)
>> +support.")
>> + (property 'libpurple?)
>> + (type boolean)))))
>> (synopsis "IRC to instant messaging gateway")
>> (description "BitlBee brings IM (instant messaging) to IRC clients, for
>> people who have an IRC client running all the time and don't want to run an
>> @@ -289,25 +305,10 @@ identi.ca and status.net).")
>> (define-public bitlbee-purple
>> ;; This variant uses libpurple, which provides support for more protocols at
>> ;; the expense of a much bigger closure.
>> - (package/inherit bitlbee
>> - (name "bitlbee-purple")
>> - (synopsis "IRC to instant messaging gateway (using Pidgin's libpurple)")
>> - (inputs `(("purple" ,pidgin)
>> - ,@(package-inputs bitlbee)))
>> - (arguments
>> - (substitute-keyword-arguments (package-arguments bitlbee)
>> - ((#:phases phases '%standard-phases)
>> - `(modify-phases ,phases
>> - (replace 'configure ;add "--purple=1"
>> - (lambda* (#:key outputs #:allow-other-keys)
>> - (invoke "./configure"
>> - (string-append "--prefix="
>> - (assoc-ref outputs "out"))
>> - "--otr=1" "--purple=1")))))
>> - ((#:tests? _ #t)
>> - ;; XXX: Tests fail to link, and ./configure says that it's "supported
>> - ;; on a best-effort basis" anyway.
>> - #f)))))
>> + (deprecated-package "bitlbee-purple"
>> + (package/inherit bitlbee
>> + (properties `((libpurple? . #t)
>> + ,@(package-properties bitlbee))))))
>>
>> (define-public bitlbee-discord
>> (package
>> --
>> 2.29.2
>>
>> From f42b68499c4e2a9bd368fe6a516932f5afa7a189 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Sun, 15 Nov 2020 16:58:52 +0100
>> Subject: [PATCH 1/4] DRAFT Add (guix parameters).
>>
>> DRAFT: Missing tests & doc.
>>
>> * guix/parameters.scm: New file.
>> * Makefile.am (MODULES): Add it.
>> ---
>> Makefile.am | 1 +
>> guix/parameters.scm | 131 ++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 132 insertions(+)
>> create mode 100644 guix/parameters.scm
>>
>> diff --git a/Makefile.am b/Makefile.am
>> index e7053ee4f4..72f955360d 100644
>> --- a/Makefile.am
>> +++ b/Makefile.am
>> @@ -235,6 +235,7 @@ MODULES = \
>> guix/build/make-bootstrap.scm \
>> guix/search-paths.scm \
>> guix/packages.scm \
>> + guix/parameters.scm \
>> guix/import/cabal.scm \
>> guix/import/cpan.scm \
>> guix/import/cran.scm \
>> diff --git a/guix/parameters.scm b/guix/parameters.scm
>> new file mode 100644
>> index 0000000000..e4f8240aa4
>> --- /dev/null
>> +++ b/guix/parameters.scm
>> @@ -0,0 +1,131 @@
>> +;;; GNU Guix --- Functional package management for GNU
>> +;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org>
>> +;;;
>> +;;; This file is part of GNU Guix.
>> +;;;
>> +;;; GNU Guix is free software; you can redistribute it and/or modify it
>> +;;; under the terms of the GNU General Public License as published by
>> +;;; the Free Software Foundation; either version 3 of the License, or (at
>> +;;; your option) any later version.
>> +;;;
>> +;;; GNU Guix is distributed in the hope that it will be useful, but
>> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
>> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +;;; GNU General Public License for more details.
>> +;;;
>> +;;; You should have received a copy of the GNU General Public License
>> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
>> +
>> +(define-module (guix parameters)
>> + #:use-module (guix packages)
>> + #:use-module (guix records)
>> + #:use-module (guix diagnostics)
>> + #:use-module (guix i18n)
>> + #:use-module (srfi srfi-1)
>> + #:use-module (srfi srfi-34)
>> + #:use-module (srfi srfi-35)
>> + #:use-module (ice-9 match)
>> + #:export (package-parameter
>> + package-parameter?
>> + package-parameter-name
>> + package-parameter-property
>> + package-parameter-type
>> + package-parameter-description
>> +
>> + boolean
>> + optionally
>> +
>> + package-parameters
>> + lookup-package-parameter
>> + package-parameter-value
>> + set-package-parameter-value))
>> +
>> +;;; Commentary:
>> +;;;
>> +;;; This module provides a way to express high-level "package parameters",
>> +;;; which allow users to customize how packages are built. Parameters are an
>> +;;; interface that package developers define, where each parameter has a name
>> +;;; and type. The user interface then converts parameter values from string
>> +;;; to Scheme values and records them in the package properties.
>> +;;;
>> +;;; Package parameters are discoverable; their description is
>> +;;; internationalized. The possible values of a parameter can be enumerated,
>> +;;; and thus the Cartesian product of all possible parameter values for a
>> +;;; package can be enumerated as well.
>> +;;;
>> +;;; Code:
>> +
>> +;; Package parameter interface.
>> +(define-record-type* <package-parameter> package-parameter
>> + make-package-parameter
>> + package-parameter?
>> + (name package-parameter-name)
>> + (property package-parameter-property (default (string->symbol name)))
>> + (type package-parameter-type)
>> + (description package-parameter-description))
>> +
>> +;; Type of a package parameter.
>> +(define-record-type* <parameter-type> parameter-type
>> + make-parameter-type
>> + parameter-type?
>> + (name parameter-type-name) ;debugging purposes only!
>> + (string->value parameter-type-string->value)
>> + (value->string parameter-type-value->string)
>> + (universe parameter-type-universe))
>> +
>> +(define boolean
>> + ;; The Boolean parameter type.
>> + (parameter-type (name 'boolean)
>> + (universe '(#true #false))
>> + (value->string
>> + (match-lambda
>> + (#f "false")
>> + (#t "true")))
>> + (string->value
>> + (lambda (str)
>> + (cond ((string-ci=? str "true")
>> + #t)
>> + ((string-ci=? str "false")
>> + #f)
>> + (else
>> + (raise (condition
>> + (&message (message "wrong value"))))))))))
>> +
>> +(define (package-parameters package)
>> + (or (assq-ref (package-properties package) 'parameters)
>> + '()))
>> +
>> +(define (package-parameter-value package parameter)
>> + (assq-ref (package-properties package)
>> + (package-parameter-property parameter)))
>> +
>> +(define (lookup-package-parameter package name)
>> + (find (lambda (parameter)
>> + (string=? (package-parameter-name parameter) name))
>> + (package-parameters package)))
>> +
>> +(define (set-package-parameter-value package name value)
>> + (let ((parameter (lookup-package-parameter package name))
>> + (location (package-field-location package 'properties)))
>> + (unless parameter
>> + (raise (apply make-compound-condition
>> + (formatted-message
>> + (G_ "~a: no such package parameter")
>> + name)
>> + (if location
>> + (list (condition
>> + (&error-location (location location))))
>> + '()))))
>> + (let* ((property (package-parameter-property parameter))
>> + (type (package-parameter-type parameter))
>> + (value ((parameter-type-string->value type) value)))
>> + (package/inherit package
>> + (properties
>> + (alist-cons property value
>> + (alist-delete property (package-properties package)
>> + eq?)))))))
>> +
>> +(define-syntax-rule (optionally property exp)
>> + (if (assq-ref (package-properties this-package) property)
>> + (list exp)
>> + '()))
>> --
>> 2.29.2
>>
>> From 5d6d81e4c3e37de53fe7f62ff7ef94da8b2df033 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Sun, 15 Nov 2020 16:59:48 +0100
>> Subject: [PATCH 2/4] DRAFT transformations: Add '--with-parameter'.
>>
>> DRAFT: Missing tests & doc.
>>
>> * guix/transformations.scm (evaluate-parameter-specs)
>> (transform-package-parameters): New procedures.
>> (%transformations, %transformation-options): Add 'with-parameter'.
>> ---
>> guix/transformations.scm | 39 +++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 39 insertions(+)
>>
>> diff --git a/guix/transformations.scm b/guix/transformations.scm
>> index 30142dd059..0f83eb470d 100644
>> --- a/guix/transformations.scm
>> +++ b/guix/transformations.scm
>> @@ -25,6 +25,7 @@
>> #:autoload (guix download) (download-to-store)
>> #:autoload (guix git-download) (git-reference? git-reference-url)
>> #:autoload (guix git) (git-checkout git-checkout? git-checkout-url)
>> + #:autoload (guix parameters) (set-package-parameter-value)
>> #:use-module (guix utils)
>> #:use-module (guix memoization)
>> #:use-module (guix gexp)
>> @@ -324,6 +325,41 @@ a checkout of the Git repository at the given URL."
>> (rewrite obj)
>> obj)))
>>
>> +(define (evaluate-parameter-specs specs proc)
>> + "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a
>> +list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called
>> +to return the replacement package. Raise an error if an element of SPECS uses
>> +invalid syntax, or if a package it refers to could not be found."
>> + (map (lambda (spec)
>> + (match (string-tokenize spec %not-equal)
>> + ((spec name value)
>> + (define (replace old)
>> + (proc old name value))
>> +
>> + (cons spec replace))
>> + (_
>> + (raise
>> + (formatted-message
>> + (G_ "invalid package parameter specification: ~s")
>> + spec)))))
>> + specs))
>> +
>> +(define (transform-package-parameters replacement-specs)
>> + "Return a procedure that, when passed a package, replaces its direct
>> +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of
>> +strings like \"guile-next=stable-3.0\" meaning that packages are built using
>> +'guile-next' from the latest commit on its 'stable-3.0' branch."
>> + (define (replace old name value)
>> + (set-package-parameter-value old name value))
>> +
>> + (let* ((replacements (evaluate-parameter-specs replacement-specs
>> + replace))
>> + (rewrite (package-input-rewriting/spec replacements)))
>> + (lambda (obj)
>> + (if (package? obj)
>> + (rewrite obj)
>> + obj))))
>> +
>> (define (package-dependents/spec top bottom)
>> "Return the list of dependents of BOTTOM, a spec string, that are also
>> dependencies of TOP, a package."
>> @@ -467,6 +503,7 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field."
>> (with-branch . ,transform-package-source-branch)
>> (with-commit . ,transform-package-source-commit)
>> (with-git-url . ,transform-package-source-git-url)
>> + (with-parameter . ,transform-package-parameters)
>> (with-c-toolchain . ,transform-package-toolchain)
>> (with-debug-info . ,transform-package-with-debug-info)
>> (without-tests . ,transform-package-tests)))
>> @@ -503,6 +540,8 @@ to the same package but with #:strip-binaries? #f in its 'arguments' field."
>> (parser 'with-commit))
>> (option '("with-git-url") #t #f
>> (parser 'with-git-url))
>> + (option '("with-parameter") #t #f
>> + (parser 'with-parameter))
>> (option '("with-c-toolchain") #t #f
>> (parser 'with-c-toolchain))
>> (option '("with-debug-info") #t #f
>> --
>> 2.29.2
[-- Attachment #1: Type: text/plain, Size: 549 bytes --] Fantastic! One of the biggest struggle we had when discussing it was figuring out what to do about parameter propagation across dependencies. For instance, what if we want to build "all packages without X support"? This means that the parameter must traverse all inputs recursively if we don't want to drag X indirectly. If I understand your change correctly, the patch is only applying parameters to the given package and it's not propagated to the inputs, is that correct? Cheers! -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --]
Hi Pierre, On Sun, 15 Nov 2020 at 18:44, Pierre Neidhardt <mail@ambrevar.xyz> wrote: > For instance, what if we want to build "all packages without X support"? > This means that the parameter must traverse all inputs recursively if we > don't want to drag X indirectly. It means: add this new ’optionally’ to all the packages supporting X to be able to turn it off with the ’boolean’ type. IIUC. > If I understand your change correctly, the patch is only applying > parameters to the given package and it's not propagated to the inputs, > is that correct? Because of ’package-input-rewriting/spec’, it is applied to all the graph, IIUC. Cheers, simon
> An important question: do we have examples of packages for which we’d
> like to have parameters? I’d grepped for “inherit” and that yields a
> few potential candidates, but also maybe a few potential non-candidates.
> Would this be a good fit for them?
I suppose Emacs would be an obvious candidate, with its "no-x" as well
as "no-x-toolkit" variants.
Taylan
[-- Attachment #1: Type: text/plain, Size: 2374 bytes --] Hi Ludo, nice feature! On Sun, 15 Nov 2020 17:33:28 +0100 Ludovic Courtès <ludo@gnu.org> wrote: > An important question: do we have examples of packages for which we’d > like to have parameters? For the embedded/flash rom side: * Enable/disable building the documentation. I really don't need a 40 MiB manual stored onto a 16 MiB firmware flash chip. If that's better done as an extra output, fair enough. * Enable/disable obscure dependencies: Library packages that pull in 400 MiB Qt into the closure for a thing no one (TM) uses should probably provide a switch to disable that GUI. Sometimes a package provides multiple GUIs for different toolkits, in which case one should be able to choose one toolkit and not build for the others. * No, even in 2020, I won't start using AmigaFS, NFSv3 and whatever old protocol/format is still around and superseded by other protocols. The gexp-functionality of being able to select individual files of a package is already very useful for use cases an embedded developer would have (and that's there for a long time already). So that's nice! For the kind of feature flags I have in mind, it usually means I don't want to have the feature *anywhere*--for example, if I don't want to have Qt or Kerberos or whatever, that's because I want to save the space and thus it should be able to be *globally* specified--at least per profile. It doesn't help one bit for space savings if package A doesn't pull in Kerberos or latex or or Haskell bindings for a Raspberry Pi serial port library (guess why that's oddly specific ;) ), but package B does. However, sometimes using static libraries is better and there's no reason to build the shared libraries. But that's very much local to whatever I'm trying to do (at least local to a profile if not package). I would advise against doing a grep -r -- --enable and introducing all those as parameters. Rather I would check the closure of stuff and if the closure goes from 1200 MiB to 50 MiB, chances are a parameter would be nice there :) I guess the kind of flags I envision would be set at profile level. From experience with Gentoo before I can tell you that the combinatory explosion is a real problem and most of the "more advanced" (toggled more switches :) ) combinations did not work the majority of the time. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --]
Dear, On Sun, 15 Nov 2020 at 21:46, Danny Milosavljevic <dannym@scratchpost.org> wrote: > * Enable/disable building the documentation. I really don't need a 40 MiB > manual stored onto a 16 MiB firmware flash chip. If that's better done as an > extra output, fair enough. Related (I hope) is: build packages with several outputs. For instance, ’git’ has several 'outputs' ("send-email", "svn", etc.), so the list of "inputs" provides e.g., "subversion" even if I am only interested by e.g., "git:send-email". This matters about closure. And it is maybe an occasion to revisit the museum, i.e., the TODO file: ** extend ‘propagated-build-inputs’ with support for multiple outputs #+BEGIN_SRC scheme (outputs '("out" "include")) (propagated-build-inputs `(((("i1" ,p1 "o1") ("i2" ,p2)) => "include") ("i3" ,p3))) #+END_SRC For one reference: <https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00284.html> All the best, simon
On Sun, 15 Nov 2020 21:46:58 +0100
Danny Milosavljevic <dannym@scratchpost.org> wrote:
> Hi Ludo,
>
> nice feature!
>
> On Sun, 15 Nov 2020 17:33:28 +0100
> Ludovic Courtès <ludo@gnu.org> wrote:
>
> > An important question: do we have examples of packages for which
> > we’d like to have parameters?
>
> For the embedded/flash rom side:
>
> * Enable/disable building the documentation. I really don't need a
> 40 MiB manual stored onto a 16 MiB firmware flash chip. If that's
> better done as an extra output, fair enough.
> ......
Alpine already achieves an incredibly tiny install size by splitting
packages into many outputs. We could and should do the same.
As far as I know, they do not have parameterized packages.
[-- Attachment #1: Type: text/plain, Size: 763 bytes --] On November 15, 2020, raingloom <raingloom@riseup.net> wrote: > Alpine already achieves an incredibly tiny install size by splitting > packages into many outputs. We could and should do the same. > As far as I know, they do not have parameterized packages. I definitely support more package-splitting and dependency tree-shaking where possible. Getting a core Guix System down to Alpine size would be fantastic. For example, `docker size python` shows a 171.4mb bundle size just for Python alone, while `docker images python:alpine` shows a 44.3mb system image, which includes a bunch of system stuff in addition to Python. It would be great to have tools to understand what's in our big Guix packages and how we could ship light bundles that get the job done. [-- Attachment #2: Type: text/html, Size: 3073 bytes --]
Dear, On Mon, 16 Nov 2020 at 01:54, Ryan Prior <ryanprior@hey.com> wrote: > For example, `docker size python` shows a 171.4mb bundle size just for How is the Docker image built? > Python alone, while `docker images python:alpine` shows a 44.3mb > system Idem? I agree with your argument, I am just wondering if apples are compared to apples. :-) > image, which includes a bunch of system stuff in addition to Python. It > would be great to have tools to understand what's in our big Guix > packages and how we could ship light bundles that get the job done. Well, the closure is sometimes an issue, indeed. The tools to tackle this are “guix size” and “guix graph”, AFAIK. What do you think is missing? All the best, simon
zimoun <zimon.toutoune@gmail.com> writes: > Dear, > > On Sun, 15 Nov 2020 at 21:46, Danny Milosavljevic <dannym@scratchpost.org> wrote: > >> * Enable/disable building the documentation. I really don't need a 40 MiB >> manual stored onto a 16 MiB firmware flash chip. If that's better done as an >> extra output, fair enough. > > Related (I hope) is: build packages with several outputs. For instance, > ’git’ has several 'outputs' ("send-email", "svn", etc.), so the list of > "inputs" provides e.g., "subversion" even if I am only interested by > e.g., "git:send-email". This matters about closure. > > And it is maybe an occasion to revisit the museum, i.e., the TODO file: > > ** extend ‘propagated-build-inputs’ with support for multiple outputs > > #+BEGIN_SRC scheme > (outputs '("out" "include")) > (propagated-build-inputs > `(((("i1" ,p1 "o1") > ("i2" ,p2)) > => "include") > ("i3" ,p3))) > #+END_SRC > > For one reference: > > <https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00284.html> > > > All the best, > simon I'd like to suggest another plan: Make every ‘output’ become a <package> object, so ‘propagated-build-inputs’ doesn’t need to change. Then we’ll have something like debian’s source/binary packages [1] and archlinux’s base/split packages [2]. Example: --8<---------------cut here---------------start------------->8--- (define-public %gtk+ (source-package (name "gtk+") (version "3.24.20") (source ...) (inputs ...) (native-inputs ...) (outputs `(("out" . (binary-package (name "gtk+") ; version inherit from source (propagated ...) ; per output propagated-inputs here (native-search-paths ...) (synopsis ...) ; can override package metadata (description ...))) ("gtk-update-icon-cache" . (binary-package (name "gtk-update-icon-cache") ...)))) (arguments ...) (home-page ...) (synopsis ...) (descirption ...) (license ...))) (define-public gtk+ (source-package->binary-package %gtk+ "out")) (define-public gtk-update-icon-cache (source-package->binary-package %gtk+ "gtk-update-icon-cache")) --8<---------------cut here---------------end--------------->8--- [1] https://sources.debian.org/src/gtk+3.0/3.24.23-2/debian/control/ [2] https://github.com/archlinux/svntogit-packages/blob/packages/gtk3/trunk/PKGBUILD We'll have to build some compatibility layer with existing ‘package’ and adjust UI though. I think this is possible and will bring various benefits!
Hi Pierre,
Pierre Neidhardt <mail@ambrevar.xyz> skribis:
> One of the biggest struggle we had when discussing it was figuring out
> what to do about parameter propagation across dependencies.
>
> For instance, what if we want to build "all packages without X support"?
> This means that the parameter must traverse all inputs recursively if we
> don't want to drag X indirectly.
>
> If I understand your change correctly, the patch is only applying
> parameters to the given package and it's not propagated to the inputs,
> is that correct?
That’s correct: in this patch set parameters are per-package, and I
think it’s easier to start simple, but we could implement what you
describe without too much hassle I think.
For example, you’d type ‘--with-parameters=x11=false’, and that’d be
applied to all the packages that have an ‘x11’ parameter.
Ludo’.
Hi, zimoun <zimon.toutoune@gmail.com> skribis: >> To me the requirements for package parameters are: >> >> 1. it must be possible to discover them and choose them from the UI; >> >> 2. they must contain on-line internationalized documentation such that >> the UI can list a package’s parameters and their type; > > Except ’boolean’, which kind of type do you have in mind? Aside that > you did not find examples of packages requiring parameters. ;-) Another example would be enumerated types. >> 3. the chosen parameters when installing a package in a profile must >> be preserved; > > You mean track the parameters with ’properties’ in <profile>/manifest, > right? Yup! >> 4. it must be possible to enumerate all the possible values of a >> parameter, and thus to build the Cartesian product of all the >> possible parameter combinations of a package (or of a package >> graph!), so we can test those combinations as much as possible. > > The values of the option are therefore known at package time, right? > However, this implies restricted possibility for the type, right? The <package-parameter> record has to specify a type, and the type must have a finite universe. So there cannot be an “integer” type, for instance, but there can be “integer between 10 and 42”. >> +(define (evaluate-parameter-specs specs proc) >> + "Parse SPECS, a list of strings like \"bitlbee=purple=true\", and return a >> +list of spec/procedure pairs, where (PROC PACKAGE PARAMETER VALUE) is called >> +to return the replacement package. Raise an error if an element of SPECS uses >> +invalid syntax, or if a package it refers to could not be found." >> + (map (lambda (spec) >> + (match (string-tokenize spec %not-equal) >> + ((spec name value) >> + (define (replace old) >> + (proc old name value)) >> + >> + (cons spec replace)) >> + (_ >> + (raise >> + (formatted-message >> + (G_ "invalid package parameter specification: ~s") >> + spec))))) >> + specs)) > > Here ’proc’ could be anything, right? But then… > >> +(define (transform-package-parameters replacement-specs) >> + "Return a procedure that, when passed a package, replaces its direct >> +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of >> +strings like \"guile-next=stable-3.0\" meaning that packages are built using >> +'guile-next' from the latest commit on its 'stable-3.0' branch." >> + (define (replace old name value) >> + (set-package-parameter-value old name value)) >> + >> + (let* ((replacements (evaluate-parameter-specs replacement-specs >> + replace)) >> + (rewrite (package-input-rewriting/spec replacements))) >> + (lambda (obj) >> + (if (package? obj) >> + (rewrite obj) >> + obj)))) > > … it is ’set-package-parameter-value’. It is not clear in my mind. Yes I could have used ‘set-package-parameter-value’ directly instead of adding this ‘proc’ parameter; it would have been more readable I guess! Thanks, Ludo’.
[-- Attachment #1: Type: text/plain, Size: 540 bytes --] Thanks for confirming. Indeed, we should start easy! I believe it's important to foresee the obstacles that are looming at the horizon nonetheless, because "without too much hassle" might be a bit optimistic ;) See point 7. about conflicts: https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00026.html and comments: https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00177.html https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00181.html Cheers! -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --]
Hi, On Mon, 16 Nov 2020 at 13:03, Pierre Neidhardt <mail@ambrevar.xyz> wrote: > See point 7. about conflicts: > > https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00026.html Quoting for instance the 2 examples: For instance, use guile-2.2.4 instead of guile for all guile libraries, or use pulseaudio everywhere, including in dependencies that are not explicitly installed to the user profile. From my understanding, the first case (guile) is now covered by the “package rewriting” transformation (see package-input-rewriting IIUC). The issue with the second case is below. > and comments: > > https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00177.html From my understanding of the Ludo’s patch and from the Marius’s message, the both looks really similar. :-) > https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00181.html Quoting: To be usable, we would need something to say "build bar and all its inputs without pulseaudio, except for some given packages". While this is OK with 1 parameter, it's quickly gets much more complicated when packages have multiple parameters that maybe conflict with one another. Is the combinatorial conflict solvable in advance? Well, from my point of view, it cannot be via package transformation. For example, let’s consider the Emacs packages and #41732. Quoting [1]: > Perhaps then, > > --8<---------------cut here---------------start------------->8--- > guix build -m manifest.scm --with-input=emacs-minimal=emacs-next \ > --with-input=emacs=emacs-next > --8<---------------cut here---------------end--------------->8--- Possibly. And then Magit uses emacs-no-x as an input, so we may need to also add --with-input=emacs-no-x=emacs-next to the command. I'm just pointing out that the process is not as straightforward as it might seem. So, it doesn't sound right to simply suggest and [2]: For example, the package emacs-magit drags emacs-no-x because of emacs-libgit, why is emacs-minimal not enough here? Well, the emacs-build-system depends (implicitly) on emacs-minimal, only. And the initial patch `package-with-emacs-next' was changing this, only. However, the package emacs-libgit is cmake-build-system and the package emacs-no-x is an explicit dependency; which is another story. :-) these both examples show that it is already complex enough just to rebuild all the Emacs packages using another Emacs VM (emacs-next or REmacs, etc.). To be clear, I am not convinced that the current package recipes are functional enough to be able to keep under control the combinatorial conflict; in the general case. 1: <http://issues.guix.gnu.org/issue/41732#14> 2: <http://issues.guix.gnu.org/issue/41732> All the best, simon
Hi, Danny Milosavljevic <dannym@scratchpost.org> skribis: > For the embedded/flash rom side: Note: it’s great if it can help reduce closure size, but that’s not the goal. > * Enable/disable building the documentation. I really don't need a 40 MiB > manual stored onto a 16 MiB firmware flash chip. If that's better done as an > extra output, fair enough. Extra output is better, yup. :-) > * Enable/disable obscure dependencies: Library packages that pull in 400 MiB Qt > into the closure for a thing no one (TM) uses should probably provide a switch to > disable that GUI. Sometimes a package provides multiple GUIs for different > toolkits, in which case one should be able to choose one toolkit and not > build for the others. Yes, that sounds like a good use case. > * No, even in 2020, I won't start using AmigaFS, NFSv3 and whatever old > protocol/format is still around and superseded by other protocols. > > The gexp-functionality of being able to select individual files of a package > is already very useful for use cases an embedded developer would have > (and that's there for a long time already). So that's nice! > > For the kind of feature flags I have in mind, it usually means I don't want > to have the feature *anywhere*--for example, if I don't want to have Qt or > Kerberos or whatever, that's because I want to save the space and thus > it should be able to be *globally* specified--at least per profile. OK, so that’s similar to what Pierre was suggesting: “global” parameters, or at least parameters that apply to all the packages that know about it, not just to one package. Sounds doable, but again, the challenge will be to build all the combinations. > I would advise against doing a grep -r -- --enable and introducing all those > as parameters. Rather I would check the closure of stuff and if the closure > goes from 1200 MiB to 50 MiB, chances are a parameter would be nice there :) Heh, sounds like a valid criterion. I guess initially we’ll have to use them sparsely (in Guix proper) so we can gain experience with them. > From experience with Gentoo before I can tell you that the combinatory > explosion is a real problem and most of the "more advanced" (toggled more > switches :) ) combinations did not work the majority of the time. Right, and that’s precisely what I’d like to avoid, at least for the packages in Guix proper (authors of external channels might have a different policy.) So I suppose in Guix we’d have a policy of using them sparsely, and only/primarily in leaf packages. Thanks, Ludo’.
Hi,
宋文武 <iyzsong@outlook.com> skribis:
> I'd like to suggest another plan: Make every ‘output’ become a <package>
> object, so ‘propagated-build-inputs’ doesn’t need to change. Then we’ll
> have something like debian’s source/binary packages [1] and archlinux’s
> base/split packages [2].
>
> Example:
>
> (define-public %gtk+
> (source-package
> (name "gtk+")
> (version "3.24.20")
> (source ...)
> (inputs ...)
> (native-inputs ...)
> (outputs
> `(("out" .
> (binary-package
> (name "gtk+") ; version inherit from source
> (propagated ...) ; per output propagated-inputs here
> (native-search-paths ...)
> (synopsis ...) ; can override package metadata
> (description ...)))
> ("gtk-update-icon-cache" .
> (binary-package
> (name "gtk-update-icon-cache")
> ...))))
> (arguments ...)
> (home-page ...)
> (synopsis ...)
> (descirption ...)
> (license ...)))
>
> (define-public gtk+
> (source-package->binary-package %gtk+ "out"))
>
> (define-public gtk-update-icon-cache
> (source-package->binary-package %gtk+ "gtk-update-icon-cache"))
Interesting. That’d be a major change but if we can find a way forward,
it could be a plan.
Ludo’.
Hi, On Mon, 16 Nov 2020 at 19:25, 宋文武 <iyzsong@outlook.com> wrote: >> And it is maybe an occasion to revisit the museum, i.e., the TODO file: [...] > I'd like to suggest another plan: Make every ‘output’ become a <package> > object, so ‘propagated-build-inputs’ doesn’t need to change. Then we’ll > have something like debian’s source/binary packages [1] and archlinux’s > base/split packages [2]. On the importance of visiting museum! :-) Cheers, simon
[-- Attachment #1: Type: text/plain, Size: 2850 bytes --] On Sun, 15 Nov 2020 22:24:29 +0100 raingloom <raingloom@riseup.net> wrote: > Alpine already achieves an incredibly tiny install size by splitting > packages into many outputs. We could and should do the same. > As far as I know, they do not have parameterized packages. That also depends on how far you want to go. Last time I looked into how LibreCMC/OpenWRT did that, they had much more optimization than that. If I recall well, they use at least: - sstrip to strip binaries as much as they could. sstrip produces smaller binaries than with strip. - compilation flags like -Os - a read-only compressed filesystem with an overlay to store the changes The issue is that despite all that, the size of the images tend to increase too rapidly over time[1]. If we manage to shrink Guix enough, it might be possible to use it on way more devices, including RYF compliant devices or potentially certifiable devices: - The Talos II BMC has 32M according to both the wiki[2] and the image sizes[4]. Its architecture is ARM. So once we have the PPC64 architecture working, it would be great to be able to run Guix both in the BMC and on the PowerPC CPU. That BMC is also available on other mainboards like the D16 which is supported by Libreboot, but the flash size is probably even smaller there. - Many WiFi access point have very few flash space. It can boils down to as low as 16M for LibreCMC/OpenWRT compatible devices, or even 8M for older devices. However they typically use the MIPS architecture which isn't supported yet in Guix. - There is a GNU/Linux distribution[6] that runs inside the flash chip where Libreboot or Coreboot typically runs. The goal is to enable more flexible and/or secure booting by using GNU/Linux to boot GNU/Linux. Here too the flash chip of computers supported by Libreboot can be quite small, like 8M for Thinkpads with GM45 chipsets. In some case it might be possible to increase the flash chip size (sometimes you don't need soldering for that), but at least with x86 mainboards, the chipset has limits on the size of the flash chip that it can see. And the size cannot be increased that much: The biggest flash chip that flashrom supports is 256M. References: ----------- [1]https://openwrt.org/supported_devices/864_warning [2]The wiki[3] mention a MX25L25635F/MX25L25645E/MX25L25665E flash chip which is 32M according to flashrom -L [3]https://wiki.raptorcs.com/wiki/Debricking_the_BMC#Flash_new_BMC_firmware_via_serial_port_.28Open_Source_Method.29 [4]Once uncompressed the image[5] size (for installation through the shell) is 32M. [5]https://wiki.raptorcs.com/wiki/File:Talos-ii-openbmc-v2.00-bundle.tar [6]https://github.com/osresearch/heads/ [7]https://github.com/osresearch/heads/tree/master/config Denis. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --]
Hi, Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> skribis: > Last time I looked into how LibreCMC/OpenWRT did that, they had much > more optimization than that. If I recall well, they use at least: > - sstrip to strip binaries as much as they could. sstrip produces > smaller binaries than with strip. > - compilation flags like -Os > - a read-only compressed filesystem with an overlay to store the > changes To me this looks like the ultimate size optimization level. Before we get there, we should first see how to get closer to package sizes typically found on Debian and that alone is a real challenge. > The issue is that despite all that, the size of the images tend to > increase too rapidly over time[1]. Yeah, that’s also the problem here: we have ‘guix size’ to profile a package at one point in time, but it’s easy to unwillingly increase its closure size the next day without noticing. Chris: does the Data Service track store item sizes (and more generally everything ‘query-path-info’ returns)? It’d be great to be able to visualize size plots over time! Ludo’.
Hi, On Fri, 20 Nov 2020 at 12:40, Ludovic Courtès <ludo@gnu.org> wrote: > > The issue is that despite all that, the size of the images tend to > > increase too rapidly over time[1]. > > Yeah, that’s also the problem here: we have ‘guix size’ to profile a > package at one point in time, but it’s easy to unwillingly increase its > closure size the next day without noticing. I think that should be part of the tooling we need to ease the "release process". I mean, that's what I have tried to describe as one starting point at the BoF discussion: tools that help to know how is the shape of Guix at one point in time (via scripts and repl or via commands and options) and then time-machine does the rest. What is currently missing (from my little experience): - check reproducibility - check size - check substitutes per build system, list the "essentials" (the ones we *absolutly* want to be substituable), maybe per groupes by topics > Chris: does the Data Service track store item sizes (and more generally > everything ‘query-path-info’ returns)? It’d be great to be able to > visualize size plots over time! Yeah, plot could be really useful. All the best, simon
[-- Attachment #1: Type: text/plain, Size: 979 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Hi, > > Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> skribis: > >> The issue is that despite all that, the size of the images tend to >> increase too rapidly over time[1]. > > Yeah, that’s also the problem here: we have ‘guix size’ to profile a > package at one point in time, but it’s easy to unwillingly increase its > closure size the next day without noticing. > > Chris: does the Data Service track store item sizes (and more generally > everything ‘query-path-info’ returns)? It’d be great to be able to > visualize size plots over time! The Guix Data Service can store narinfos, which I believe contain all the information you need to do something similar to what `guix size` does. I believe you'd be able to track the size of individual items, as well as closure size through the reference information. So, yeah, I think it's definately possible to do what you describe. Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --]