* Cross-compilation broken on canonical packages. @ 2019-12-14 11:49 Mathieu Othacehe 2019-12-14 11:52 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2019-12-14 11:49 UTC (permalink / raw) To: guix-devel Hello, This command fails: --8<---------------cut here---------------start------------->8--- guix build -e "((@ (gnu packages base) canonical-package) (@ (gnu packages base) grep))" --8<---------------cut here---------------end--------------->8--- with this output: --8<---------------cut here---------------start------------->8--- building /gnu/store/nlq0iw749hnyy7nk3cq6pjwqkxv437ca-make-boot0-4.2.1.drv... starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds starting phase `set-paths' environment variable `PATH' set to `/gnu/store/36zgy21hfy185nmdgj87ll6q4hj5riiw-gcc-cross-aarch64-linux-gnu-7.4.0/bin:/gnu/store/4d68mhjjwqc95lyrm63yrsnrlqzkbpyr-binutils-cross-aarch64-linux-gnu-2.32/bin' environment variable `CROSS_LIBRARY_PATH' set to `/gnu/store/pgg0nxh1l39dybb5pwqyskb9qxqz57r9-glibc-mesboot-2.16.0/lib:/gnu/store/p0s4wym402gsnqzicr87zrli43dvfbms-binutils-mesboot-2.20.1a/lib:/gnu/store/bb6s2y6121pxvhwyr1n6dg7pwj88bsbj-gcc-mesboot-4.9.4/lib:/gnu/store/p5r7gprzg774vq8l94924m598i86ryya-glibc-cross-aarch64-linux-gnu-2.29/lib:/gnu/store/z86g7zwj2mdl1328qpgmim262sppjxqp-glibc-cross-aarch64-linux-gnu-2.29-static/lib' environment variable `CROSS_CPATH' set to `/gnu/store/pgg0nxh1l39dybb5pwqyskb9qxqz57r9-glibc-mesboot-2.16.0/include:/gnu/store/p0s4wym402gsnqzicr87zrli43dvfbms-binutils-mesboot-2.20.1a/include:/gnu/store/bb6s2y6121pxvhwyr1n6dg7pwj88bsbj-gcc-mesboot-4.9.4/include:/gnu/store/bp1ap2whzmwvh2myj31hfmg9pi4sly0r-linux-libre-headers-bootstrap-0/include:/gnu/store/p5r7gprzg774vq8l94924m598i86ryya-glibc-cross-aarch64-linux-gnu-2.29/include:/gnu/store/f3izsng5fzwh6zf2z7aaj5v04jp5hmcy-linux-libre-headers-cross-aarch64-linux-gnu-4.19.56/include' environment variable `GUIX_LOCPATH' unset environment variable `C_INCLUDE_PATH' set to `/gnu/store/36zgy21hfy185nmdgj87ll6q4hj5riiw-gcc-cross-aarch64-linux-gnu-7.4.0/include' environment variable `CPLUS_INCLUDE_PATH' set to `/gnu/store/36zgy21hfy185nmdgj87ll6q4hj5riiw-gcc-cross-aarch64-linux-gnu-7.4.0/include' environment variable `LIBRARY_PATH' set to `/gnu/store/36zgy21hfy185nmdgj87ll6q4hj5riiw-gcc-cross-aarch64-linux-gnu-7.4.0/lib' environment variable `C_INCLUDE_PATH' set to `/gnu/store/36zgy21hfy185nmdgj87ll6q4hj5riiw-gcc-cross-aarch64-linux-gnu-7.4.0/include' phase `set-paths' succeeded after 0.0 seconds starting phase `install-locale' warning: failed to install 'en_US.utf8' locale: Invalid argument phase `install-locale' succeeded after 0.0 seconds starting phase `unpack' In execvp of tar: No such file or directory command "tar" "xvf" "/gnu/store/cr04i9xi5nbkn7lqb1nnxkqvpwy92m78-make-4.2.1.tar.xz" failed with status 127 builder for `/gnu/store/nlq0iw749hnyy7nk3cq6pjwqkxv437ca-make-boot0-4.2.1.drv' failed with exit code 1 build of /gnu/store/nlq0iw749hnyy7nk3cq6pjwqkxv437ca-make-boot0-4.2.1.drv failed View build log at '/var/log/guix/drvs/nl/q0iw749hnyy7nk3cq6pjwqkxv437ca-make-boot0-4.2.1.drv.gz'. cannot build derivation `/gnu/store/xpb7jhgjghxy624gz7rfy5zb8rsdri3i-grep-3.3.drv': 1 dependencies couldn't be built guix build: error: build of `/gnu/store/xpb7jhgjghxy624gz7rfy5zb8rsdri3i-grep-3.3.drv' failed --8<---------------cut here---------------end--------------->8--- This is topic has already been discussed here[1][2]. Usually I work around it by passing "--no-grafts" or avoiding canonical-packages. Now that cross compiling a system is possible, avoiding canonical packages is hard (used in %base-packages and "system" derivation). Any ideas where to start on this problem? Thanks, Mathieu [1]: https://guix-devel.gnu.narkive.com/LsByvXrK/cross-building-bootstrap-binaries-fail-in-current-master#post5 [2]: https://lists.gnu.org/archive/html/bug-guix/2017-10/msg00101.html ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-14 11:49 Cross-compilation broken on canonical packages Mathieu Othacehe @ 2019-12-14 11:52 ` Mathieu Othacehe 2019-12-19 16:48 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2019-12-14 11:52 UTC (permalink / raw) To: guix-devel Small mistake sorry. This fails: --8<---------------cut here---------------start------------->8--- guix build --target=aarch64-linux-gnu -e "((@ (gnu packages base) canonical-package) (@ (gnu packages base) grep))" --8<---------------cut here---------------end--------------->8--- while this succeeds: --8<---------------cut here---------------start------------->8--- guix build -e "((@ (gnu packages base) canonical-package) (@ (gnu packages base) grep))" --8<---------------cut here---------------end--------------->8--- Mathieu ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-14 11:52 ` Mathieu Othacehe @ 2019-12-19 16:48 ` Ludovic Courtès 2019-12-22 16:31 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2019-12-19 16:48 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: guix-devel Hi Mathieu, Mathieu Othacehe <m.othacehe@gmail.com> skribis: > Small mistake sorry. This fails: > > guix build --target=aarch64-linux-gnu -e "((@ (gnu packages base) canonical-package) (@ (gnu packages base) grep))" > > > while this succeeds: > > guix build -e "((@ (gnu packages base) canonical-package) (@ (gnu packages base) grep))" This is expected: packages in ‘%final-inputs’ (those returned by ‘canonical-package’) are rooted in the bootstrap graph and cannot be cross-compiled. Perhaps ‘%base-packages’ should use packages form (gnu packages base) instead? The reason it uses ‘canonical-package’ is just to avoid introducing redundant packages in the system, but I don’t think it makes that much sense. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-19 16:48 ` Ludovic Courtès @ 2019-12-22 16:31 ` Mathieu Othacehe 2019-12-30 18:48 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2019-12-22 16:31 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1855 bytes --] Hello Ludo, Thanks for your explanation :) > This is expected: packages in ‘%final-inputs’ (those returned by > ‘canonical-package’) are rooted in the bootstrap graph and cannot be > cross-compiled. Looking at canonical-package in (gnu packages commencement), I see that there's already a switch on (%current-target-system). The given package is directly returned if (%current-target-system) is set, which appears to be what we want! --8<---------------cut here---------------start------------->8--- ;; In general we want CANON, except if we're cross-compiling: CANON ;; uses explicit inputs, so it is "anchored" in the bootstrapped ;; process, with dependencies on things that cannot be ;; cross-compiled. (if (%current-target-system) package canon)) --8<---------------cut here---------------end--------------->8--- But, this doesn't work as expected. I guess it is because of (%current-target-system) evaluation time. As I'm not fully understand everything here, I would propose to define a gexp-compiler for "canonical-packages", but there's maybe a better thing to do? Anyway, the snippet below works fine with a gexp-compiler (patch attached) and doesn't work with the current implementation of canonical-package. --8<---------------cut here---------------start------------->8--- (use-modules (guix) (gnu packages base)) (run-with-store (open-connection) (mlet* %store-monad ((gexp -> #~(#$(canonical-package grep))) (drv (gexp->script "test.scm" gexp #:target "aarch64-linux-gnu")) (build (built-derivations (list drv)))) (return #t))) --8<---------------cut here---------------end--------------->8--- WDYT? Thanks, Mathieu [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-wip-Add-a-canonical-package-gexp-compiler.patch --] [-- Type: text/x-diff, Size: 3698 bytes --] From 6266c46181d2880684c89999519423be8d8a8ea3 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe <m.othacehe@gmail.com> Date: Sun, 22 Dec 2019 17:25:59 +0100 Subject: [PATCH] wip: Add a canonical-package gexp compiler. --- gnu/packages/base.scm | 6 ++++++ gnu/packages/commencement.scm | 40 +++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm index e8150708c0..22b6e05dae 100644 --- a/gnu/packages/base.scm +++ b/gnu/packages/base.scm @@ -1337,6 +1337,12 @@ package needs iconv ,@(libiconv-if-needed) should be added." (proc (module-ref iface 'canonical-package))) (proc package))) +(define-public (canonical-package* package) + ;; Avoid circular dependency by lazily resolving 'commencement'. + (let* ((iface (resolve-interface '(gnu packages commencement))) + (proc (module-ref iface 'make-canonical-package))) + (proc package))) + (define-public (%final-inputs) "Return the list of \"final inputs\"." ;; Avoid circular dependency by lazily resolving 'commencement'. diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm index 14ecf246d4..41ab430388 100644 --- a/gnu/packages/commencement.scm +++ b/gnu/packages/commencement.scm @@ -54,8 +54,11 @@ #:use-module (guix build-system gnu) #:use-module (guix build-system trivial) #:use-module (guix memoization) + #:use-module (guix gexp) + #:use-module (guix records) #:use-module (guix utils) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) #:use-module (srfi srfi-26) #:use-module (ice-9 vlist) #:use-module (ice-9 match) @@ -2613,5 +2616,42 @@ Fortran development to be installed in user profiles. This includes gfortran, as well as libc (headers and binaries, plus debugging symbols in the @code{debug} output), and binutils."))) +(define-record-type <canonical-package> + (%make-canonical-package name) + canonical-package? + (name canonical-package-name)) + +(define-public (make-canonical-package package) + (%make-canonical-package package)) + +(define-gexp-compiler (canonical-package-compiler + (canonical-package <canonical-package>) system target) + ;; Return the 'canonical' variant of PACKAGE---i.e., if PACKAGE is one of + ;; the implicit inputs of 'gnu-build-system', return that one, otherwise + ;; return PACKAGE. + ;; + ;; The goal is to avoid duplication in cases like GUILE-FINAL vs. GUILE-2.2, + ;; COREUTILS-FINAL vs. COREUTILS, etc. + (let ((name->package (fold (lambda (input result) + (match input + ((_ package . outputs) + (vhash-cons (package-full-name package) + package result)))) + vlist-null + `(("guile" ,guile-final) + ,@%final-inputs))) + (package (canonical-package-name canonical-package))) + ;; XXX: This doesn't handle dependencies of the final inputs, such as + ;; libunistring, GMP, etc. + (match (vhash-assoc (package-full-name package) name->package) + ((_ . canon) + ;; In general we want CANON, except if we're cross-compiling: CANON + ;; uses explicit inputs, so it is "anchored" in the bootstrapped + ;; process, with dependencies on things that cannot be + ;; cross-compiled. + (if target + (package->cross-derivation package target system) + (package->derivation (pk canon) system))) + (_ (package->derivation package system))))) ;;; commencement.scm ends here -- 2.24.0 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-22 16:31 ` Mathieu Othacehe @ 2019-12-30 18:48 ` Ludovic Courtès 2019-12-31 9:55 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2019-12-30 18:48 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: guix-devel Hi, Mathieu Othacehe <m.othacehe@gmail.com> skribis: >> This is expected: packages in ‘%final-inputs’ (those returned by >> ‘canonical-package’) are rooted in the bootstrap graph and cannot be >> cross-compiled. > > Looking at canonical-package in (gnu packages commencement), I see that > there's already a switch on (%current-target-system). The given package > is directly returned if (%current-target-system) is set, which appears > to be what we want! > > ;; In general we want CANON, except if we're cross-compiling: CANON > ;; uses explicit inputs, so it is "anchored" in the bootstrapped > ;; process, with dependencies on things that cannot be > ;; cross-compiled. > (if (%current-target-system) > package > canon)) > > > But, this doesn't work as expected. I guess it is because of > (%current-target-system) evaluation time. Ah yeah. The hack works when you use ‘canonical-package’ in a package input, because package input fields are “thunked”, so they’re evaluated in a dynamic context where ‘%current-target-system’ has the right value. However, the hack doesn’t work for things like the ‘packages’ field of <operating-system> because it’s not a thunked field. Two simple solutions here, I think: 1. Make ‘packages’ a thunked field. 2. Stop using ‘canonical-package’ altogether in ‘%base-packages’. I actually have a preference for #2. We’d need to check what impact it has on the system closure size, but I suspect it’s going to be minimal. Thoughts? (The <canonical-package> trick you posted should work of course, but it’s even better if we can avoid it!) Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-30 18:48 ` Ludovic Courtès @ 2019-12-31 9:55 ` Mathieu Othacehe 2020-01-02 18:07 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2019-12-31 9:55 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 973 bytes --] Hola, > However, the hack doesn’t work for things like the ‘packages’ field of > <operating-system> because it’s not a thunked field. I see! Thanks for explaining. > > Two simple solutions here, I think: > > 1. Make ‘packages’ a thunked field. > > 2. Stop using ‘canonical-package’ altogether in ‘%base-packages’. > > I actually have a preference for #2. We’d need to check what impact it > has on the system closure size, but I suspect it’s going to be minimal. > > Thoughts? Option #2 seems nicer. There are other canonical-package calls when building services. I have a patch attached that removes all those calls. The image produced when running "guix system disk-image bare-bones.tmpl" increases by: * 30MB when removing the canonical-package of %base-packages * 30MB for all other calls. So altogether, that's a 60MB increase on a 1.5GB image (4%). I find it acceptable, WDYT? Thanks, Mathieu [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-system-Stop-using-canonical-package.patch --] [-- Type: text/x-diff, Size: 8022 bytes --] From a55eb86975c5f1c2783b0af265364821259e3351 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe <m.othacehe@gmail.com> Date: Tue, 31 Dec 2019 09:56:51 +0100 Subject: [PATCH] system: Stop using canonical-package. Usage of canonical-package outside of thunked fields breaks cross-compilation, see: https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00410.html. * gnu/installer.scm (installer-program): Remove canonical-package. * gnu/services/base.scm (<nscd-cache>): Ditto, (%base-services): ditto. * gnu/services/xorg.scm: Remove useless canonical-package import. * gnu/system.scm (%base-packages): Remove canonical-package. * gnu/system/install.scm (%installation-services): Ditto, (installation-os): ditto. * gnu/system/locale.scm (single-locale-directory): Ditto, (%default-locale-libcs): ditto. --- gnu/installer.scm | 22 +++++++++++----------- gnu/services/base.scm | 10 ++++------ gnu/services/xorg.scm | 2 +- gnu/system.scm | 11 ++--------- gnu/system/install.scm | 7 +++---- gnu/system/locale.scm | 4 ++-- 6 files changed, 23 insertions(+), 33 deletions(-) diff --git a/gnu/installer.scm b/gnu/installer.scm index 167653263f..7f12b39ffc 100644 --- a/gnu/installer.scm +++ b/gnu/installer.scm @@ -290,17 +290,17 @@ selected keymap." (define set-installer-path ;; Add the specified binary to PATH for later use by the installer. #~(let* ((inputs - '#$(append (list bash ;start subshells - connman ;call connmanctl - cryptsetup - dosfstools ;mkfs.fat - e2fsprogs ;mkfs.ext4 - btrfs-progs - kbd ;chvt - guix ;guix system init call - util-linux ;mkwap - shadow) - (map canonical-package (list coreutils))))) + '#$(list bash ;start subshells + connman ;call connmanctl + cryptsetup + dosfstools ;mkfs.fat + e2fsprogs ;mkfs.ext4 + btrfs-progs + kbd ;chvt + guix ;guix system init call + util-linux ;mkwap + shadow + coreutils))) (with-output-to-port (%make-void-port "w") (lambda () (set-path-environment-variable "PATH" '("bin" "sbin") inputs))))) diff --git a/gnu/services/base.scm b/gnu/services/base.scm index b1eff89ecc..b7cde09611 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -44,7 +44,7 @@ #:select (alsa-utils crda eudev e2fsprogs fuse gpm kbd lvm2 rng-tools)) #:use-module (gnu packages bash) #:use-module ((gnu packages base) - #:select (canonical-package coreutils glibc glibc-utf8-locales)) + #:select (coreutils glibc glibc-utf8-locales)) #:use-module (gnu packages package-management) #:use-module ((gnu packages gnupg) #:select (guile-gcrypt)) #:use-module (gnu packages linux) @@ -1208,7 +1208,7 @@ the tty to run, among other things." (name-services nscd-configuration-name-services ;list of <packages> (default '())) (glibc nscd-configuration-glibc ;<package> - (default (canonical-package glibc)))) + (default glibc))) (define-record-type* <nscd-cache> nscd-cache make-nscd-cache nscd-cache? @@ -2441,9 +2441,7 @@ to handle." (rules (list lvm2 fuse alsa-utils crda)))) (service special-files-service-type - `(("/bin/sh" ,(file-append (canonical-package bash) - "/bin/sh")) - ("/usr/bin/env" ,(file-append (canonical-package coreutils) - "/bin/env")))))) + `(("/bin/sh" ,(file-append bash "/bin/sh")) + ("/usr/bin/env" ,(file-append coreutils "/bin/env")))))) ;;; base.scm ends here diff --git a/gnu/services/xorg.scm b/gnu/services/xorg.scm index 9c84f7413f..4f6a8aec1e 100644 --- a/gnu/services/xorg.scm +++ b/gnu/services/xorg.scm @@ -29,7 +29,7 @@ #:use-module (gnu system keyboard) #:use-module (gnu services base) #:use-module (gnu services dbus) - #:use-module ((gnu packages base) #:select (canonical-package)) + #:use-module (gnu packages base) #:use-module (gnu packages guile) #:use-module (gnu packages xorg) #:use-module (gnu packages fonts) diff --git a/gnu/system.scm b/gnu/system.scm index abdbb081e6..3efa496194 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -591,18 +591,11 @@ of PROVENANCE-SERVICE-TYPE to its services." bash-completion - ;; XXX: We don't use (canonical-package guile-2.2) here because that - ;; would create a collision in the global profile between the GMP - ;; variant propagated by 'guile-final' and the GMP variant propagated - ;; by 'gnutls', itself propagated by 'guix'. guile-2.2 guile-readline guile-colorized - ;; The packages below are also in %FINAL-INPUTS, so take them from - ;; there to avoid duplication. - (map canonical-package - (list bash coreutils findutils grep sed - diffutils patch gawk tar gzip bzip2 xz lzip)))) + bash coreutils findutils grep sed + diffutils patch gawk tar gzip bzip2 xz lzip)) (define %default-issue ;; Default contents for /etc/issue. diff --git a/gnu/system/install.scm b/gnu/system/install.scm index 4d1612ac7f..64fd1d1747 100644 --- a/gnu/system/install.scm +++ b/gnu/system/install.scm @@ -383,8 +383,7 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m ;; Having /bin/sh is a good idea. In particular it allows Tramp ;; connections to this system to work. (service special-files-service-type - `(("/bin/sh" ,(file-append (canonical-package bash) - "/bin/sh")))) + `(("/bin/sh" ,(file-append bash "/bin/sh")))) ;; Loopback device, needed by OpenSSH notably. (service static-networking-service-type @@ -407,7 +406,7 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m (list bare-bones-os glibc-utf8-locales texinfo - (canonical-package guile-2.2)))))) + guile-2.2))))) (define %issue ;; Greeting. @@ -479,7 +478,7 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m ;; Explicitly allow for empty passwords. (base-pam-services #:allow-empty-passwords? #t)) - (packages (cons* (canonical-package glibc) ;for 'tzselect' & co. + (packages (cons* glibc ;for 'tzselect' & co. parted gptfdisk ddrescue fontconfig font-dejavu font-gnu-unifont diff --git a/gnu/system/locale.scm b/gnu/system/locale.scm index 8466d5b07d..c7a371e9bf 100644 --- a/gnu/system/locale.scm +++ b/gnu/system/locale.scm @@ -86,7 +86,7 @@ or #f on failure." #f))) (define* (single-locale-directory locales - #:key (libc (canonical-package glibc))) + #:key (libc glibc)) "Return a directory containing all of LOCALES for LIBC compiled. Because locale data formats are incompatible when switching from one libc to @@ -147,7 +147,7 @@ data format changes between libc versions." (define %default-locale-libcs ;; The libcs for which we build locales by default. - (list (canonical-package glibc))) + (list glibc)) (define %default-locale-definitions ;; Arbitrary set of locales that are built by default. They are here mostly -- 2.24.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2019-12-31 9:55 ` Mathieu Othacehe @ 2020-01-02 18:07 ` Ludovic Courtès 2020-01-02 22:00 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2020-01-02 18:07 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: guix-devel Hola! Mathieu Othacehe <m.othacehe@gmail.com> skribis: >> Two simple solutions here, I think: >> >> 1. Make ‘packages’ a thunked field. >> >> 2. Stop using ‘canonical-package’ altogether in ‘%base-packages’. >> >> I actually have a preference for #2. We’d need to check what impact it >> has on the system closure size, but I suspect it’s going to be minimal. >> >> Thoughts? > > Option #2 seems nicer. There are other canonical-package calls when > building services. I have a patch attached that removes all those calls. > > The image produced when running "guix system disk-image bare-bones.tmpl" > increases by: > > * 30MB when removing the canonical-package of %base-packages > * 30MB for all other calls. OK. (It may be slightly more accurate to look at the output of: guix size $(guix system build …).) > So altogether, that's a 60MB increase on a 1.5GB image (4%). I find it > acceptable, WDYT? It’s more than I thought but I think it’s OK. (Also, how come bare-bones takes 1.5 GiB?!) > From a55eb86975c5f1c2783b0af265364821259e3351 Mon Sep 17 00:00:00 2001 > From: Mathieu Othacehe <m.othacehe@gmail.com> > Date: Tue, 31 Dec 2019 09:56:51 +0100 > Subject: [PATCH] system: Stop using canonical-package. > > Usage of canonical-package outside of thunked fields breaks cross-compilation, > see: https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00410.html. > > * gnu/installer.scm (installer-program): Remove canonical-package. > * gnu/services/base.scm (<nscd-cache>): Ditto, > (%base-services): ditto. > * gnu/services/xorg.scm: Remove useless canonical-package import. > * gnu/system.scm (%base-packages): Remove canonical-package. > * gnu/system/install.scm (%installation-services): Ditto, > (installation-os): ditto. > * gnu/system/locale.scm (single-locale-directory): Ditto, > (%default-locale-libcs): ditto. [...] > --- a/gnu/system/locale.scm > +++ b/gnu/system/locale.scm > @@ -86,7 +86,7 @@ or #f on failure." > #f))) > > (define* (single-locale-directory locales > - #:key (libc (canonical-package glibc))) > + #:key (libc glibc)) > "Return a directory containing all of LOCALES for LIBC compiled. > > Because locale data formats are incompatible when switching from one libc to > @@ -147,7 +147,7 @@ data format changes between libc versions." > > (define %default-locale-libcs > ;; The libcs for which we build locales by default. > - (list (canonical-package glibc))) > + (list glibc)) I would leave this bit unchanged if possible, because that way the glibc used would really match that programs are linked against. The rest LGTM, thanks! Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-01-02 18:07 ` Ludovic Courtès @ 2020-01-02 22:00 ` Mathieu Othacehe 2020-01-03 12:03 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2020-01-02 22:00 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Hi, > It’s more than I thought but I think it’s OK. (Also, how come > bare-bones takes 1.5 GiB?!) That's one of my next subject of investigation :) >> (define %default-locale-libcs >> ;; The libcs for which we build locales by default. >> - (list (canonical-package glibc))) >> + (list glibc)) > > I would leave this bit unchanged if possible, because that way the glibc > used would really match that programs are linked against. operating-system-locale-libcs defaults to %default-locale-libcs, so keeping (canonical-package glibc) prevents cross-compilation :( We could use a thunk field to write something like: --8<---------------cut here---------------start------------->8--- (define (default-locale-libcs ...) (if target glibc (canonical-package glibc)) --8<---------------cut here---------------end--------------->8--- But I'd like to avoid that. I'll see if I can figure out something better after a good night. Thanks, Mathieu ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-01-02 22:00 ` Mathieu Othacehe @ 2020-01-03 12:03 ` Mathieu Othacehe 2020-02-11 13:01 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2020-01-03 12:03 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 472 bytes --] Hello, > We could use a thunk field to write something like: > > --8<---------------cut here---------------start------------->8--- > (define (default-locale-libcs ...) > (if target glibc (canonical-package glibc)) > --8<---------------cut here---------------end--------------->8--- > > But I'd like to avoid that. I'll see if I can figure out something > better after a good night. Couldn't find much better, here's a patch, tell my what you think! Thanks, Mathieu [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-defaut-locale-libc.patch --] [-- Type: text/x-diff, Size: 2253 bytes --] From e1728d6c79f63ad72bcfb4e45111ccb9f43f1251 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe <m.othacehe@gmail.com> Date: Fri, 3 Jan 2020 09:23:51 +0100 Subject: [PATCH] defaut locale libc. --- gnu/system.scm | 3 ++- gnu/system/locale.scm | 10 +++++----- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/gnu/system.scm b/gnu/system.scm index 0842bd47c7..faff0320d3 100644 --- a/gnu/system.scm +++ b/gnu/system.scm @@ -211,7 +211,8 @@ (locale-definitions operating-system-locale-definitions ; list of <locale-definition> (default %default-locale-definitions)) (locale-libcs operating-system-locale-libcs ; list of <packages> - (default %default-locale-libcs)) + (thunked) + (default (default-locale-libcs))) (name-service-switch operating-system-name-service-switch ; <name-service-switch> (default %default-nss)) diff --git a/gnu/system/locale.scm b/gnu/system/locale.scm index c7a371e9bf..361c024eeb 100644 --- a/gnu/system/locale.scm +++ b/gnu/system/locale.scm @@ -37,7 +37,7 @@ locale-name->definition locale-directory - %default-locale-libcs + default-locale-libcs %default-locale-definitions glibc-supported-locales)) @@ -123,7 +123,7 @@ of LIBC." (computed-file (string-append "locale-" version) build)) (define* (locale-directory locales - #:key (libcs %default-locale-libcs)) + #:key (libcs (default-locale-libcs))) "Return a locale directory containing all of LOCALES for each libc package listed in LIBCS. @@ -145,9 +145,9 @@ data format changes between libc versions." #:options '(#:local-build? #t #:substitutable? #f)))))) -(define %default-locale-libcs - ;; The libcs for which we build locales by default. - (list glibc)) +(define* (default-locale-libcs + #:key (target (%current-target-system))) + (if target (list glibc) (list (canonical-package glibc)))) (define %default-locale-definitions ;; Arbitrary set of locales that are built by default. They are here mostly -- 2.24.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-01-03 12:03 ` Mathieu Othacehe @ 2020-02-11 13:01 ` Mathieu Othacehe 2020-02-11 14:04 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2020-02-11 13:01 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Hey Ludo, >> But I'd like to avoid that. I'll see if I can figure out something >> better after a good night. > > Couldn't find much better, here's a patch, tell my what you think! I pushed a patch removing all canonical-packages calls, except the one for the default-locales-libcs variable. About this variable, I don't know if the dirty patch I proposed is better than accepting that the glibc used for locales is different from the one programs are linked against. WDYT? Thanks, Mathieu ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-02-11 13:01 ` Mathieu Othacehe @ 2020-02-11 14:04 ` Ludovic Courtès 2020-02-12 10:20 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2020-02-11 14:04 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: guix-devel Hi, Mathieu Othacehe <m.othacehe@gmail.com> skribis: >>> But I'd like to avoid that. I'll see if I can figure out something >>> better after a good night. >> >> Couldn't find much better, here's a patch, tell my what you think! > > I pushed a patch removing all canonical-packages calls, except the one > for the default-locales-libcs variable. > > About this variable, I don't know if the dirty patch I proposed is > better than accepting that the glibc used for locales is different from > the one programs are linked against. WDYT? On second thought, I think removing ‘canonical-package’ calls from (gnu system locale) should work. What matters there is that we build locales with the matching libc (same version, not necessarily same store item.) The only downside will be potentially an extra glibc download/build to build the locale set, but that’s probably OK. WDYT? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-02-11 14:04 ` Ludovic Courtès @ 2020-02-12 10:20 ` Mathieu Othacehe 2020-02-24 16:11 ` Ludovic Courtès 0 siblings, 1 reply; 14+ messages in thread From: Mathieu Othacehe @ 2020-02-12 10:20 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 566 bytes --] Hello, > The only downside will be potentially an extra glibc download/build to > build the locale set, but that’s probably OK. > > WDYT? I think you're right! Pushed it as f30d84d32db0f4f6cb84e139868e1727a7dc0a51 on core-updates. Now the good news is that we are one patch away from having core-updates to cross-compile bare-bones.tmpl system! This is the patch attached, we've been discussing it here[1] without reaching a conclusion. WDYT? Thanks, Mathieu [1]: https://lists.gnu.org/archive/html/guix-patches/2019-12/msg00751.html [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-gexp-Default-to-current-target.patch --] [-- Type: text/x-diff, Size: 7340 bytes --] From f5d773dc0a627ba6059f1025189d33a9e0d083b5 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe <m.othacehe@gmail.com> Date: Sat, 28 Dec 2019 21:29:06 +0100 Subject: [PATCH 1/2] gexp: Default to current target. * guix/gexp.scm (lower-object): Set target argument to 'current by default and look for the current target system at bind time if needed, (gexp->file): ditto, (gexp->script): ditto. --- guix/gexp.scm | 87 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 51 insertions(+), 36 deletions(-) diff --git a/guix/gexp.scm b/guix/gexp.scm index 12331052a6..11d4e86037 100644 --- a/guix/gexp.scm +++ b/guix/gexp.scm @@ -215,7 +215,7 @@ procedure to expand it; otherwise return #f." (define* (lower-object obj #:optional (system (%current-system)) - #:key target) + #:key (target 'current)) "Return as a value in %STORE-MONAD the derivation or store item corresponding to OBJ for SYSTEM, cross-compiling for TARGET if TARGET is true. OBJ must be an object that has an associated gexp compiler, such as a @@ -225,7 +225,10 @@ OBJ must be an object that has an associated gexp compiler, such as a (raise (condition (&gexp-input-error (input obj))))) (lower ;; Cache in STORE the result of lowering OBJ. - (mlet %store-monad ((graft? (grafting?))) + (mlet %store-monad ((target (if (eq? target 'current) + (current-target-system) + (return target))) + (graft? (grafting?))) (mcached (let ((lower (lookup-compiler obj))) (lower obj system target)) obj @@ -1571,16 +1574,19 @@ are searched for in PATH. Return #f when MODULES and EXTENSIONS are empty." #:key (guile (default-guile)) (module-path %load-path) (system (%current-system)) - target) + (target 'current)) "Return an executable script NAME that runs EXP using GUILE, with EXP's imported modules in its search path. Look up EXP's modules in MODULE-PATH." - (mlet %store-monad ((set-load-path - (load-path-expression (gexp-modules exp) - module-path - #:extensions - (gexp-extensions exp) - #:system system - #:target target))) + (mlet* %store-monad ((target (if (eq? target 'current) + (current-target-system) + (return target))) + (set-load-path + (load-path-expression (gexp-modules exp) + module-path + #:extensions + (gexp-extensions exp) + #:system system + #:target target))) (gexp->derivation name (gexp (call-with-output-file (ungexp output) @@ -1609,7 +1615,7 @@ imported modules in its search path. Look up EXP's modules in MODULE-PATH." (module-path %load-path) (splice? #f) (system (%current-system)) - target) + (target 'current)) "Return a derivation that builds a file NAME containing EXP. When SPLICE? is true, EXP is considered to be a list of expressions that will be spliced in the resulting file. @@ -1620,36 +1626,45 @@ Lookup EXP's modules in MODULE-PATH." (define modules (gexp-modules exp)) (define extensions (gexp-extensions exp)) - (if (or (not set-load-path?) - (and (null? modules) (null? extensions))) - (gexp->derivation name - (gexp - (call-with-output-file (ungexp output) - (lambda (port) - (for-each (lambda (exp) - (write exp port)) - '(ungexp (if splice? - exp - (gexp ((ungexp exp))))))))) - #:local-build? #t - #:substitutable? #f - #:system system - #:target target) - (mlet %store-monad ((set-load-path - (load-path-expression modules module-path - #:extensions extensions - #:system system - #:target target))) + (mlet* %store-monad + ((target (if (eq? target 'current) + (current-target-system) + (return target))) + (no-load-path? -> (or (not set-load-path?) + (and (null? modules) + (null? extensions)))) + (set-load-path + (or (return no-load-path?) + (load-path-expression modules module-path + #:extensions extensions + #:system system + #:target target)))) + (if no-load-path? + (gexp->derivation name + (gexp + (call-with-output-file (ungexp output) + (lambda (port) + (for-each + (lambda (exp) + (write exp port)) + '(ungexp (if splice? + exp + (gexp ((ungexp exp))))))))) + #:local-build? #t + #:substitutable? #f + #:system system + #:target target) (gexp->derivation name (gexp (call-with-output-file (ungexp output) (lambda (port) (write '(ungexp set-load-path) port) - (for-each (lambda (exp) - (write exp port)) - '(ungexp (if splice? - exp - (gexp ((ungexp exp))))))))) + (for-each + (lambda (exp) + (write exp port)) + '(ungexp (if splice? + exp + (gexp ((ungexp exp))))))))) #:module-path module-path #:local-build? #t #:substitutable? #f -- 2.24.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-02-12 10:20 ` Mathieu Othacehe @ 2020-02-24 16:11 ` Ludovic Courtès 2020-03-06 9:19 ` Mathieu Othacehe 0 siblings, 1 reply; 14+ messages in thread From: Ludovic Courtès @ 2020-02-24 16:11 UTC (permalink / raw) To: Mathieu Othacehe; +Cc: guix-devel Hi! Mathieu Othacehe <m.othacehe@gmail.com> skribis: > I think you're right! Pushed it as > f30d84d32db0f4f6cb84e139868e1727a7dc0a51 on core-updates. > > Now the good news is that we are one patch away from having core-updates > to cross-compile bare-bones.tmpl system! This is the patch attached, > we've been discussing it here[1] without reaching a conclusion. Yay! > From f5d773dc0a627ba6059f1025189d33a9e0d083b5 Mon Sep 17 00:00:00 2001 > From: Mathieu Othacehe <m.othacehe@gmail.com> > Date: Sat, 28 Dec 2019 21:29:06 +0100 > Subject: [PATCH 1/2] gexp: Default to current target. > > * guix/gexp.scm (lower-object): Set target argument to 'current by default and > look for the current target system at bind time if needed, > (gexp->file): ditto, > (gexp->script): ditto. Do we need to do it for ‘lower-object’? I think that one is (almost) always called from a gexp compiler where it’s explicitly passed ‘system’ and ‘target’. Also, a test or two would be welcome as it’s easy to break it without noticing. Otherwise LGTM, thank you, and sorry for the delay! Ludo’. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Cross-compilation broken on canonical packages. 2020-02-24 16:11 ` Ludovic Courtès @ 2020-03-06 9:19 ` Mathieu Othacehe 0 siblings, 0 replies; 14+ messages in thread From: Mathieu Othacehe @ 2020-03-06 9:19 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Hey Ludo, > Do we need to do it for ‘lower-object’? I think that one is (almost) > always called from a gexp compiler where it’s explicitly passed ‘system’ > and ‘target’. Yes because of lower-object call in "system-derivation" procedure of (gnu services). > Also, a test or two would be welcome as it’s easy to break it without > noticing. > > Otherwise LGTM, thank you, and sorry for the delay! Pushed with a few tests on core-updates, thanks for reviewing! Mathieu ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-03-06 9:19 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-14 11:49 Cross-compilation broken on canonical packages Mathieu Othacehe 2019-12-14 11:52 ` Mathieu Othacehe 2019-12-19 16:48 ` Ludovic Courtès 2019-12-22 16:31 ` Mathieu Othacehe 2019-12-30 18:48 ` Ludovic Courtès 2019-12-31 9:55 ` Mathieu Othacehe 2020-01-02 18:07 ` Ludovic Courtès 2020-01-02 22:00 ` Mathieu Othacehe 2020-01-03 12:03 ` Mathieu Othacehe 2020-02-11 13:01 ` Mathieu Othacehe 2020-02-11 14:04 ` Ludovic Courtès 2020-02-12 10:20 ` Mathieu Othacehe 2020-02-24 16:11 ` Ludovic Courtès 2020-03-06 9:19 ` Mathieu Othacehe
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.