* bug#41182: Profile hooks ignore system and target
@ 2020-05-10 20:38 Ludovic Courtès
2020-05-11 21:13 ` Ludovic Courtès
0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-10 20:38 UTC (permalink / raw)
To: 41182; +Cc: Mathieu Othacehe
Hello!
As janneke found (and maybe Mathieu also), the profile hooks called by
‘profile-derivation’ for some reason end up using a different system and
target, as can be seen in this example:
--8<---------------cut here---------------start------------->8---
$ guix describe
Generacio 141 May 10 2020 11:41:44 (nuna)
guix 279569c
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 279569ca8251b9ae3f96e4486e4614e340d0fe4f
$ guix pack --target=arm-linux-gnueabihf --no-grafts idutils -d
/gnu/store/rjbs4a5l1vi3winbf71xz3d0hhmdwnjj-tarball-pack.tar.gz.drv
$ guix gc -R /gnu/store/rjbs4a5l1vi3winbf71xz3d0hhmdwnjj-tarball-pack.tar.gz.drv | grep idutils-4.6.drv
/gnu/store/kfv7bwzapb3lfirdpyjh5zcrrpld90ni-idutils-4.6.drv
/gnu/store/1y5rjcvs6giag414wg4ngz7cp4mxy76v-idutils-4.6.drv
$ guix build idutils --no-grafts -d
/gnu/store/kfv7bwzapb3lfirdpyjh5zcrrpld90ni-idutils-4.6.drv
$ guix build idutils --no-grafts -d --target=arm-linux-gnueabihf
/gnu/store/1y5rjcvs6giag414wg4ngz7cp4mxy76v-idutils-4.6.drv
--8<---------------cut here---------------end--------------->8---
We should be seeing only the cross-compile derivation. Where does the
native derivation come from?
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph --path -t derivation /gnu/store/rjbs4a5l1vi3winbf71xz3d0hhmdwnjj-tarball-pack.tar.gz.drv /gnu/store/kfv7bwzapb3lfirdpyjh5zcrrpld90ni-idutils-4.6.drv
/gnu/store/rjbs4a5l1vi3winbf71xz3d0hhmdwnjj-tarball-pack.tar.gz.drv
/gnu/store/hxp1gqbmmq4hjwnb8m1amp15k0ax455m-profile.drv
/gnu/store/3180cdca49sl6xhn9prx9xcwv20jlpdl-fonts-dir.drv
/gnu/store/kfv7bwzapb3lfirdpyjh5zcrrpld90ni-idutils-4.6.drv
--8<---------------cut here---------------end--------------->8---
Actually all the profile hooks refer to the native derivation.
To be continued…
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-10 20:38 bug#41182: Profile hooks ignore system and target Ludovic Courtès
@ 2020-05-11 21:13 ` Ludovic Courtès
2020-05-12 8:35 ` Mathieu Othacehe
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-11 21:13 UTC (permalink / raw)
To: 41182; +Cc: Mathieu Othacehe
[-- Attachment #1: Type: text/plain, Size: 984 bytes --]
Hi,
Ludovic Courtès <ludo@gnu.org> skribis:
> Actually all the profile hooks refer to the native derivation.
I’ve looked at it and this problem is surprisingly tricky to address.
I’ve tried to address it in an API-compatible way, which meant setting
the ‘%current-system’ and ‘%current-target-system’ parameters around the
hook calls, but that is ugly, hard to get right (dynamic binding and
monadic code really don’t go together well :-/), and actually raises
another issue (‘mapm/accumulate-builds’ appears to ignore the initial
dynamic bindings for these two parameters). Hacky patch attached to
illustrate.
So I’m very much tempted to instead require each hook to take ‘system’
and ‘#:target’ arguments and pass them to ‘gexp->derivation’. It’ll
break the API, in case someone out there has custom profile hooks
(unlikely given that it’s not really documented), but I’d say that’s OK.
Thoughts?
Ludo’.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3875 bytes --]
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 59ef5d078b..4f90e9e41d 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3013,9 +3013,10 @@ memoized as a function of '%current-system'."
'/memoized))))
#'(begin
(define memoized
- (mlambda (system) exp))
+ (mlambda (system target) exp))
(define-syntax identifier
- (identifier-syntax (memoized (%current-system))))))))))
+ (identifier-syntax (memoized (%current-system)
+ (%current-target-system))))))))))
(define/system-dependent linux-libre-headers-boot0
;; Note: this is wrapped in a thunk to nicely handle circular dependencies
diff --git a/guix/profiles.scm b/guix/profiles.scm
index 25ff146bdf..58d7e0e450 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -1580,6 +1580,18 @@ This is one of the things to do for the result to be relocatable.
When TARGET is true, it must be a GNU triplet, and the packages in MANIFEST
are cross-built for TARGET."
+ (define (call-with-system+target system target thunk)
+ (mlet* %store-monad ((system0 (set-current-system system))
+ (target0 (set-current-target (pk 'set-target target)))
+ (result (thunk)))
+ (mbegin %store-monad
+ (set-current-system system0)
+ (set-current-target target0)
+ (return result))))
+
+ (define-syntax-rule (with-system+target system target exp)
+ (call-with-system+target system target (lambda () exp)))
+
(mlet* %store-monad ((system (if system
(return system)
(current-system)))
@@ -1592,9 +1604,12 @@ are cross-built for TARGET."
#:target target)))
(extras (if (null? (manifest-entries manifest))
(return '())
- (mapm/accumulate-builds (lambda (hook)
- (hook manifest))
- hooks))))
+ (with-system+target
+ system
+ target
+ (mapm/accumulate-builds (lambda (hook)
+ (hook manifest))
+ hooks)))))
(define inputs
(append (filter-map (lambda (drv)
(and (derivation? drv)
@@ -1689,6 +1704,8 @@ are cross-built for TARGET."
(match profile
(($ <profile> name manifest hooks
locales? allow-collisions? relative-symlinks?)
+ (pk 'prof-c system target (%current-target-system))
+ ;; (display-backtrace (make-stack #t) (current-error-port) #f 80)
(profile-derivation manifest
#:name name
#:hooks hooks
diff --git a/guix/store.scm b/guix/store.scm
index 6c7c07fd2d..92158bd658 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1899,7 +1899,10 @@ coalesce them into a single call."
(values (map/accumulate-builds store
(lambda (obj)
(run-with-store store
- (mproc obj)))
+ (mproc obj)
+ ;; #:system (%current-system)
+ ;; #:target (%current-target-system)
+ ))
lst)
store)))
^ permalink raw reply related [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-11 21:13 ` Ludovic Courtès
@ 2020-05-12 8:35 ` Mathieu Othacehe
2020-05-13 9:45 ` Jan Nieuwenhuizen
2020-05-14 12:24 ` Ludovic Courtès
2 siblings, 0 replies; 7+ messages in thread
From: Mathieu Othacehe @ 2020-05-12 8:35 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 41182
Hey Ludo,
> So I’m very much tempted to instead require each hook to take ‘system’
> and ‘#:target’ arguments and pass them to ‘gexp->derivation’. It’ll
> break the API, in case someone out there has custom profile hooks
> (unlikely given that it’s not really documented), but I’d say that’s OK.
>
> Thoughts?
What seems strange to me is that gexp->derivation has target set to
'current by default, so it should use the defined target. Now, that I
look at it, it's using "%current-target-system".
Would it make any difference to switch:
--8<---------------cut here---------------start------------->8---
(target -> (if (eq? target 'current)
(%current-target-system)
target))
--8<---------------cut here---------------end--------------->8---
to
--8<---------------cut here---------------start------------->8---
(target (if (eq? target 'current)
(current-target-system)
(return target)))
--8<---------------cut here---------------end--------------->8---
like for lower-object, gexp->file and gexp->script?
Regarding breaking the profile hooks API, it's fine by me.
Thanks for investigating this,
Mathieu
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-11 21:13 ` Ludovic Courtès
2020-05-12 8:35 ` Mathieu Othacehe
@ 2020-05-13 9:45 ` Jan Nieuwenhuizen
2020-05-13 10:37 ` Mathieu Othacehe
2020-05-14 12:24 ` Ludovic Courtès
2 siblings, 1 reply; 7+ messages in thread
From: Jan Nieuwenhuizen @ 2020-05-13 9:45 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 41182, Mathieu Othacehe
Ludovic Courtès writes:
Hello!
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Actually all the profile hooks refer to the native derivation.
>
> I’ve looked at it and this problem is surprisingly tricky to address.
Thank you so much for looking into this!
> I’ve tried to address it in an API-compatible way, which meant setting
> the ‘%current-system’ and ‘%current-target-system’ parameters around the
> hook calls, but that is ugly, hard to get right (dynamic binding and
> monadic code really don’t go together well :-/), and actually raises
> another issue (‘mapm/accumulate-builds’ appears to ignore the initial
> dynamic bindings for these two parameters). Hacky patch attached to
> illustrate.
>
> So I’m very much tempted to instead require each hook to take ‘system’
> and ‘#:target’ arguments and pass them to ‘gexp->derivation’. It’ll
> break the API, in case someone out there has custom profile hooks
> (unlikely given that it’s not really documented), but I’d say that’s OK.
>
> Thoughts?
I'm all for breaking the API if that gets us further. How about doing
that on the wip-hurd-vm branch, complete building the vm-image with
services and when we have found this (and possible next steps) are
useful fixes, commit to core-updates?
It would be useful to get substitutes for such a change, though --
testing this took all night.
Other than that I can offer my observations trying to build a kind of
minimal Hurd system using
fb120a69a8 services: hurd: Use activation-service, hurd-etc-service.
both
./pre-inst-env guix system build --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl
./pre-inst-env guix system vm-image --target=i586-pc-gnu gnu/system/examples/bare-hurd.tmpl
still want to build a native Hurd (in my case for x86_64).
So, I tried applying what your patch seems to be suggesting:
--8<---------------cut here---------------start------------->8---
diff --git a/guix/store.scm b/guix/store.scm
index 92158bd658..b27ad0fab3 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1900,8 +1900,8 @@ coalesce them into a single call."
(lambda (obj)
(run-with-store store
(mproc obj)
- ;; #:system (%current-system)
- ;; #:target (%current-target-system)
+ #:system (%current-system)
+ #:target (%current-target-system)
))
lst)
store)))
--8<---------------cut here---------------end--------------->8---
and with that "system build" succeeds (after a while) but "system vm-image" says
guix system: error: gnu/packages/glib.scm:404:2: gobject-introspection@1.62.0: build system `meson' does not support cross builds
This could be mostly harmless...still building a full (non-tiny/minimal)
qemu or grub maybe?
Greetings
janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply related [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-13 9:45 ` Jan Nieuwenhuizen
@ 2020-05-13 10:37 ` Mathieu Othacehe
0 siblings, 0 replies; 7+ messages in thread
From: Mathieu Othacehe @ 2020-05-13 10:37 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 41182
Hello Jan,
> and with that "system build" succeeds (after a while) but "system vm-image" says
>
> guix system: error: gnu/packages/glib.scm:404:2: gobject-introspection@1.62.0: build system `meson' does not support cross builds
>
> This could be mostly harmless...still building a full (non-tiny/minimal)
> qemu or grub maybe?
I remember fixing a very similar issue with
d4ddf22d54f9374715c651aaebda2315e9f89272. The issue was cross-building
QEMU does not work, because it's dragging packages built with
meson-build-system.
Turns out we want to use a native build of QEMU here, so I replaced
#$qemu by #+qemu. Now there must be a similar issue somewhere. I'm
building your branch to see if I can figure it out.
Thanks for sharing your progress!
Mathieu
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-11 21:13 ` Ludovic Courtès
2020-05-12 8:35 ` Mathieu Othacehe
2020-05-13 9:45 ` Jan Nieuwenhuizen
@ 2020-05-14 12:24 ` Ludovic Courtès
2020-05-14 15:23 ` Ludovic Courtès
2 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-14 12:24 UTC (permalink / raw)
To: 41182; +Cc: Mathieu Othacehe
Hi,
Ludovic Courtès <ludo@gnu.org> skribis:
> I’ve tried to address it in an API-compatible way, which meant setting
> the ‘%current-system’ and ‘%current-target-system’ parameters around the
> hook calls, but that is ugly, hard to get right (dynamic binding and
> monadic code really don’t go together well :-/), and actually raises
> another issue (‘mapm/accumulate-builds’ appears to ignore the initial
> dynamic bindings for these two parameters). Hacky patch attached to
> illustrate.
I was able to boil this second sub-problem down to a simple case:
--8<---------------cut here---------------start------------->8---
$ cat /tmp/t.scm
(use-modules (guix)
(guix grafts)
(gnu packages idutils))
(define target
(getenv "REAL_TARGET"))
(%graft? #f)
(with-store s
(parameterize ((%current-target-system (getenv "TARGET")))
(pk (if target
(package-cross-derivation s idutils target)
(package-derivation s idutils)))))
$ REAL_TARGET=arm-linux-gnueabihf ./pre-inst-env guile /tmp/t.scm
;;; (#<derivation /gnu/store/1y5rjcvs6giag414wg4ngz7cp4mxy76v-idutils-4.6.drv => /gnu/store/6kq4ick0jljrfjnhw0v2yghr8nalhrqi-idutils-4.6 7f0867c0de10>)
$ TARGET=arm-linux-gnueabihf REAL_TARGET=arm-linux-gnueabihf ./pre-inst-env guile /tmp/t.scm
;;; (#<derivation /gnu/store/4k4nqr1rpm07ypq9inhvsghrqma5yacy-idutils-4.6.drv => /gnu/store/v4rgm5yhyx5ir3622hhxcaz3a10flcyr-idutils-4.6 7f7a4b1010a0>)
--8<---------------cut here---------------end--------------->8---
IOW, the initial value of ‘%current-target-system’ leads us to pick
ld-wrapper -> guile -> bdw-gc -> libatomic-ops in the second case, which
is wrong and due to this conditional in libgc’s inputs:
(propagated-inputs
(if (%current-target-system)
;; The build system refuses to check for compiler intrinsics when
;; cross-compiling, and demands using libatomic-ops instead.
`(("libatomic-ops" ,libatomic-ops))
'()))
As it turns out, ‘guix pack’ and ‘guix system’ are the only programs
that set ‘%current-target-system’ at the top level, via
‘run-with-store’.
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#41182: Profile hooks ignore system and target
2020-05-14 12:24 ` Ludovic Courtès
@ 2020-05-14 15:23 ` Ludovic Courtès
0 siblings, 0 replies; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-14 15:23 UTC (permalink / raw)
To: 41182-done; +Cc: Mathieu Othacehe
Alright, fixed!
80963744a2 store: 'mapm/accumulate-builds' preserves '%current-target-system'.
f52fbf7094 packages: Ensure bags are insensitive to '%current-target-system'.
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-05-14 15:43 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-10 20:38 bug#41182: Profile hooks ignore system and target Ludovic Courtès
2020-05-11 21:13 ` Ludovic Courtès
2020-05-12 8:35 ` Mathieu Othacehe
2020-05-13 9:45 ` Jan Nieuwenhuizen
2020-05-13 10:37 ` Mathieu Othacehe
2020-05-14 12:24 ` Ludovic Courtès
2020-05-14 15:23 ` Ludovic Courtès
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.