* [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.
@ 2020-09-01 14:46 Jan Nieuwenhuizen
2020-09-01 21:19 ` Ludovic Courtès
0 siblings, 1 reply; 5+ messages in thread
From: Jan Nieuwenhuizen @ 2020-09-01 14:46 UTC (permalink / raw)
To: 43155
[-- Attachment #1: Type: text/plain, Size: 832 bytes --]
Hi!
With bug https://bugs.gnu.org/43106 just closed we now have a nice way
to inject secrets into the Childhurds.
Using the attached patch, which needs a fresh pull and reconfigure on
berlin (at least the nodes 101,102 that run Childhurds), we can create a
tree of childhurd secrets like so
--8<---------------cut here---------------start------------->8---
/etc/childhurd/etc/guix/signing-key.pub
/etc/childhurd/etc/guix/signing-key.sec
/etc/childhurd/etc/ssh/ssh_host_ed25519_key
/etc/childhurd/etc/ssh/ssh_host_ecdsa_key
/etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
/etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
--8<---------------cut here---------------end--------------->8---
...and then we should be able to start offloading builds for the Hurd.
(I guess we then also need to add a cuirass jobs for the Hurd?)
Janneke
[-- Attachment #2: 0001-hydra-build-machines-Update-childhurd-net-options-fo.patch --]
[-- Type: text/x-patch, Size: 2793 bytes --]
From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Tue, 1 Sep 2020 16:31:42 +0200
Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
secret-service.
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8
* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
[childhurd-net-options]: Include secret-service local QEMU forwarding.
Use variables from (gnu services virtualization).
---
hydra/modules/sysadmin/build-machines.scm | 31 ++++++++++++++++-------
1 file changed, 22 insertions(+), 9 deletions(-)
diff --git a/hydra/modules/sysadmin/build-machines.scm b/hydra/modules/sysadmin/build-machines.scm
index b4afcbe..0385b6a 100644
--- a/hydra/modules/sysadmin/build-machines.scm
+++ b/hydra/modules/sysadmin/build-machines.scm
@@ -120,15 +120,28 @@ EMULATED-ARCHITECTURES, unless it's empty."
(define (childhurd-net-options . config)
"Expose SSH and VNC ports on 0.0.0.0; for first Childhurd VM those
-are 10022 and 15900."
- (let ((id 0))
- (define (qemu-vm-port base)
- (number->string (+ base (* 1000 id))))
- `("--device" "rtl8139,netdev=net0"
- "--netdev" ,(string-append
- "user,id=net0"
- ",hostfwd=tcp:0.0.0.0:" (qemu-vm-port 10022) "-:2222"
- ",hostfwd=tcp:0.0.0.0:" (qemu-vm-port 15900) "-:5900"))))
+are 10022 and 15900. Keep secret-service port local."
+ `("--device" "rtl8139,netdev=net0"
+ "--netdev" ,(string-append
+ "user,id=net0"
+ ",hostfwd=tcp:127.0.0.1:"
+ (number->string (hurd-vm-port
+ config
+ (@@ (gnu services virtualization)
+ %hurd-vm-secrets-port)))
+ "-:1004"
+ ",hostfwd=tcp:0.0.0.0:"
+ (number->string (hurd-vm-port
+ config
+ (@@ (gnu services virtualization)
+ %hurd-vm-ssh-port)))
+ "-:2222"
+ ",hostfwd=tcp:0.0.0.0:"
+ (number->string (hurd-vm-port
+ config
+ (@@ (gnu services virtualization)
+ %hurd-vm-vnc-port)))
+ "-:5900")))
(define sysadmins
(list (sysadmin (name "ludo")
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
[-- Attachment #3: Type: text/plain, Size: 152 bytes --]
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply related [flat|nested] 5+ messages in thread* [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.
2020-09-01 14:46 [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service Jan Nieuwenhuizen
@ 2020-09-01 21:19 ` Ludovic Courtès
2020-09-02 5:58 ` Jan Nieuwenhuizen
0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2020-09-01 21:19 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 43155
Hi!
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
> to inject secrets into the Childhurds.
>
> Using the attached patch, which needs a fresh pull and reconfigure on
> berlin (at least the nodes 101,102 that run Childhurds), we can create a
> tree of childhurd secrets like so
>
> /etc/childhurd/etc/guix/signing-key.pub
> /etc/childhurd/etc/guix/signing-key.sec
> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>
> ...and then we should be able to start offloading builds for the Hurd.
Yup! Probably we’ll create /etc/childhurd/HOST for each VM, so we also
need to adjust <hurd-vm-configuration> accordingly, right?
(I realize that the current code will silently keep going if we forget
to put the secret files in place; IOW, the service config doesn’t show
the files we intended to push as secrets. Oh well, we’ll see that
later.)
> (I guess we then also need to add a cuirass jobs for the Hurd?)
Yes, or maybe just change ‘systems’ in the Cuirass specs for
‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
which doesn’t sound like a great idea for now. Perhaps we can simply
add a separate jobset pulling from ‘master’ but building only for
i586-gnu and only the “core” package set?
>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
> Date: Tue, 1 Sep 2020 16:31:42 +0200
> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
> secret-service.
> Content-Transfer-Encoding: 8bit
> Content-Type: text/plain; charset=UTF-8
>
> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
> [childhurd-net-options]: Include secret-service local QEMU forwarding.
> Use variables from (gnu services virtualization).
LGTM, thanks!
Ludo’.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.
2020-09-01 21:19 ` Ludovic Courtès
@ 2020-09-02 5:58 ` Jan Nieuwenhuizen
2020-09-02 20:08 ` Ludovic Courtès
0 siblings, 1 reply; 5+ messages in thread
From: Jan Nieuwenhuizen @ 2020-09-02 5:58 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 43155
[-- Attachment #1: Type: text/plain, Size: 3048 bytes --]
Ludovic Courtès writes:
Hi!
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
>> to inject secrets into the Childhurds.
>>
>> Using the attached patch, which needs a fresh pull and reconfigure on
>> berlin (at least the nodes 101,102 that run Childhurds), we can create a
>> tree of childhurd secrets like so
>>
>> /etc/childhurd/etc/guix/signing-key.pub
>> /etc/childhurd/etc/guix/signing-key.sec
>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>>
>> ...and then we should be able to start offloading builds for the Hurd.
>
> Yup! Probably we’ll create /etc/childhurd/HOST for each VM, so we also
> need to adjust <hurd-vm-configuration> accordingly, right?
Yes, we can add something like
(secret-root (format #f "/etc/childhurd/~a" id))
to the
(service hurd-vm-service-type
(hurd-vm-configuration
...
(i'm a bit curious, though, why we would want to differentiate between
childhurds, they can be all identical?)
> (I realize that the current code will silently keep going if we forget
> to put the secret files in place; IOW, the service config doesn’t show
> the files we intended to push as secrets. Oh well, we’ll see that
> later.)
Yes, I guess that's a feature -- "you" can start it once, then do
something like
mkdir -p /etc/childhurd/etc
scp -r childhurd:/etc/guix /etc/childhurd/etc
scp -r childhurd:/etc/ssh /etc/childhurd/etc
>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>
> Yes, or maybe just change ‘systems’ in the Cuirass specs for
> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
> which doesn’t sound like a great idea for now.
I agree, not much sense in that yet.
> Perhaps we can simply add a separate jobset pulling from ‘master’ but
> building only for i586-gnu and only the “core” package set?
Hmm, why can't I find the definition of "core"?. Anyway, It would be a
great first step to build (everything needef for) "hello", after that we
want to have/try "guile-3.0" and possibly "guix".
>>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
>> Date: Tue, 1 Sep 2020 16:31:42 +0200
>> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
>> secret-service.
>> Content-Transfer-Encoding: 8bit
>> Content-Type: text/plain; charset=UTF-8
>>
>> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
>> [childhurd-net-options]: Include secret-service local QEMU forwarding.
>> Use variables from (gnu services virtualization).
>
> LGTM, thanks!
Great, pushed to guix-maintenance as 04c0fc1ea110b82d6180bbc1b2f895e55e746cd8
Janneke
...after first pushing this -- Ooopss typo fix
[-- Attachment #2: 0001-hydra-build-machines-Oops-typo-in-childhurd-net-opti.patch --]
[-- Type: text/x-patch, Size: 1371 bytes --]
From 35dd1de08f1b812a22184e925b089ffc471c52de Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Wed, 2 Sep 2020 07:52:13 +0200
Subject: [PATCH 1/2] hydra/build-machines: Oops, typo in
childhurd-net-options.
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8
* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
[childhurd-net-options]: Remove stray dot from parameter list.
---
hydra/modules/sysadmin/build-machines.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hydra/modules/sysadmin/build-machines.scm b/hydra/modules/sysadmin/build-machines.scm
index b4afcbe..0a3e113 100644
--- a/hydra/modules/sysadmin/build-machines.scm
+++ b/hydra/modules/sysadmin/build-machines.scm
@@ -118,7 +118,7 @@ EMULATED-ARCHITECTURES, unless it's empty."
(mcron-configuration (jobs (list gc-job))))
(operating-system-user-services %hurd-vm-operating-system)))))
- (define (childhurd-net-options . config)
+ (define (childhurd-net-options config)
"Expose SSH and VNC ports on 0.0.0.0; for first Childhurd VM those
are 10022 and 15900."
(let ((id 0))
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
[-- Attachment #3: Type: text/plain, Size: 152 bytes --]
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply related [flat|nested] 5+ messages in thread* [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.
2020-09-02 5:58 ` Jan Nieuwenhuizen
@ 2020-09-02 20:08 ` Ludovic Courtès
2020-09-03 10:19 ` Jan Nieuwenhuizen
0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2020-09-02 20:08 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 43155
Hi,
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> Ludovic Courtès writes:
>
> Hi!
>
>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>>> With bug https://bugs.gnu.org/43106 just closed we now have a nice way
>>> to inject secrets into the Childhurds.
>>>
>>> Using the attached patch, which needs a fresh pull and reconfigure on
>>> berlin (at least the nodes 101,102 that run Childhurds), we can create a
>>> tree of childhurd secrets like so
>>>
>>> /etc/childhurd/etc/guix/signing-key.pub
>>> /etc/childhurd/etc/guix/signing-key.sec
>>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key
>>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key
>>> /etc/childhurd/etc/ssh/ssh_host_ed25519_key.pub
>>> /etc/childhurd/etc/ssh/ssh_host_ecdsa_key.pub
>>>
>>> ...and then we should be able to start offloading builds for the Hurd.
>>
>> Yup! Probably we’ll create /etc/childhurd/HOST for each VM, so we also
>> need to adjust <hurd-vm-configuration> accordingly, right?
>
> Yes, we can add something like
>
> (secret-root (format #f "/etc/childhurd/~a" id))
>
> to the
>
> (service hurd-vm-service-type
> (hurd-vm-configuration
> ...
Sounds good.
> (i'm a bit curious, though, why we would want to differentiate between
> childhurds, they can be all identical?)
Well, dunno if it really matters for our specific use case, but it seems
“cleaner” to me to give each childhurd its identity. OTOH, these are
VMs and they run on the same physical machine, so…
>> (I realize that the current code will silently keep going if we forget
>> to put the secret files in place; IOW, the service config doesn’t show
>> the files we intended to push as secrets. Oh well, we’ll see that
>> later.)
>
> Yes, I guess that's a feature -- "you" can start it once, then do
> something like
>
> mkdir -p /etc/childhurd/etc
> scp -r childhurd:/etc/guix /etc/childhurd/etc
> scp -r childhurd:/etc/ssh /etc/childhurd/etc
Right, that can be convenient. OTOH, from the perspective of having
declarative OS configs, it’s not great because this aspect of the config
are left out. But maybe that’s an issue we can have if/when we
generalize ‘secret-service-type’.
>>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>>
>> Yes, or maybe just change ‘systems’ in the Cuirass specs for
>> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
>> which doesn’t sound like a great idea for now.
>
> I agree, not much sense in that yet.
>
>> Perhaps we can simply add a separate jobset pulling from ‘master’ but
>> building only for i586-gnu and only the “core” package set?
>
> Hmm, why can't I find the definition of "core"?. Anyway, It would be a
> great first step to build (everything needef for) "hello", after that we
> want to have/try "guile-3.0" and possibly "guix".
Sure. The “core” subset is defined in (gnu ci).
>>>>From 6d1c388ed82c260af27b556c0677e780ee410b05 Mon Sep 17 00:00:00 2001
>>> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
>>> Date: Tue, 1 Sep 2020 16:31:42 +0200
>>> Subject: [PATCH] hydra//build-machines: Update childhurd-net-options for
>>> secret-service.
>>> Content-Transfer-Encoding: 8bit
>>> Content-Type: text/plain; charset=UTF-8
>>>
>>> * hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os)
>>> [childhurd-net-options]: Include secret-service local QEMU forwarding.
>>> Use variables from (gnu services virtualization).
>>
>> LGTM, thanks!
>
> Great, pushed to guix-maintenance as 04c0fc1ea110b82d6180bbc1b2f895e55e746cd8
Thanks!
Ludo’.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service.
2020-09-02 20:08 ` Ludovic Courtès
@ 2020-09-03 10:19 ` Jan Nieuwenhuizen
0 siblings, 0 replies; 5+ messages in thread
From: Jan Nieuwenhuizen @ 2020-09-03 10:19 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 43155
Ludovic Courtès writes:
Hi,
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>> Yes, we can add something like
>>
>> (secret-root (format #f "/etc/childhurd/~a" id))
>>
>> to the
>>
>> (service hurd-vm-service-type
>> (hurd-vm-configuration
>> ...
>
> Sounds good.
>
>> (i'm a bit curious, though, why we would want to differentiate between
>> childhurds, they can be all identical?)
>
> Well, dunno if it really matters for our specific use case, but it seems
> “cleaner” to me to give each childhurd its identity. OTOH, these are
> VMs and they run on the same physical machine, so…
Right...
>>> (I realize that the current code will silently keep going if we forget
>>> to put the secret files in place; IOW, the service config doesn’t show
>>> the files we intended to push as secrets. Oh well, we’ll see that
>>> later.)
>>
>> Yes, I guess that's a feature -- "you" can start it once, then do
>> something like
>>
>> mkdir -p /etc/childhurd/etc
>> scp -r childhurd:/etc/guix /etc/childhurd/etc
>> scp -r childhurd:/etc/ssh /etc/childhurd/etc
>
> Right, that can be convenient. OTOH, from the perspective of having
> declarative OS configs, it’s not great because this aspect of the config
> are left out. But maybe that’s an issue we can have if/when we
> generalize ‘secret-service-type’.
Ah, I see -- it could lead to "silent" failure/differences if
/etc/childhurd somehow disappears -- isn't re-created upon new install.
It makes sense to at least be less than silent, "fail early" is always
good.
>>>> (I guess we then also need to add a cuirass jobs for the Hurd?)
>>>
>>> Yes, or maybe just change ‘systems’ in the Cuirass specs for
>>> ‘guix-master’, but then it’ll try to build everything for GNU/Hurd,
>>> which doesn’t sound like a great idea for now.
>>
>> I agree, not much sense in that yet.
>>
>>> Perhaps we can simply add a separate jobset pulling from ‘master’ but
>>> building only for i586-gnu and only the “core” package set?
>>
>> Hmm, why can't I find the definition of "core"?. Anyway, It would be a
>> great first step to build (everything needef for) "hello", after that we
>> want to have/try "guile-3.0" and possibly "guix".
>
> Sure. The “core” subset is defined in (gnu ci).
As discussed on IRC that could get an update. Would you like to do
that, seems like an easy edit but I'm a bit unsure about the choices and
consequences there?
I think once the offloading works we'll want to try building guix; and
it could be nice if as many dependencies that "just happen to build" are
actually available. It's waay to early to try to build everything but
we may want something in between. Or add "guix" to core-packages,
maybe? Just wondering out loud here...
Janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-09-03 10:21 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-01 14:46 [bug#43155] [PATCH] hydra//build-machines: Update childhurd-net-options for secret-service Jan Nieuwenhuizen
2020-09-01 21:19 ` Ludovic Courtès
2020-09-02 5:58 ` Jan Nieuwenhuizen
2020-09-02 20:08 ` Ludovic Courtès
2020-09-03 10:19 ` Jan Nieuwenhuizen
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).