Re: cloud-init?

[Simon Josefsson via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 6966 bytes --]

The idea of cloud-init support for Guix came back to me now, and to
better articulate what would be the lithmus test for me would be if I'm
able to run the following command on a Trisquel 11 machine:

printf '%s\n' '#cloud-config' 'fqdn: foo.sjd.se' > my-user-data
virt-install --cloud-init user-data=my-user-data guix:1.4

This should get me a libvirt VM running Guix whose hostname is
'foo.sjd.se'.  I believe this should be achievable and that it would be
a good thing.

Does the above goal make sense?  Thoughts?

Supporting all parameters in user-data, meta-data, network-config etc is
a never ending work, but doable.  It is not like the Debian, Ubuntu,
Fedora, AlmaLinux etc cloud-init images behave the same with the same
user-data/meta-data/network-config inputs, so people are used to
OS-specific issues.  And this is a continous catch-up in those OS images
as well, even though it is a bit simpler since they use upstream
cloud-init that is teached how to work with different OS's.

To make this happen we need to do:

1) Add Guix to libosinfo database, with URLs for cloud-init compatible
Guix images.

2) Host those cloud-init compatible Guix images at a stable URL.

3) Build those cloud-init Guix images in a reproducable fashion and
produce checksums for them and sign them with an OpenPGP key.

4) Write the glue required for the image to self-configure itself.

The challenging part is of course 4), but let's break it down.  I think
there are a couple of different ways to do it:

A) Add upstream 'cloud-init' packages to Guix and teach upstream
'cloud-init' to configure Guix properly, possibly combined with some
Guix-specific package that does the big chunk of work.

B) Add a Guix service like you started working on that would configure
the booted image inline.  This involves modifying files in /etc (I
think?) which is a bit un-Guix'y, but I think it has the chance of being
the simplest and fastest way to solve my use-case.  I'm not sure I
understand your code, is this what you are trying to achieve?

C) Add code (a different Guix service?) to build a new /etc/config.scm
stanza based on the cloud-init Guix image and the user-provided
user-data/meta-data etc, perform 'guix system reconfigure
/etc/config.scm' on that and reboot into the newly built system.

Other ideas?

I would prefer option C) to keep the API separation clean: then we only
have to think about how to convert cloud-init user-data/meta-data/etc
parameters into /etc/config.scm parameters, rebuild the image and reboot
into it.  This has the ability to be easy to debug and to be stable.

Eventually C) may grow into be a part of solution A) once we have proven
to the cloud-init people that there is interest in supporting Guix with
cloud-init.  So then we could package upstream cloud-init and somehow
make that part of the overall solution.  Also, solution B) could grow
into also doing C) or vice-versa, depending on some configuration.

I was about to start work to implement this, but I realize I may be
running out of time (as usual), and there may be a long delay until I
come back to this (as usual), and there is a great risk that I have
forgotten about all these thoughts when I return (as usual), so I'm
writing this to future self and hope that it sparks ideas in other
people's heads meanwhile.  It would be nice to return and see this
already implemented. :-)

/Simon

David Dashyan <mail@davie.li> writes:

> Hi Simon!
>
> I've been interested in adding cloud-init support for a while already.
> It would make Guix much easier to use in a cloud setting.  I did ask
> people weather anyone is interested in it couple of times in #guix and
> also mentioned it on the mailing list couple of times.  People didn't
> seem to express much interest but once they have it they'll like it a
> lot I think :) It is common practice to spawn other distro type and turn
> it into Guix or install Guix on it and do "guix system init" on mounted
> volume and then swap them.  Not to mention the fact that every now and
> then there is a new question on running Guix on some other vendor.  Guix
> deploy was made exactly for that in mind, wasn't it?
>
> I hacked together a little module that provides "cloud-init" service
> that makes a query to DigitalOcean HTTP API hosted on link-local address
> and does some essential stuff like configure the network, add SSH key
> and resize partitions and filesystem.
>
> https://github.com/ipdb/bigchaindb-guix/blob/master/bigchaindb-guix/services/cloud-init.scm
>
> So as you see I went in a different direction - rather than integrating
> actual cloud-init package I just made a Guile substitute for it.  I am
> not sure which approach is best though.
>
> There are many cloud vendors and they all have slightly different APIs.
> On the other side, my guess is that cloud-init (the real one) is
> designed specifically to work with systemd and Debian derived
> distributions.  I am not sure which way is a bigger hassle — to rewrite
> it our-self or try and add Guix support to cloud-init.  I doubt that
> cloud-init team would be interested in Guix support upstreaming either.
>
> I tend to think that adding Guix-style cloud-init service could still be
> easier in the end.  APIs don't differ that much and are pretty stable,
> plus all investigation is already done by cloud-init team — I didn't
> even read DigitalOcean docs when I wrote my module I just looked up what
> cloud-init does.  But I'm interested in your opinion on it.
>
> Wait there is more on Guix cloud-init service.  If you followed the code
> above, what it essentially does is it compiles a giant g-expression that
> does everything.  As a result I borrowed some code from already existing
> Guix services.  If cloud-init service was to do more it would mean even
> more duplication.  But it is not how I wanted in a first place.  It
> ended up like this because as I found out later that 1.) not all module
> are importable into a g-expression so I can't just import other Guix
> services procedures and reuse them 2.) you need to know all service-type
> arguments in advance (which not compatible with having idea of
> cloud-init at all :)).  I think you can't spawn one service from another
> service either, you'll probably need to dig deeper and reach shepherd
> for that.  But I am not sure about that.  Someone mentioned "recursive
> derivations" that are not yet possible when I discussed that topic.  But
> it would be cool if there was a way for Guix cloud-init service to query
> the environment compose and start other Guix services at startup.
> So-called "user-data" can be also used in interesting way too.  There
> could be a "user-data" guile record that extends operating system
> somehow rather that bash-script.
>
> All in all I am glad I am not the only one interested in it.
>
> Cheers,

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]