Status of "GuixOps"?

Status of "GuixOps"?

[Hartmut Goebel]

Hi,

in Ludo's presentation at GHM he presented "GuixOps" on a slide. What is
the status of this approach? I'm very interested in trying it out and
contributing.

I contributed to DebOps when it was "young". So my point of view is
influenced by how DebOps works. DebOps is a collection of interoperating
role/recipes for Ansible. Debops has become quite complex and I would
like to migrate to GuixSD for new systems.

Q1: I did not follow the development closely, but I seem to recall that
there is some guix sub-command for configuring a remote system. But
grepping the manual for "remote", I did not find it, neither one of the
commands did attract me. How is it called?

Q2: DebOps has some tooling to securely store credentials, certificates,
etc. It uses a gpg-encrypted container which is mounted using FUSE. When
I unlock this container, the appropriate data is transferred to the
target system. How can this be handled with GuixSD? AFAIU with GuixSD
all data in the system-configuration is world-readable in the store. So
how can I automatically transfer e.g. passwords and private keys the the
target system?

Q3: One of DepOps' main features for me is easy use and the automatic
refresh of Let's Encrypt certificates. Basically I just say: "Create
certificates for hostnames A, B, C" and everything happens
automatically: Configuration of nginx, creating the CSR, requesting the
certificate, renewal, etc. What is the status for something like this
for GuixSD?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Status of "GuixOps"?

[Ricardo Wurmus]

Hi Hartmut,

> in Ludo's presentation at GHM he presented "GuixOps" on a slide. What is
> the status of this approach?

GuixOps does not exist yet.

> Q1: I did not follow the development closely, but I seem to recall that
> there is some guix sub-command for configuring a remote system. But
> grepping the manual for "remote", I did not find it, neither one of the
> commands did attract me. How is it called?

It does not exist yet, but it’s one of the first targets.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Status of "GuixOps"?

[Christopher Allan Webber]

Hartmut Goebel writes:

> Hi,
>
> in Ludo's presentation at GHM he presented "GuixOps" on a slide. What is
> the status of this approach? I'm very interested in trying it out and
> contributing.
>
> I contributed to DebOps when it was "young". So my point of view is
> influenced by how DebOps works. DebOps is a collection of interoperating
> role/recipes for Ansible. Debops has become quite complex and I would
> like to migrate to GuixSD for new systems.
>
> Q1: I did not follow the development closely, but I seem to recall that
> there is some guix sub-command for configuring a remote system. But
> grepping the manual for "remote", I did not find it, neither one of the
> commands did attract me. How is it called?

There's a verrrry out of date branch on git origin called wip-deploy.
It needs a lot more work!

> Q2: DebOps has some tooling to securely store credentials, certificates,
> etc. It uses a gpg-encrypted container which is mounted using FUSE. When
> I unlock this container, the appropriate data is transferred to the
> target system. How can this be handled with GuixSD? AFAIU with GuixSD
> all data in the system-configuration is world-readable in the store. So
> how can I automatically transfer e.g. passwords and private keys the the
> target system?

Not sure the right answer for this one :)
But the right system might be user-hackable since Guix is Just Scheme (TM)?
Probably the right route is to remote-copy the files while pushing the
new state of the system over.  Maybe having a loopback device with that
data mounted in it is indeed a good idea, I don't know.

> Q3: One of DepOps' main features for me is easy use and the automatic
> refresh of Let's Encrypt certificates. Basically I just say: "Create
> certificates for hostnames A, B, C" and everything happens
> automatically: Configuration of nginx, creating the CSR, requesting the
> certificate, renewal, etc. What is the status for something like this
> for GuixSD?

There's a wip-lets-encrypt branch on origin too!  In fact I'm using it
on a server!

I'd really like to work on guix-deploy but I won't be able to until next
year.  It sounds like you have experience hacking similar systems; maybe
look at wip-deploy and read David Thompson's old thread about it?  (I'm
too tired to look it up...)

Happy hacking!  I'm off for happy sleeping. :)
 - Chris

Re: Status of "GuixOps"?

[Thompson, David]

On Sun, Sep 17, 2017 at 2:34 PM, Hartmut Goebel
<h.goebel@crazy-compilers.com> wrote:
> Hi,
>
> in Ludo's presentation at GHM he presented "GuixOps" on a slide. What is
> the status of this approach? I'm very interested in trying it out and
> contributing.

I made a wip-deploy branch years ago that allowed you to describe a
cluster of machines and then launch them all simultaneously with qemu.
It was an interesting prototype but nothing has been done since.  I
have since become a full-time DevOps person and have some opinions
about how 'guix deploy' ought to work.  The most important thing is
that there needs to be a generic interface for
creating/updating/destroying machines that can be implemented for bare
metal machines on some LAN, VMs at some VPS provider, VMs at AWS, VMs
on OpenStack, etc.  I think it's also important to support both
in-place updates and "immutable deploys".  A bare metal machine would
need an in-place update, but an Amazon EC2 instance in an auto-scaling
group could be replaced entirely.  As a long term thing, I think that
we need to leave a door open for supporting the all-inclusive
infrastructure creation tools like OpenStack's HEAT templates and
Amazon's CloudFormation.  I use CloudFormation extensively at my day
job and a tool that just managed EC2 instances for me wouldn't cut it.
In summary I think that we need to think carefully about what a
convenient interface is for someone managing a few personal servers
and implement that without making it impossible to scale up to more
"enterprise" use-cases later.

> I contributed to DebOps when it was "young". So my point of view is
> influenced by how DebOps works. DebOps is a collection of interoperating
> role/recipes for Ansible. Debops has become quite complex and I would
> like to migrate to GuixSD for new systems.

Yeah, anything based on the status quo configuration management tools
is bound to be overly complex.  With Guix we can do soooo much better,
we just need some free time, a plan, and a few good hackers. :)

> Q1: I did not follow the development closely, but I seem to recall that
> there is some guix sub-command for configuring a remote system. But
> grepping the manual for "remote", I did not find it, neither one of the
> commands did attract me. How is it called?

I haven't tried it but I think Ludo said that it was going to be a
flag to 'guix system'.  Has that code been merged?  Not being able to
easily connect to a remote daemon is what blocked me when I first
tried writing 'guix deploy'.

> Q2: DebOps has some tooling to securely store credentials, certificates,
> etc. It uses a gpg-encrypted container which is mounted using FUSE. When
> I unlock this container, the appropriate data is transferred to the
> target system. How can this be handled with GuixSD? AFAIU with GuixSD
> all data in the system-configuration is world-readable in the store. So
> how can I automatically transfer e.g. passwords and private keys the the
> target system?

I don't think that is the best approach, or at least it should not be
an approach that GuixSD users *have to* use.  I use Chef at work, and
the equivalent technology there is called "encrypted data bags"
(terrible name, I know).  They are a pain and thus we do not manage
secrets with it.  You're right, everything in the store is
world-readable, therefore you should never store secrets there.  The
most important thing to know is that secrets are stateful, which
contrasts with many things in Guix that are stateless.  Secrets can't
be configured at build-time, it has to be done at runtime.  So how do
we get secrets onto machines?  I think this is a job for GuixSD system
services.  Let's take for example a web application that connects to a
MariaDB database.  When the web application service starts, it will
fetch the database password from the secret store and initialize the
application with it by whatever means is supported (command line flag,
environment variable, etc.).  The secrets store could be a local
GPG-encrypted file (getting the decryption key onto the machine is
non-trivial, of course) or an external service.  To give you an idea
of the possibilities available when it comes to secrets management,
I'll give a more complex example.  All of my experience with this is
with Amazon web services, so if you don't know AWS stuff you can skip
this part but I'll describe it anyway for people that might be
interested: EC2 instances are assigned an IAM role that allows them to
decrypt a set of secrets that have been encrypted using a KMS key.
The encrypted secrets happen to be stored in a DynamoDB table.  A
command line tool for fetching the secrets is installed on the
instances and is integrated into the application startup processes
that need secrets.  When we want to rotate a key, we replace the
DynamoDB record with a new one and restart the relevant services.
Whether the secrets are stored in an encrypted file on each machine or
in some external database, the basic process is the same: applications
will receive their secrets when they are started, not when the system
is configured.

> Q3: One of DepOps' main features for me is easy use and the automatic
> refresh of Let's Encrypt certificates. Basically I just say: "Create
> certificates for hostnames A, B, C" and everything happens
> automatically: Configuration of nginx, creating the CSR, requesting the
> certificate, renewal, etc. What is the status for something like this
> for GuixSD?

We have an nginx service, so you can write a configuration that points
to certs generated by Let's Encrypt.  The cron service could be
configured to periodically run the Let's Encrypt tool that refreshes
the certificates periodically.  The pieces are all there, you just
need to put them together.  This is what I plan to do when I finally
get around to switching my VPS from Debian to GuixSD.  Since this is
all just Scheme code, we could write abstractions that make this
common use-case "just work" for people so they don't have to worry
about the details.  One step at a time. :)

Hope this helps,

- Dave

Re: Status of "GuixOps"?

[Ludovic Courtès]

Hi!

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Sun, Sep 17, 2017 at 2:34 PM, Hartmut Goebel
> <h.goebel@crazy-compilers.com> wrote:

[...]

>> Q1: I did not follow the development closely, but I seem to recall that
>> there is some guix sub-command for configuring a remote system. But
>> grepping the manual for "remote", I did not find it, neither one of the
>> commands did attract me. How is it called?
>
> I haven't tried it but I think Ludo said that it was going to be a
> flag to 'guix system'.  Has that code been merged?  Not being able to
> easily connect to a remote daemon is what blocked me when I first
> tried writing 'guix deploy'.

The idea that I had in mind is to support:

  guix system reconfigure --target=host.example.org config.scm

which would build locally and send the new system (or build it directly
on the target), and then run remotely the effectful bits of
‘reconfigure’: running activation scripts, upgrading Shepherd services.

To get there, the idea I have is to:

  1. Isolate the effectful bits in (guix scripts system) and wrap them
     in gexps.

  2. Provide ‘eval-gexp’ (which would combine ‘build-derivations’ and
     ‘eval’) and ‘eval-gexp-remotely’ (likewise, but it would use
     Guile-SSH’s ‘node-eval’ to evaluate the gexp remotely.)

  3. Change (guix scripts system) to use ‘eval-gexp’ or
     ‘eval-gexp-remotely’ depending on whether --target was found.

This is just a subset of the functionality in you envision for ‘guix
deploy’, David, but it’s a good building block and useful in its own.

> I don't think that is the best approach, or at least it should not be
> an approach that GuixSD users *have to* use.  I use Chef at work, and
> the equivalent technology there is called "encrypted data bags"
> (terrible name, I know).  They are a pain and thus we do not manage
> secrets with it.  You're right, everything in the store is
> world-readable, therefore you should never store secrets there.  The
> most important thing to know is that secrets are stateful, which
> contrasts with many things in Guix that are stateless.  Secrets can't
> be configured at build-time, it has to be done at runtime.  So how do
> we get secrets onto machines?  I think this is a job for GuixSD system
> services.  Let's take for example a web application that connects to a
> MariaDB database.  When the web application service starts, it will
> fetch the database password from the secret store and initialize the
> application with it by whatever means is supported (command line flag,
> environment variable, etc.).  The secrets store could be a local
> GPG-encrypted file (getting the decryption key onto the machine is
> non-trivial, of course) or an external service.  To give you an idea
> of the possibilities available when it comes to secrets management,
> I'll give a more complex example.  All of my experience with this is
> with Amazon web services, so if you don't know AWS stuff you can skip
> this part but I'll describe it anyway for people that might be
> interested: EC2 instances are assigned an IAM role that allows them to
> decrypt a set of secrets that have been encrypted using a KMS key.
> The encrypted secrets happen to be stored in a DynamoDB table.  A
> command line tool for fetching the secrets is installed on the
> instances and is integrated into the application startup processes
> that need secrets.  When we want to rotate a key, we replace the
> DynamoDB record with a new one and restart the relevant services.
> Whether the secrets are stored in an encrypted file on each machine or
> in some external database, the basic process is the same: applications
> will receive their secrets when they are started, not when the system
> is configured.

Sounds interesting.  We don’t have a good story on how to deal with
secrets currently, and this is problematic for deployments.

>> Q3: One of DepOps' main features for me is easy use and the automatic
>> refresh of Let's Encrypt certificates. Basically I just say: "Create
>> certificates for hostnames A, B, C" and everything happens
>> automatically: Configuration of nginx, creating the CSR, requesting the
>> certificate, renewal, etc. What is the status for something like this
>> for GuixSD?
>
> We have an nginx service, so you can write a configuration that points
> to certs generated by Let's Encrypt.  The cron service could be
> configured to periodically run the Let's Encrypt tool that refreshes
> the certificates periodically.  The pieces are all there, you just
> need to put them together.  This is what I plan to do when I finally
> get around to switching my VPS from Debian to GuixSD.  Since this is
> all just Scheme code, we could write abstractions that make this
> common use-case "just work" for people so they don't have to worry
> about the details.  One step at a time. :)

There’s even a patch that brings a Cerbot service!

  https://bugs.gnu.org/26685

People commented on it but the patch still needs love to reach its final
form.  Maybe you can help?  :-)

Thanks,
Ludo’.