Intermediate abstraction of system service configuration

Intermediate abstraction of system service configuration

[Dale Mellor]

Hello,

   I am in the process of moving my production machine
to a pure Guix setup, and feeling some pain which I
feel is somewhat unnecessary.

   There are two extremes supported by the Guix system
in configuring services: either a native application
configuration file is given and used verbatim, without
Guix really understanding what is happening, or else a
huge pile of scheme code can be constructed which
resembles the configuration file structurally and is
eventually translated into an actual configuration
file.  The benefit of the former approach is that all
of the configuration options of the application are
available and that the package importer has no work to
do to support the configuration (in writing
translators), but the disadvantage is that this cannot
be introspected or (easily) manipulated by the Guix
system (for example if another application needs to
make modifications so that they can work harmoniously
together).  The advantage of the latter approach is
that the configuration can be more dynamically
constructed, but the disadvantage is that configuration
is restricted to those aspects which the package
importers have gone to (a lot of trouble) to implement,
and for which they have to diligently track changes in
application configuration specification across versions
of the application.

   The two packages which immediately come to mind
which show this extreme dichotomy well are /nginx/ and
/dovecot/; the manual pages for those packages show
reams of configuration clauses available in the Guix
system configuration file (neither of which are
exhaustive, ouch!), and they both also allow a single
native configuration file to be used verbatim.

   There is no middle ground.

   I think that Guix taking the view of a configuration
file as a nested set of string-named blocks containing
lists of string-named, string-valued pairs would be an
intermediate level of abstraction more suited to the
Guix system configuration view of the world.  It has
the advantage that it is introspectible and
programmatically modifiable, yet is flexible enough
that the full configurability of services is available,
and I /think/ it is sufficiently general that it would
cover all cases of application configurations.  It
means that application configuration can be handled
more uniformly across applications by the Guix system,
and importers of applications have a much easier time
coding up the configuration file translator, which
would be relatively trivial in most cases.

   I also believe it would make life easier for people
configuring systems for themselves, especially where
they wish to deviate from 'standard' setups, which is
where I met with most pain.

   So shoot me down  :)

Dale

Re: Intermediate abstraction of system service configuration

[Carlo Zancanaro]

Hi Dale!

On Tue, Feb 06 2024, Dale Mellor wrote:
>    There is no middle ground.

The nginx{,-upstream,-server}-configuration records have fields for
{extra,raw}-content, which do allow arbitrary configuration lines to be
added to their config. I think this is a sensible escape hatch to have
to provide this middle ground.

Andrew Tropin has done something interesting on the idea of more
free-form configuration for nginx specifically[1], but I don't think
that idea has found its way out of rde and into Guix yet.

>   I think that Guix taking the view of a configuration
> file as a nested set of string-named blocks containing
> lists of string-named, string-valued pairs would be an
> intermediate level of abstraction more suited to the
> Guix system configuration view of the world.

This sounds quite similar to the way that Nix represents their internal
configuration. Their model of configuring services is to have "Attribute
Sets" (i.e. key/value mappings) which specify the configuration of
services. Nix doesn't have service extension in the same way as Guix
does, however, so interactions between different services look very
different. We'd have to think carefully about how the Guix model of
service extension would work with such an intermediate representation.

Carlo

[1]: https://fosstodon.org/@abcdw/110180191828598277

Re: Intermediate abstraction of system service configuration

[Dale Mellor]

Hello Carlo,

On Tue, 2024-02-06 at 23:10 +1100, Carlo Zancanaro wrote:
> The nginx{,-upstream,-server}-configuration records have fields for
> {extra,raw}-content, which do allow arbitrary configuration lines to
> be
> added to their config. I think this is a sensible escape hatch to
> have
> to provide this middle ground.

   The problem is that some configurations, especially nginx, are
sensitive to the order in which items appear and may require slightly
intricate nesting, and you can't always achieve that with intermixed
raw configuration sections.  And those sections are not (easily)
introspectable.

> Andrew Tropin has done something interesting on the idea of more
> free-form configuration for nginx specifically[1], but I don't think
> that idea has found its way out of rde and into Guix yet.
> 
> >   I think that Guix taking the view of a configuration
> > file as a nested set of string-named blocks containing
> > lists of string-named, string-valued pairs would be an
> > intermediate level of abstraction more suited to the
> > Guix system configuration view of the world.
> 
> We'd have to think carefully about how the Guix model of
> service extension would work with such an intermediate
> representation.

   Agree that it would have repercussions throughout the package
ecosystem.  I would have thought that it would make things easier.  You
could argue that you don't benefit from the Guile syntax checker, but
at the end of the day you don't find out you've made a mistake until
you try to run `guix system reconfigure` anyway.

Re: Intermediate abstraction of system service configuration

[Carlo Zancanaro]

On Tue, Feb 06 2024, Dale Mellor wrote:
>    Agree that it would have repercussions throughout the package
> ecosystem.  I would have thought that it would make things easier.  You
> could argue that you don't benefit from the Guile syntax checker, but
> at the end of the day you don't find out you've made a mistake until
> you try to run `guix system reconfigure` anyway.

It might make things "easier" in one sense, but loosening the
constraints on one thing often makes some other part harder. In
particular, because of service extensions you can't rely on the whole
configuration being written in the same place, so we would need some
sort of merging logic for free-form configurations.

This merging logic would have to be service specific, but would
presumably need to perform its own shape checking (for good error
messages, if nothing else). Presumably this would require some sort of
type definitions, similar to what our current configuration system
requires (maybe with less boilerplate).

I'm sure it's possible to write a system that uses these things, and
there has been some idea of doing this in the past for nginx
specifically[1], but I'm not yet convinced it would actually be easier
across the board. Having used Nix (which has more free-form
configuration), I'm not sure that it's better than what Guix has.

I'm happy to be convinced, though, if you'd like to put together an
implementation. Maybe you could start with a single service (nginx) and
see how it comes together. I'm not a committer, so convincing me doesn't
get things into Guix, but presumably if you can convince me you can
convince a committer as well.

Carlo

[1]: https://issues.guix.gnu.org/37388