Are declarative app configs worth it?

Are declarative app configs worth it?

[Sergey Trofimov]

Hi guix,

I want to start a discussion around how to manage user app config 
files.
Copying my message from https://issues.guix.gnu.org/68010, where 
home-zathura-configuration with 76 fields is proposed.

I have mixed feelings about pulling 3rd-party software 
configurations in guix:
- adding it to guix increases maintenance burden: new versions 
  could add or remove config options
- it requires documentation/translation, another hidden cost
- it bloats guix: imagine if we add configs for every 
  user-configurable app
- such configs are not easily transferrable: if I were to use the 
  same app in non-guix env, I'd have to maintain 2 configs

Another recent example is `oci-container-configuration` which 
defines a subset of docker-cli startup arguments. The problem is 
that `docker run` command has 96 options and the configuration 
only uses a handful, lacking a way to provide the remaining ones.

I think guix should not embed config generators for user software. 
The only need I see for such generators is when there are options 
which should be the same among multiple applications (e.g. color 
schemes or shared directories). For such usecase guix should 
provide better text manipulation tools which home owners could use 
to parameterise configs.

Best regards,
Sergey Trofimov

Re: Are declarative app configs worth it?

[Ricardo Wurmus]

Sergey Trofimov <sarg@sarg.org.ru> writes:

> - adding it to guix increases maintenance burden: new versions   could
>  add or remove config options

This is why there should be automated tests.  There are too few of them.

> - it requires documentation/translation, another hidden cost

We should only accept configuration procedures that have proper
documentation, yes.

> - it bloats guix: imagine if we add configs for every
>   user-configurable app

That would be nice.

If we started to accept the term bloat we could easily apply it to
anything in Guix: all that R stuff?  Bloat!  All that bioinfo stuff?
Bloat!

> - such configs are not easily transferrable: if I were to use the
>   same app in non-guix env, I'd have to maintain 2 configs

We are generating configuration files from our config languages.  So you
would only need to generate them and copy them for your non-guix
environment.

> Another recent example is `oci-container-configuration` which defines
> a subset of docker-cli startup arguments. The problem is that `docker
> run` command has 96 options and the configuration only uses a handful,
> lacking a way to provide the remaining ones.

All config bindings need to have an escape hatch.

-- 
Ricardo

Re: Are declarative app configs worth it?

[John Soo]

[-- Attachment #1: Type: text/plain, Size: 209 bytes --]

Hi there,

I think specifying each option is too much to maintain - however I what about an alist or hashmap?

Nixos has used the freeform module type to great success and this feels like the same situation.


[-- Attachment #2: Type: text/html, Size: 510 bytes --]

Re: Are declarative app configs worth it?

[Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1749 bytes --]

On 2023-12-26 at 15:56+01:00, Ricardo Wurmus wrote:
> On 2023-12-26 at 14:53+01:00, Sergey Trofimov wrote:
> > - adding it to guix increases maintenance burden: new versions
> >   could add or remove config options
>
> This is why there should be automated tests.
> There are too few of them.

This is easier said than done.  A program could require a display
and audio server so it's not trivial to run as CI, and even for
contributors and maintainers, some configurations could only
be tested with the program's input file, and even then
some tests would need to measure imprecise things like
text colors which is aliased.

On 2023-12-26 at 14:53+01:00, Sergey Trofimov wrote:
> I think guix should not embed config generators for user software. 
> The only need I see for such generators is when there are options 
> which should be the same among multiple applications (e.g. color 
> schemes or shared directories). For such usecase guix should 
> provide better text manipulation tools which home owners could use 
> to parameterise configs.

One other case is when it involves other packages,
like native messager for IceCat:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_messaging

I would also want to share my experience as a user, that having
to run guix home reconfigure iteratively is not exactly pleasant
due to the high delay.  Home files service suffers the same problem,
and I'm suggesting to allow an option to install the symlink
(e.g. /gnu/store/...-foo -> $HOME/.../foo) to make this convenient.
The user is likely to version control the parent directory anyway,
and the home files config does not include any hash to justify
copying the file to the guix store.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

Re: Are declarative app configs worth it?

[Ricardo Wurmus]

Nguyễn Gia Phong <cnx@loang.net> writes:

> [[PGP Signed Part:Undecided]]
> [1. text/plain]
> On 2023-12-26 at 15:56+01:00, Ricardo Wurmus wrote:
>> On 2023-12-26 at 14:53+01:00, Sergey Trofimov wrote:
>> > - adding it to guix increases maintenance burden: new versions
>> >   could add or remove config options
>>
>> This is why there should be automated tests.
>> There are too few of them.
>
> This is easier said than done.  A program could […]

Yes, it’s easy to say a lot of things.  This is not an argument against
providing tests *where feasible*.

> I would also want to share my experience as a user, that having
> to run guix home reconfigure iteratively is not exactly pleasant
> due to the high delay.

What’s a high delay?  Is a one second reconfiguration too long for what
it does?  I don’t think so.

-- 
Ricardo

Re: Are declarative app configs worth it?

[Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

On 2023-12-26 at 17:50+01:00, Ricardo Wurmus wrote:
> Nguyễn Gia Phong <cnx@loang.net> writes:
> > On 2023-12-26 at 15:56+01:00, Ricardo Wurmus wrote:
> >> On 2023-12-26 at 14:53+01:00, Sergey Trofimov wrote:
> >> > - adding it to guix increases maintenance burden: new versions
> >> >   could add or remove config options
> >>
> >> This is why there should be automated tests.
> >> There are too few of them.
> >
> > This is easier said than done.  A program could […]
>
> Yes, it’s easy to say a lot of things.  This is not an argument against
> providing tests *where feasible*.

I'm not against tests, but pointing out that relying on automated
tests to detect breakage is an unrealisitic expectation
for most programs.

On 2023-12-26 at 17:50+01:00, Ricardo Wurmus wrote:
> > I would also want to share my experience as a user, that having
> > to run guix home reconfigure iteratively is not exactly pleasant
> > due to the high delay.
>
> What’s a high delay?  Is a one second reconfiguration too long for what
> it does?  I don’t think so.

Any reconfig that isn't cached takes over 5s on my fairly modern
amd64 laptop, which is considerable for the repetitive
edit-config-then-test flow.  Even a cached home takes a whole 2s.

Reconfiguration just rebuild each config file in guix store regardless
if it's changed, so running it is a wasite of time and resources.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

Re: Are declarative app configs worth it?

[Attila Lendvai]

> > - adding it to guix increases maintenance burden: new versions could
> > add or remove config options
> 
> 
> This is why there should be automated tests. There are too few of them.


early detection of the breakage is just one part of the story. then it also needs to be fixed -- before dropping the hammer and abandoing the worksite.

writing and maintaining the tests have a cost, too.


> > - it requires documentation/translation, another hidden cost
> 
> 
> We should only accept configuration procedures that have proper
> documentation, yes.


in this context i recommed:

What is Seen and What is Not Seen
by Bastiat

https://oll.libertyfund.org/page/wswns

or specifically:

"In the sphere of economics an action, a habit, an institution or a law engenders not just one effect but a series of effects. Of these effects only the first is immediate; it is revealed simultaneously with its cause, it is seen. The others merely occur successively, they are not seen;3 we are lucky if we foresee them."

if you demand that e.g. all services accepted into guix have a configuration entry for every possible config field, and that the documentation of these fields are duplicated into the guix codebase... then whatever is included into guix will have a 100% coverage. this is what is seen.

but what about the lost potential? because i can guarantee you that while you'll get 100% coverage, you'll also only get a fraction of the total number services and fields.

which one will yield a better guix experience?

what i'm doing with my own services, and what i also recommend, is to always have an 'extra-arguments or 'extra-fields that allows defining any config value, and serializes it as-is. that way the user can rely on the documentation of the daemon, and blindly apply it while writing the guix config.

and only reify those couple of config fields into scheme code that can provide something useful beyond merely serializing the value as-is.

this way:
 - the guix codebase remains smaller (OAOO principle)
 - updating the app's package in guix is simpler
 - guaranteed not to get out of sync with the app
 - smaller threshold for new contributions
 - which translates to more supported services

i find the free-form module type, as suggested by John Soo above, to be a good idea. so much so that i may even look into writing a prototype and try to use it to replace my two inline shepherd-service instances.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“War is a ritual, a deadly ritual, not the result of aggressive self-assertion, but of self-transcending identification. Without loyalty to tribe, church, flag or ideal, there would be no wars.”
	— Arthur Koestler (1905–1983)

Re: Are declarative app configs worth it?

[Ricardo Wurmus]

Attila Lendvai <attila@lendvai.name> writes:

> if you demand that e.g. all services accepted into guix have a
> configuration entry for every possible config field, and that the
> documentation of these fields are duplicated into the guix
> codebase... then whatever is included into guix will have a 100%
> coverage. this is what is seen.

This is not what I argued for.  We also have had a couple of service
configurations that only provided options for a handful of options,
relegating all other possible options to a free text escape hatch.

There’s no point in talking percentages.

-- 
Ricardo

Re: Are declarative app configs worth it?

[Sergey Trofimov]

Ricardo Wurmus <rekado@elephly.net> writes:

> Sergey Trofimov <sarg@sarg.org.ru> writes:
>
>> - adding it to guix increases maintenance burden: new versions 
>> could
>>  add or remove config options
>
> This is why there should be automated tests.  There are too few 
> of them.
>

that adds up to the pile of boilerplate to implement a simple 
config. If guix mandates it for new packages, it'll raise the bar 
for contribution even higher than it already is.

>> - it bloats guix: imagine if we add configs for every
>>   user-configurable app
>
> That would be nice.
>
> If we started to accept the term bloat we could easily apply it 
> to
> anything in Guix: all that R stuff?  Bloat!  All that bioinfo 
> stuff?
> Bloat!
>

imo, R and bioinfo should be in channels.

>> - such configs are not easily transferrable: if I were to use 
>> the
>>   same app in non-guix env, I'd have to maintain 2 configs
>
> We are generating configuration files from our config languages. 
> So you
> would only need to generate them and copy them for your non-guix
> environment.
>

Sure, that's why I wrote "not easily". My non-guix env is a 
corporate Mac laptop. Currently I just clone my dotfiles, symlink 
required configs and it's done. I can make changes in both 
environments and there is no unnecessary "compiling" step 
involved.

>> Another recent example is `oci-container-configuration` which 
>> defines
>> a subset of docker-cli startup arguments. The problem is that 
>> `docker
>> run` command has 96 options and the configuration only uses a 
>> handful,
>> lacking a way to provide the remaining ones.
>
> All config bindings need to have an escape hatch.
>
That would be great.

Re: Are declarative app configs worth it?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2565 bytes --]

On Wed, Dec 27, 2023 at 08:38:24AM +0100, Sergey Trofimov wrote:
> 
> Ricardo Wurmus <rekado@elephly.net> writes:
> 
> > Sergey Trofimov <sarg@sarg.org.ru> writes:
> > 
> > > - adding it to guix increases maintenance burden: new versions could
> > >  add or remove config options
> > 
> > This is why there should be automated tests.  There are too few of them.
> > 
> 
> that adds up to the pile of boilerplate to implement a simple config. If
> guix mandates it for new packages, it'll raise the bar for contribution even
> higher than it already is.
> 
> > > - it bloats guix: imagine if we add configs for every
> > >   user-configurable app
> > 
> > That would be nice.
> > 
> > If we started to accept the term bloat we could easily apply it to
> > anything in Guix: all that R stuff?  Bloat!  All that bioinfo stuff?
> > Bloat!
> > 
> 
> imo, R and bioinfo should be in channels.

That wasn't a serious suggestion.

> > > - such configs are not easily transferrable: if I were to use the
> > >   same app in non-guix env, I'd have to maintain 2 configs
> > 
> > We are generating configuration files from our config languages. So you
> > would only need to generate them and copy them for your non-guix
> > environment.
> > 
> 
> Sure, that's why I wrote "not easily". My non-guix env is a corporate Mac
> laptop. Currently I just clone my dotfiles, symlink required configs and
> it's done. I can make changes in both environments and there is no
> unnecessary "compiling" step involved.

You're not required to use the guix-home bits. I didn't for a long time,
and there are still a lot of config files that I have that I either
symlink into place or I write out in my guix-home config file to be
splatted into place.  I still regularly scp my .screenrc and .inputrc to
other machines.

> > > Another recent example is `oci-container-configuration` which
> > > defines
> > > a subset of docker-cli startup arguments. The problem is that
> > > `docker
> > > run` command has 96 options and the configuration only uses a
> > > handful,
> > > lacking a way to provide the remaining ones.
> > 
> > All config bindings need to have an escape hatch.
> > 
> That would be great.

Most services have an extra-options (or similarly named) field where you
can add extra bits to the config.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Are declarative app configs worth it?

[Murad Mamedov]

Hi,

I was adding to guix fail2ban, greetd services with their 
configurations
and pending configuration for connman. Basically the overall idea 
behind
guix (i.e. having declarative configuration) is really nice. 
Having such
mechanism powered by general purpose language like scheme is 
another
very nice. In the ideal world, where all configurations are 
provided and
ported to guix with included documentation whould be beautiful 
from user
perspective.

However, we are not in an ideal world. From what I see, these 
problems
araise from three main sources:

- Programs they selves. For instance if every program would follow 
  let's
  say something like 12-factor app, then such programs would have 
  very
  clear purpose, and thus clear configuration. One of such good 
  examples
  in my practice would be greetd. Worst example would be fail2ban, 
  where
  half of program logic is expressed in the configuration, very 
  ugly
  very bad. Users have hard time to understand how it is 
  configured
  in plain, porting such mechanism to another dsl, whole another 
  level
  of pain. Rarely programs demand complex configuration, as some 
  one
  noted, there is already a conventional mechanism to add 
  `extra-config`
  field or having `plain-config-file-like` option saves the day.

- Porters of programs and services to guix. Such person should not
  blindly port the configuration to the guix, but clearly 
  understand
  how close or far away program from let's say 12-factor app (if 
  we
  take it as reference). If very close, providing detailed strict
  configuration records is a bliss. If very far, porter have to
  think and add `extra-config` and `plain-config-file-like` 
  options.

- Users of guix have to understand, that what they write in scheme
  for guix is not a configuration but a program. Once you trully
  understand that, as a user you may change your behavior to have
  configurations stored in the variables instead of inlining them
  within service configuration. Then they can be used in service,
  or manualy serialized to a file. Or even, one might write a 
  program
  to traverse services and/or operating-system and to serialize
  configuration on disk for LFS-like or systemd-like 
  compatibility.

IMHO, separating configuration (applications) from configuration 
(guix
applications?) is not very wise way and will defeat very idea of
having single, programming language based declarative 
configuration.

What we need is more tooling around configuration and 
documentation,
for instance:
- tool to extract documentation from man pages
- tool to "copy-paste" configuration documentation into guix.texi
- tool to cross link packages, configuration documentations
- code completion with documentation hinting in writing guix 
  scheme
- etc.

Thanks in advance,
muradm

Sergey Trofimov <sarg@sarg.org.ru> writes:

> Hi guix,
>
> I want to start a discussion around how to manage user app 
> config
> files.
> Copying my message from https://issues.guix.gnu.org/68010, where
> home-zathura-configuration with 76 fields is proposed.
>
> I have mixed feelings about pulling 3rd-party software 
> configurations
> in guix:
> - adding it to guix increases maintenance burden: new versions 
> could
>  add or remove config options
> - it requires documentation/translation, another hidden cost
> - it bloats guix: imagine if we add configs for every
>   user-configurable app
> - such configs are not easily transferrable: if I were to use 
> the
>   same app in non-guix env, I'd have to maintain 2 configs
>
> Another recent example is `oci-container-configuration` which 
> defines
> a subset of docker-cli startup arguments. The problem is that 
> `docker
> run` command has 96 options and the configuration only uses a 
> handful,
> lacking a way to provide the remaining ones.
>
> I think guix should not embed config generators for user 
> software. The
> only need I see for such generators is when there are options 
> which
> should be the same among multiple applications (e.g. color 
> schemes or
> shared directories). For such usecase guix should provide better 
> text
> manipulation tools which home owners could use to parameterise
> configs.
>
> Best regards,
> Sergey Trofimov