Best practices for writing services

Best practices for writing services

[Xinglu Chen]

Hi Guix,

I am going to write an mcron service for `guix home`[1][2] and before
proceding, I would like to get some suggestions on what the best
practices are for writing services in Guix.

Currently there seems to be two main ways to do this, the first one
is the define one or more records for the configuration field of a
service using `define-record-type*`, see the tor service in (gnu
services networking) for example.  The other method is to use
`define-configuration` to declare the configuration fields of a service,
see the transmission service in (gnu services file-sharing) for example.

The first method seems to be the more common one but the developer
usually has to write a lot of things manually.  The "real" configuration
file for the program or is usually created by concatenating a bunch of
strings and the developer has to write documentation for all the
configuration fields manually.

The second method removes quite a lot of boilerplate and the developer
will define different serializers that convert scheme syntax like lists,
alist, boolean... to the "real" configuration syntax of the program.  It
also does some automatic typechecking to some degree and allows the
developer to write docstrings for each configuration field.  There is
then a procedure called `generate-documentation` which can automatically
generate texinfo documenation from the docstrings.

I couldn't find any information in the manual regarding what conventions
should be used when writing services for Guix and would like to hear
from more experienced Guix hackers what the best practices are.

[1]: https://yhetil.org/guix-devel/878s6u2pco.fsf@trop.in
[2]: https://lists.sr.ht/~abcdw/rde-devel/%3C87h7l9r9xm.fsf%40yoctocell.xyz%3E

Re: Best practices for writing services

[Joshua Branson]

Xinglu Chen <public@yoctocell.xyz> writes:

> Hi Guix,
>
> I am going to write an mcron service for `guix home`[1][2] and before
> proceding, I would like to get some suggestions on what the best
> practices are for writing services in Guix.

Please note that I am a guix documentation contributor, and I am not
quite a guix developer yet.  :)  Though I do have a patch pending on
http://issues.guix.gnu.org/39136  (not certain why the 2/2 patch does
not show, 'cause I saw that come through in my inbox).

>
> Currently there seems to be two main ways to do this, the first one
> is the define one or more records for the configuration field of a
> service using `define-record-type*`, see the tor service in (gnu
> services networking) for example.  The other method is to use
> `define-configuration` to declare the configuration fields of a service,
> see the transmission service in (gnu services file-sharing) for example.

I believe that the first method via define-record-type* seems to be the
recommended method to do this.  I only say that because I feel like more
services are defined that way now.  :)

> The second method removes quite a lot of boilerplate and the developer
> will define different serializers that convert scheme syntax like lists,
> alist, boolean... to the "real" configuration syntax of the program.  It
> also does some automatic typechecking to some degree and allows the
> developer to write docstrings for each configuration field.  There is
> then a procedure called `generate-documentation` which can automatically
> generate texinfo documenation from the docstrings.

I do like that generate-documentation procedure.  Perhaps we could add
that for the define-record-type*  somehow...

>
> I couldn't find any information in the manual regarding what conventions
> should be used when writing services for Guix and would like to hear
> from more experienced Guix hackers what the best practices are.

You might try asking in irc in #guix too!

I've been working on a sway service and an endlessh service in my
hacking videos
(https://video.hardlimit.com/accounts/joshua_branson/video-channels).  I
was running into issues, where I could compile the service, but trying
to reconfigure my system would result in errors.  The errors messages
were a little vague.  I will also say that the better method I have
found in writing a guix service is to

1) write the service as simply as possible first.  I personally would
copy the simplest service that you can find in gnu/services/  and modify
that via a M-x anzu-query-replace-regexp.  If re-configuring works, make a commit.

2) If possible, containerize the service.  If it works, make a commit.

3) Now start adding in all the features you left out before.

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: Best practices for writing services

[Xinglu Chen]

On Fri, Mar 19 2021, Joshua Branson wrote:

>> Currently there seems to be two main ways to do this, the first one
>> is the define one or more records for the configuration field of a
>> service using `define-record-type*`, see the tor service in (gnu
>> services networking) for example.  The other method is to use
>> `define-configuration` to declare the configuration fields of a service,
>> see the transmission service in (gnu services file-sharing) for example.
>
> I believe that the first method via define-record-type* seems to be the
> recommended method to do this.  I only say that because I feel like more
> services are defined that way now.  :)

I decided to use `define-record-type*` since the service for mcron is
pretty simple because it is already configured in Guile. :)

> I've been working on a sway service and an endlessh service in my
> hacking videos
> (https://video.hardlimit.com/accounts/joshua_branson/video-channels).

Cool, great to see more videos on Guix, and it's on PeerTube.

> I was running into issues, where I could compile the service, but
> trying to reconfigure my system would result in errors.

The Guile compiler seems to miss quite a few errors in my exprience too.

> The errors messages were a little vague.  I will also say that the
> better method I have found in writing a guix service is to
>
> 1) write the service as simply as possible first.  I personally would
> copy the simplest service that you can find in gnu/services/  and modify
> that via a M-x anzu-query-replace-regexp.  If re-configuring works, make a commit.
>
> 2) If possible, containerize the service.  If it works, make a commit.
>
> 3) Now start adding in all the features you left out before.

Thanks for the suggestions, I will take them into account in the future.

Re: Best practices for writing services

[Maxim Cournoyer]

Hi Xinglu,

Xinglu Chen <public@yoctocell.xyz> writes:

> Hi Guix,
>
> I am going to write an mcron service for `guix home`[1][2] and before
> proceding, I would like to get some suggestions on what the best
> practices are for writing services in Guix.
>
> Currently there seems to be two main ways to do this, the first one
> is the define one or more records for the configuration field of a
> service using `define-record-type*`, see the tor service in (gnu
> services networking) for example.  The other method is to use
> `define-configuration` to declare the configuration fields of a service,
> see the transmission service in (gnu services file-sharing) for example.
>
> The first method seems to be the more common one but the developer
> usually has to write a lot of things manually.  The "real" configuration
> file for the program or is usually created by concatenating a bunch of
> strings and the developer has to write documentation for all the
> configuration fields manually.
>
> The second method removes quite a lot of boilerplate and the developer
> will define different serializers that convert scheme syntax like lists,
> alist, boolean... to the "real" configuration syntax of the program.  It
> also does some automatic typechecking to some degree and allows the
> developer to write docstrings for each configuration field.  There is
> then a procedure called `generate-documentation` which can automatically
> generate texinfo documenation from the docstrings.
>
> I couldn't find any information in the manual regarding what conventions
> should be used when writing services for Guix and would like to hear
> from more experienced Guix hackers what the best practices are.

That's a very good question, which I asked myself recently when
attempting to write my first non-trivial service for Guix
(jami-daemon-service-type).  I'd say 'define-configuration' is
technically superior because of the automatic type checking it provides.
It's also neat to be able to define the doc at one place and
auto-generate it (although that's still manual for now).  It has
definitely been collecting some dust compared to the simpler approach
using 'define-record-type*' directly, and the doc output it can generate
doesn't match what is currently the norm in the manual, so it'd need a
bit of a refresh.

Still, I see great potential for define-configuration.

Maxim

Re: Best practices for writing services

[Xinglu Chen]

Hi,

On Tue, Apr 20 2021, Maxim Cournoyer wrote:

> That's a very good question, which I asked myself recently when
> attempting to write my first non-trivial service for Guix
> (jami-daemon-service-type).  I'd say 'define-configuration' is
> technically superior because of the automatic type checking it provides.
> It's also neat to be able to define the doc at one place and
> auto-generate it (although that's still manual for now).

You mean that one has to manually evaluate the ‘generate-documentation’
sexp and paste the output of it in the manual?  I thought it would
automatically update the manual when running ‘make’. ;)

> It has definitely been collecting some dust compared to the simpler
> approach using 'define-record-type*' directly, and the doc output it
> can generate doesn't match what is currently the norm in the manual,
> so it'd need a bit of a refresh.

Yeah, I noticed that some sections in the manual looked a bit different
than the rest.

One thing that I find a little annoying is that you have to specify a
default value for the fields.  This doesn’t make much sense in some
cases, e.g. there is no good default value for ‘user.name = NAME’ in the
Git config, the user should have to specify that themselves.

Another thing is that I don’t always want to “serialize-” the value for a
field, so I sometimes end up defining a bunch of dummy serializers that
just return an empty string.

Re: Best practices for writing services

[Maxim Cournoyer]

Hi Xinglu,

Xinglu Chen <public@yoctocell.xyz> writes:

> Hi,
>
> On Tue, Apr 20 2021, Maxim Cournoyer wrote:
>
>> That's a very good question, which I asked myself recently when
>> attempting to write my first non-trivial service for Guix
>> (jami-daemon-service-type).  I'd say 'define-configuration' is
>> technically superior because of the automatic type checking it provides.
>> It's also neat to be able to define the doc at one place and
>> auto-generate it (although that's still manual for now).
>
> You mean that one has to manually evaluate the ‘generate-documentation’
> sexp and paste the output of it in the manual?  I thought it would
> automatically update the manual when running ‘make’. ;)

Yes, that's my current understanding of how things are.  It could be
changed, for sure.

>> It has definitely been collecting some dust compared to the simpler
>> approach using 'define-record-type*' directly, and the doc output it
>> can generate doesn't match what is currently the norm in the manual,
>> so it'd need a bit of a refresh.
>
> Yeah, I noticed that some sections in the manual looked a bit different
> than the rest.
>
> One thing that I find a little annoying is that you have to specify a
> default value for the fields.  This doesn’t make much sense in some
> cases, e.g. there is no good default value for ‘user.name = NAME’ in the
> Git config, the user should have to specify that themselves.

There's a 'define-maybe' that can be used to declare a field that can
take more than one type, e.g.:

(define-maybe string)

Will generate a definition like so:

--8<---------------cut here---------------start------------->8---
(define (maybe-string? val)
               (or (eq? val 'disabled) (string? val)))
--8<---------------cut here---------------end--------------->8---

Which the validator of define-configuration will use if you specify a
field with the type 'maybe-string'.

'disabled is a bit semantically broken in some cases ('unspecified'
could be nicer), but it does the job.

> Another thing is that I don’t always want to “serialize-” the value for a
> field, so I sometimes end up defining a bunch of dummy serializers that
> just return an empty string.

Good point!  I've tried addressing that, without success so far [0].

Maxim

[0]  https://lists.gnu.org/archive/html/guix-devel/2021-04/msg00184.html

Re: Best practices for writing services

[Xinglu Chen]

On Wed, Apr 21 2021, Maxim Cournoyer wrote:

>> One thing that I find a little annoying is that you have to specify a
>> default value for the fields.  This doesn’t make much sense in some
>> cases, e.g. there is no good default value for ‘user.name = NAME’ in the
>> Git config, the user should have to specify that themselves.
>
> There's a 'define-maybe' that can be used to declare a field that can
> take more than one type, e.g.:
>
> (define-maybe string)
>
> Will generate a definition like so:
>
> --8<---------------cut here---------------start------------->8---
> (define (maybe-string? val)
>                (or (eq? val 'disabled) (string? val)))
> --8<---------------cut here---------------end--------------->8---
>
> Which the validator of define-configuration will use if you specify a
> field with the type 'maybe-string'.
>
> 'disabled is a bit semantically broken in some cases ('unspecified'
> could be nicer), but it does the job.

But the problem here is that it doesn’t force the user to configure the
field.  In a Git config for example, the user should be forced to set
‘user.name’ and ‘user.email’, otherwise they can’t commit anything.  You
will just have to set the default value to ‘disabled’, like this:

#+begin_src scheme
(define (serialize-string field-name val) ...)
(define-maybe string)
(define-configuration test-config
  (config
    (maybe-string ’disabled))
    "docs"")
#+end_src

>> Another thing is that I don’t always want to “serialize-” the value for a
>> field, so I sometimes end up defining a bunch of dummy serializers that
>> just return an empty string.
>
> Good point!  I've tried addressing that, without success so far [0].

Cool, that would definitely be an improvement!

Re: Best practices for writing services

[Maxim Cournoyer]

Hello,

Xinglu Chen <public@yoctocell.xyz> writes:

> On Wed, Apr 21 2021, Maxim Cournoyer wrote:
>
>>> One thing that I find a little annoying is that you have to specify a
>>> default value for the fields.  This doesn’t make much sense in some
>>> cases, e.g. there is no good default value for ‘user.name = NAME’ in the
>>> Git config, the user should have to specify that themselves.
>>
>> There's a 'define-maybe' that can be used to declare a field that can
>> take more than one type, e.g.:
>>
>> (define-maybe string)
>>
>> Will generate a definition like so:
>>
>> --8<---------------cut here---------------start------------->8---
>> (define (maybe-string? val)
>>                (or (eq? val 'disabled) (string? val)))
>> --8<---------------cut here---------------end--------------->8---
>>
>> Which the validator of define-configuration will use if you specify a
>> field with the type 'maybe-string'.
>>
>> 'disabled is a bit semantically broken in some cases ('unspecified'
>> could be nicer), but it does the job.
>
> But the problem here is that it doesn’t force the user to configure the
> field.  In a Git config for example, the user should be forced to set
> ‘user.name’ and ‘user.email’, otherwise they can’t commit anything.  You
> will just have to set the default value to ‘disabled’, like this:
>
> #+begin_src scheme
> (define (serialize-string field-name val) ...)
> (define-maybe string)
> (define-configuration test-config
>   (config
>     (maybe-string ’disabled))
>     "docs"")
> #+end_src

Ah, thanks for explaining, now I understand your point well.

I've just tried something:

--8<---------------cut here---------------start------------->8---
(define-configuration test-config
                       (name (string #f) "Your name"))
scheme@(guile-user)> (test-config)
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
ERROR:
  1. &message: "Invalid value for field name: #f"
  2. &configuration-error
--8<---------------cut here---------------end--------------->8---

So you could choose an invalid default value, which would force the user
to specify it (else they'd get the not so obvious error message above).
It should be improved too!  I'll see if I can do something.

Thanks!

Maxim

Re: Best practices for writing services

[Xinglu Chen]

Hi,

On Fri, Apr 23 2021, Maxim Cournoyer wrote:

>> But the problem here is that it doesn’t force the user to configure the
>> field.  In a Git config for example, the user should be forced to set
>> ‘user.name’ and ‘user.email’, otherwise they can’t commit anything.  You
>> will just have to set the default value to ‘disabled’, like this:
>>
>> #+begin_src scheme
>> (define (serialize-string field-name val) ...)
>> (define-maybe string)
>> (define-configuration test-config
>>   (config
>>     (maybe-string ’disabled))
>>     "docs"")
>> #+end_src
>
> Ah, thanks for explaining, now I understand your point well.
>
> I've just tried something:
>
> --8<---------------cut here---------------start------------->8---
> (define-configuration test-config
>                        (name (string #f) "Your name"))
> scheme@(guile-user)> (test-config)
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> ERROR:
>   1. &message: "Invalid value for field name: #f"
>   2. &configuration-error
> --8<---------------cut here---------------end--------------->8---
>
> So you could choose an invalid default value, which would force the user
> to specify it (else they'd get the not so obvious error message above).
> It should be improved too!  I'll see if I can do something.

That would be a workaround for now.  Thank you!

Re: Best practices for writing services

[raingloom]

On Wed, 21 Apr 2021 12:45:21 +0200
Xinglu Chen <public@yoctocell.xyz> wrote:

> One thing that I find a little annoying is that you have to specify a
> default value for the fields.

Are you sure? If you don't specify a default, won't the user just be
forced to write
(service whatever
	(whatever-configuration
		(mandatory-field 'bleepbloop)))

instead of the shorter (service whatever)?

Re: Best practices for writing services

[Xinglu Chen]

On Thu, Apr 22 2021, raingloom wrote:

>> One thing that I find a little annoying is that you have to specify a
>> default value for the fields.
>
> Are you sure? If you don't specify a default, won't the user just be
> forced to write
> (service whatever
> 	(whatever-configuration
> 		(mandatory-field 'bleepbloop)))
>
> instead of the shorter (service whatever)?

If I write something like this

#+begin_src scheme
(use-modules (gnu services configuration))
(define (serialize-list field-name val) "")
(define-configuration test-config
  (config
   (list)
   "configuration for test"))
#+end_src

and evaluate it, I will get an error.  I have to specify a default value
for the ‘config’ field to make it work.

#+begin_src scheme
(use-modules (gnu services configuration))
(define (serialize-list field-name val) "")
(define-configuration test-config
  (config
   (list '())                           ;default to '()
   "configuration for test"))
#+end_src

Re: Best practices for writing services

[raingloom]

On Thu, 22 Apr 2021 09:19:56 +0200
Xinglu Chen <public@yoctocell.xyz> wrote:

> On Thu, Apr 22 2021, raingloom wrote:
> 
> >> One thing that I find a little annoying is that you have to
> >> specify a default value for the fields.  
> >
> > Are you sure? If you don't specify a default, won't the user just be
> > forced to write
> > (service whatever
> > 	(whatever-configuration
> > 		(mandatory-field 'bleepbloop)))
> >
> > instead of the shorter (service whatever)?  
> 
> If I write something like this
> 
> #+begin_src scheme
> (use-modules (gnu services configuration))
> (define (serialize-list field-name val) "")
> (define-configuration test-config
>   (config
>    (list)
>    "configuration for test"))
> #+end_src
> 
> and evaluate it, I will get an error.  I have to specify a default
> value for the ‘config’ field to make it work.
> 
> #+begin_src scheme
> (use-modules (gnu services configuration))
> (define (serialize-list field-name val) "")
> (define-configuration test-config
>   (config
>    (list '())                           ;default to '()
>    "configuration for test"))
> #+end_src
> 

Weird. I'd consider this a bug in define-configuration.
define-record-type* does not have this limitation.