A friendlier API for operating-system declarations

A friendlier API for operating-system declarations

[Edouard Klein]

Dear Guixers,

For my clients and my own use, I use a layer on top of operating-system
declarations in which I create functions that take an os as a first
argument, and return an os.

With the help of the handy -> macro, I can chain them, thus allowing an
easy stacking of multiple "roles" or "functionalities" on the same
machine, and an easy transfer of said functionality to another machine:
just cut and paste one line from one declaration to the other.

I have written a tutorial for my clients here:

https://guix-hosting.com/docs/tuto/getting-started.html

that gives an example of what an os configuration may look like:
#+begin_src scheme
(->
 (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
 (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
 (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
 (add-services my-db))
#+end_src

The code of the function is on my channel:
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm

After a few months of experience, and positive feedback from my clients,
my question to you guys is: would you be interested in mainlining this,
or should I keep my development efforts separate in my channel ?

I do think this API is easier than manipulating services, and although
extendable services are awesome and a very nifty piece of engineering,
they require quite a good knowledge of scheme and take a while to be
used to, while this new API, while way less powerful, lowers the barrier
to entry for newcomers.

They are an easy way to maintain a declarative whole operating system
configuration, with a syntax similar enough to docker and ansible that
sysadmins familiar with it can quickly get up and running, thus exposing
more people to Guix.

What do you think ?

Cheers,

Edouard.

Re: A friendlier API for operating-system declarations

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1742 bytes --]

Hi Edouard,

Edouard Klein <edk@beaver-labs.com> writes:

> #+begin_src scheme
> (->
>  (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
>  (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
>  (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
>  (add-services my-db))
> #+end_src
>
> The code of the function is on my channel:
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm
>
> After a few months of experience, and positive feedback from my clients,
> my question to you guys is: would you be interested in mainlining this,
> or should I keep my development efforts separate in my channel ?

I am quite in favor of using operating-system transformations rather
than inheritance, because they're composable!  This could be leveraged
to get a nice CLI API as well.

> I do think this API is easier than manipulating services, and although
> extendable services are awesome and a very nifty piece of engineering,
> they require quite a good knowledge of scheme and take a while to be
> used to, while this new API, while way less powerful, lowers the barrier
> to entry for newcomers.

By this, do you mean that there's no way with your syntax to modify a
given service?  Is there a reason for this?  It does seem to me that it
could probably be done.

> They are an easy way to maintain a declarative whole operating system
> configuration, with a syntax similar enough to docker and ansible that
> sysadmins familiar with it can quickly get up and running, thus exposing
> more people to Guix.
>
> What do you think ?

You've got me interested :) especially since you already have customer
feedback!

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: A friendlier API for operating-system declarations

[Edouard Klein]

Hi Josselin,

Josselin Poiret <dev@jpoiret.xyz> writes:

> [[PGP Signed Part:Undecided]]
> Hi Edouard,
>
> Edouard Klein <edk@beaver-labs.com> writes:
>
>> #+begin_src scheme
>> (->
>>  (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
>>  (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
>>  (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
>>  (add-services my-db))
>> #+end_src
>>
>> The code of the function is on my channel:
>> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm
>>
>> After a few months of experience, and positive feedback from my clients,
>> my question to you guys is: would you be interested in mainlining this,
>> or should I keep my development efforts separate in my channel ?
>
> I am quite in favor of using operating-system transformations rather
> than inheritance, because they're composable!  This could be leveraged
> to get a nice CLI API as well.
>

I hadn't thought of that but now that you mention it, starting a basic
web server or database, isolated in a container, directly from the
command line may come in handy sometimes. This is a good idea ! I'll
think about how to achieve this :)

In the meantime something like:

#+begin_src bash
guix system container -e "(begin (use-modules (beaver system))
(http-static-content minimal-container))" --share=somedir=/var/www
#+end_src

Will spin up such a server. Kind of like python3 -m http.server, but
without the security flaws (you get a full blown nginx), and isolated in
a way that even if your process gets owned, it's only an unpriviledged
user in a container.


>> I do think this API is easier than manipulating services, and although
>> extendable services are awesome and a very nifty piece of engineering,
>> they require quite a good knowledge of scheme and take a while to be
>> used to, while this new API, while way less powerful, lowers the barrier
>> to entry for newcomers.
>
> By this, do you mean that there's no way with your syntax to modify a
> given service?  Is there a reason for this?  It does seem to me that it
> could probably be done.
>

I'm not sure I understand your question.

What I meant is that my proposal is only syntactic sugar over the
existing system, and not meant to replace it.

The existing system is actually very good and the way you
can define how to collate the instances of a given service type, even if
they are instanciated far away from one another, likely by different
authors is really clever and well made. I'm talking here about things
like how you can easily request that a new user be made for your
service, and account-service-type will collate all the users that need
to exist, and actually get the job done. Basically what fold-services does.

I just want to hide all of this complexity for people who just want to
activate one or more instances of a service coded up by somebody else.

If by modifying a service you mean the parameters like the port ones
listen to, or the directory the data should be put in etc. the functions
just take keyword parameters and pass them to the underlying
*-configuration scheme record. They are quite easy to edit and compose.
Maybe we could even devise a semi-automated way of generating one function
for every *-configuration structure that exists.

See
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm#L236

how http-static-content is basically just a wrapper over nginx-server-configuration.

If you meant editing the service after the function has run, you then
need to inspect the operating-system record and fall back to the
original API, by finding the service in the record and editing its
members, but that seems complex to me. I'm probably misunderstanding
the use case.



>> They are an easy way to maintain a declarative whole operating system
>> configuration, with a syntax similar enough to docker and ansible that
>> sysadmins familiar with it can quickly get up and running, thus exposing
>> more people to Guix.
>>
>> What do you think ?
>
> You've got me interested :) especially since you already have customer
> feedback!
>

Thanks :)



Cheers,

Edouard.
> Best,

Re: A friendlier API for operating-system declarations

[Liliana Marie Prikler]

Am Donnerstag, dem 23.03.2023 um 09:06 +0100 schrieb Edouard Klein:
> After a few months of experience, and positive feedback from my
> clients, my question to you guys is: would you be interested in
> mainlining this, or should I keep my development efforts separate in
> my channel ?
Having just dropped a series of "old-style" service procedures, I don't
think that maintaining a separate API on the mainline is a good idea.

> I do think this API is easier than manipulating services, and
> although extendable services are awesome and a very nifty piece of
> engineering, they require quite a good knowledge of scheme and take a
> while to be used to, while this new API, while way less powerful,
> lowers the barrier to entry for newcomers.
I see how this style (and a similar one that has already been shown for
guix home) can improve configuration file clarity and that's a good
thing.  On the other hand, it looks like you currently have to manually
craft a lot of the procedures that go into the -> syntax and that's...
well, not so good.

If we could enable this style easier – say if we made it so that
define-configuration and service-type also emitted ready-to-use
constructors to use with a keyword style at little extra cost other
than compilation time, I think that'd be a net positive for everyone
involved.

Cheers

Re: A friendlier API for operating-system declarations

[Ludovic Courtès]

Hi Edouard,

Edouard Klein <edk@beaver-labs.com> skribis:

> For my clients and my own use, I use a layer on top of operating-system
> declarations in which I create functions that take an os as a first
> argument, and return an os.
>
> With the help of the handy -> macro, I can chain them, thus allowing an
> easy stacking of multiple "roles" or "functionalities" on the same
> machine, and an easy transfer of said functionality to another machine:
> just cut and paste one line from one declaration to the other.
>
> I have written a tutorial for my clients here:
>
> https://guix-hosting.com/docs/tuto/getting-started.html

Very nice!

> After a few months of experience, and positive feedback from my clients,
> my question to you guys is: would you be interested in mainlining this,
> or should I keep my development efforts separate in my channel ?

I think it’s worth considering, taking into account maintenance
considerations as Liliana wrote.

> I do think this API is easier than manipulating services, and although
> extendable services are awesome and a very nifty piece of engineering,
> they require quite a good knowledge of scheme and take a while to be
> used to, while this new API, while way less powerful, lowers the barrier
> to entry for newcomers.

Normally, adding a service amounts to adding a line in the ‘services’
field:

  (operating-system
    ;; …
    (services (append (list (service nginx-service-type) …)
                      %base-services)))

It is more verbose that what you have, but it’s still roughly one line
(or one entry) per service.

Another interesting bit in your channel are templates though: an OS
template for hosting at OVH, etc.  These are very valuable because they
take care of all the boilerplate.

Thanks for sharing!

Ludo’.

Re: A friendlier API for operating-system declarations

[Edouard Klein]

Hi !
Ludovic Courtès <ludo@gnu.org> writes:

> Hi Edouard,
>
> Edouard Klein <edk@beaver-labs.com> skribis:
>
>> For my clients and my own use, I use a layer on top of operating-system
>> declarations in which I create functions that take an os as a first
>> argument, and return an os.
>>
>> With the help of the handy -> macro, I can chain them, thus allowing an
>> easy stacking of multiple "roles" or "functionalities" on the same
>> machine, and an easy transfer of said functionality to another machine:
>> just cut and paste one line from one declaration to the other.
>>
>> I have written a tutorial for my clients here:
>>
>> https://guix-hosting.com/docs/tuto/getting-started.html
>
> Very nice!
>

:)

>> After a few months of experience, and positive feedback from my clients,
>> my question to you guys is: would you be interested in mainlining this,
>> or should I keep my development efforts separate in my channel ?
>
> I think it’s worth considering, taking into account maintenance
> considerations as Liliana wrote.
>

Indeed. This was started as a quick and dirty overlay over guix to make
it easier for people used to Ansible to deal with declarative OS
configuration with guix, but it now need to be easy to maintain.

>> I do think this API is easier than manipulating services, and although
>> extendable services are awesome and a very nifty piece of engineering,
>> they require quite a good knowledge of scheme and take a while to be
>> used to, while this new API, while way less powerful, lowers the barrier
>> to entry for newcomers.
>
> Normally, adding a service amounts to adding a line in the ‘services’
> field:
>
>   (operating-system
>     ;; …
>     (services (append (list (service nginx-service-type) …)
>                       %base-services)))
>
> It is more verbose that what you have, but it’s still roughly one line
> (or one entry) per service.
>

You wouldn't believe the irrational fear of nested parentheses some
people have ;-)

I do agree a purely functional interface to OS declaration wouldn't
bring much over the existing one. My goal is to then compose those
primitive functions into usually-used-together bundles, such as e.g.
nginx+gunicorn+flask in a function called, maybe, 'python-webapp'.

This is kind of subjective, though. I have my own usage and some
customer feedback to guide me, but we can keep it on my channel until we
feel confident that the additional value is worth the maintenance cost.


> Another interesting bit in your channel are templates though: an OS
> template for hosting at OVH, etc.  These are very valuable because they
> take care of all the boilerplate.
>

I'd be happy to mainline it. I'll submit a patch when I get back home.


> Thanks for sharing!
>

Well thanks to you for starting the whole schmilblick.

> Ludo’.

Edouard.

Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Dear Guixers,

Here is a quick status update on my proposition to expose composable
functions to change operating-system declarations.

Thank you all for the feedback you gave me :) It's very nice to not be
talking in the void.

After Liliana opined that these functions should not put too much burden
on the maintainers, a position with which I wholeheartedly agree, here
is what I came up with:

https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/functional-services.scm

(file copied at the end of this email to make the list self-content
instead of relying on external services)

That's a macro-writing macro called "define-os-macros-for" that, for any
service (for example nginx), defines five forms. They all evaluate to a
modified os:

(.nginx os)  ;; Add nginx in its default configuration
(.nginx os toto titi tutu... )  ;; Add nginx, with toto... given as
arguments to nginx-configuration
(+nginx os toto titi tutu...)  ;; Extend an existing nginx service, giving
toto... as arguments to simple-service
(~nginx os toto...)  ;; Edit an existing nginx service, passing toto... as
arguments to modify-service
(-nginx os)  ;; Removes nginx

You can see all of them used in:
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm?ref_type=heads

e.g. the mkdir-p function is a (+activation ...),
all web server functions use (+nginx...),
mumble is (.mumble... ),
os/git uses (~openssh... ).

I've use (-foo) forms while testing, but not in production. The others
are currently running in prod on guix-hosting.com and the-dam.org.

That'll be the meat of the first half of my presentation during the
informal Paris meetup in december.

Do you folks think something like that could one day make it in the
mainline ?

Cheers,

Edouard.


-----------------functional-services.scm----------------
(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:export (*-append))

(define (syntax->string s)
  "Shorthand to convert a piece of syntax to a string"
  (symbol->string (syntax->datum s)))

(define (*->string s)
  "Convert into a string any symbol-like type we may encounter in a macro"
  (cond
   [(string? s) s]
   [(symbol? s) (symbol->string s)]
   [else (syntax->string s)]))  ;; assume syntax

(define* (*-append #:rest args)
  "Return a symbol which is a concatenation of the given symbol-like args"
  (string->symbol (apply string-append (map *->string args))))


(define-syntax define-os-macros-for
  ;; This macro, called like e.g. (define-os-macros-for foo) will define four
  ;; forms, .foo, +foo, ~foo and -foo
  ;;
  ;; These forms take an operating-system as their first argument, and
  ;; evaluate to an operating-system, which allows one to compose them
  ;; (e.g. (foo (bar os))).
  (lambda (x)
    (syntax-case x ()
      [(_ foo)
       (with-syntax
        ([dot-foo           (datum->syntax x (*-append "." #'foo))]
         [plus-foo          (datum->syntax x (*-append "+" #'foo))]
         [tilde-foo         (datum->syntax x (*-append "~" #'foo))]
         [minus-foo         (datum->syntax x (*-append "-" #'foo))]
         [foo-service-type  (datum->syntax x (*-append #'foo "-service-type"))]
         [foo-configuration (datum->syntax x (*-append #'foo "-configuration"))])
        #'(begin
            ;; The fist macro is .foo
            (define-syntax dot-foo
              (syntax-rules ()
                ;; Its simplest form is (.foo os), which will add an instance
                ;; of service foo to os.
                [(dot-foo os)
                 ((lambda (x)  ;; It is wrapped in a lamba to make sure os is
                    ;; evaluated once only. It it wasn't in a labmda, whatever
                    ;; form os is in the calling code would be repeated
                    ;; multiple times, and so if the form was e.g. (some-func
                    ;; os), then some-func would be called multiple times,
                    ;; which may not be desirable.
                    (operating-system
                      (inherit x)
                      (services
                       (cons
                        (service foo-service-type)
                        (operating-system-user-services x)))))
                  os)]
                ;; This other form, (.foo os something bar baz...) takes
                ;; multiple forms as arguments, which are reproduced as is to
                ;; construct the foo-configuration
                [(dot-foo os forms (... ...))
                 ((lambda (x)  ;; Wrapping in a lambda for the same reasons as above
                    (operating-system
                      (inherit x)
                      (services
                       (cons
                        (service foo-service-type
                                 (foo-configuration forms (... ...)))
                        (operating-system-user-services x)))))
                  os)]))
            ;; The second one is +foo. It extends a service, reproducing the
            ;; given forms as arguments to the simple-service procedure
            (define-syntax plus-foo
              (syntax-rules ()
                [(plus-foo os forms (... ...))
                 ((lambda (x)
                    (operating-system
                      (inherit x)
                      (services
                       (cons
                        (simple-service (format  #f "A ~a extension" (*->string #'foo))
                                        foo-service-type
                                        forms (... ...))
                        (operating-system-user-services x)))))
                  os)]))
            ;; The third one is ~foo. It modifies existing instances of a service,
            ;; passing the given forms as arguments to the modify-service macro
            (define-syntax tilde-foo
              (syntax-rules ()
                [(tilde-foo os forms (... ...))
                 ((lambda (x)
                    (operating-system
                      (inherit x)
                      (services
                       (modify-services (operating-system-user-services x)
                         (foo-service-type
                          config =>
                          (foo-configuration
                           (inherit config)
                           forms (... ...)))))))
                  os)]))
            ;; The fourth one is -foo. It removes foo-service-type from the
            ;; os' services
            (define-syntax minus-foo
              (syntax-rules ()
                [(minus-foo os)
                 ((lambda (x)
                    (operating-system
                      (inherit x)
                      (services
                       (modify-services (operating-system-user-services x)
                         (delete foo-service-type)))))
                  os)]))
            ;;Don't forget to make .foo, +foo, ~foo, and -foo visible
            (export dot-foo plus-foo tilde-foo minus-foo)))])))


;; What follows is generated by running the following fish command in the
;; guix repo :

;; grep -E "define[^\(]*-service-type" **.scm | sort | uniq | sed 's|/| |g' | sed -E 's/(gnu|guix)/\(use-modules \(\1/' | sed -E 's/.scm/\)\)/' | sed -E 's/:\(define(-public|-syntax)* / \(define-os-macros-for /' | grep -v define-record-type | grep -Ev 'lookup.*-service-types' | grep -v python-os-service-types  | sed -E 's/([^ ]*)-service-type/\1\)/'
(use-modules (gnu home services)) (define-os-macros-for home-activation)
(use-modules (gnu home services)) (define-os-macros-for home-environment-variables)
(use-modules (gnu home services)) (define-os-macros-for home-files)
(use-modules (gnu home services)) (define-os-macros-for home-profile)
(use-modules (gnu home services)) (define-os-macros-for home-provenance)
(use-modules (gnu home services)) (define-os-macros-for home-run-on-change)
(use-modules (gnu home services)) (define-os-macros-for home-run-on-first-login)
(use-modules (gnu home services)) (define-os-macros-for home)
(use-modules (gnu home services)) (define-os-macros-for home-xdg-configuration-files)
(use-modules (gnu home services)) (define-os-macros-for home-xdg-data-files)
(use-modules (gnu home services desktop)) (define-os-macros-for home-dbus)
(use-modules (gnu home services desktop)) (define-os-macros-for home-redshift)
(use-modules (gnu home services desktop)) (define-os-macros-for home-unclutter)
(use-modules (gnu home services desktop)) (define-os-macros-for home-xmodmap)
(use-modules (gnu home services fontutils)) (define-os-macros-for home-fontconfig)
(use-modules (gnu home services gnupg)) (define-os-macros-for home-gpg-agent)
(use-modules (gnu home services guix)) (define-os-macros-for home-channels)
(use-modules (gnu home services mail)) (define-os-macros-for home-msmtp)
(use-modules (gnu home services mcron)) (define-os-macros-for home-mcron)
(use-modules (gnu home services media)) (define-os-macros-for home-kodi)
(use-modules (gnu home services messaging)) (define-os-macros-for home-znc)
(use-modules (gnu home services pm)) (define-os-macros-for home-batsignal)
(use-modules (gnu home services shells)) (define-os-macros-for home-bash)
(use-modules (gnu home services shells)) (define-os-macros-for home-fish)
(use-modules (gnu home services shells)) (define-os-macros-for home-shell-profile)
(use-modules (gnu home services shells)) (define-os-macros-for home-zsh)
(use-modules (gnu home services shepherd)) (define-os-macros-for home-shepherd)
(use-modules (gnu home services sound)) (define-os-macros-for home-pulseaudio-rtp-sink)
(use-modules (gnu home services sound)) (define-os-macros-for home-pulseaudio-rtp-source)
(use-modules (gnu home services ssh)) (define-os-macros-for home-openssh)
(use-modules (gnu home services ssh)) (define-os-macros-for home-ssh-agent)
(use-modules (gnu home services symlink-manager)) (define-os-macros-for home-symlink-manager)
(use-modules (gnu home services xdg)) (define-os-macros-for home-xdg-base-directories)
(use-modules (gnu home services xdg)) (define-os-macros-for home-xdg-mime-applications)
(use-modules (gnu home services xdg)) (define-os-macros-for home-xdg-user-directories)
(use-modules (gnu services)) (define-os-macros-for activation)
(use-modules (gnu services)) (define-os-macros-for boot)
(use-modules (gnu services)) (define-os-macros-for cleanup)
(use-modules (gnu services)) (define-os-macros-for etc)
(use-modules (gnu services)) (define-os-macros-for firmware)
(use-modules (gnu services)) (define-os-macros-for gc-root)
(use-modules (gnu services)) (define-os-macros-for hurd-startup)
(use-modules (gnu services)) (define-os-macros-for linux-builder)
(use-modules (gnu services)) (define-os-macros-for linux-loadable-module)
(use-modules (gnu services)) (define-os-macros-for profile)
(use-modules (gnu services)) (define-os-macros-for provenance)
(use-modules (gnu services)) (define-os-macros-for setuid-program)
(use-modules (gnu services)) (define-os-macros-for special-files)
(use-modules (gnu services)) (define-os-macros-for system)
(use-modules (gnu services admin)) (define-os-macros-for log-cleanup)
(use-modules (gnu services admin)) (define-os-macros-for rottlog)
(use-modules (gnu services admin)) (define-os-macros-for unattended-upgrade)
(use-modules (gnu services audio)) (define-os-macros-for mpd)
(use-modules (gnu services audio)) (define-os-macros-for mympd)
(use-modules (gnu services auditd)) (define-os-macros-for auditd)
(use-modules (gnu services authentication)) (define-os-macros-for fprintd)
(use-modules (gnu services authentication)) (define-os-macros-for nslcd)
(use-modules (gnu services avahi)) (define-os-macros-for avahi)
(use-modules (gnu services base)) (define-os-macros-for agetty)
(use-modules (gnu services base)) (define-os-macros-for console-font)
(use-modules (gnu services base)) (define-os-macros-for file-system)
(use-modules (gnu services base)) (define-os-macros-for fstab)
(use-modules (gnu services base)) (define-os-macros-for gpm)
(use-modules (gnu services base)) (define-os-macros-for greetd)
(use-modules (gnu services base)) (define-os-macros-for guix-publish)
(use-modules (gnu services base)) (define-os-macros-for guix)
(use-modules (gnu services base)) (define-os-macros-for host-name)
(use-modules (gnu services base)) (define-os-macros-for hosts)
(use-modules (gnu services base)) (define-os-macros-for kmscon)
(use-modules (gnu services base)) (define-os-macros-for login)
(use-modules (gnu services base)) (define-os-macros-for mingetty)
(use-modules (gnu services base)) (define-os-macros-for nscd)
(use-modules (gnu services base)) (define-os-macros-for pam-limits)
(use-modules (gnu services base)) (define-os-macros-for rngd)
(use-modules (gnu services base)) (define-os-macros-for root-file-system)
(use-modules (gnu services base)) (define-os-macros-for static-networking)
(use-modules (gnu services base)) (define-os-macros-for swap)
(use-modules (gnu services base)) (define-os-macros-for syslog)
(use-modules (gnu services base)) (define-os-macros-for udev)
(use-modules (gnu services base)) (define-os-macros-for urandom-seed)
(use-modules (gnu services base)) (define-os-macros-for virtual-terminal)
(use-modules (gnu services certbot)) (define-os-macros-for certbot)
(use-modules (gnu services cgit)) (define-os-macros-for cgit)
(use-modules (gnu services ci)) (define-os-macros-for laminar)
(use-modules (gnu services cuirass)) (define-os-macros-for cuirass-remote-worker)
(use-modules (gnu services cuirass)) (define-os-macros-for cuirass)
(use-modules (gnu services cups)) (define-os-macros-for cups)
(use-modules (gnu services databases)) (define-os-macros-for memcached)
(use-modules (gnu services databases)) (define-os-macros-for mysql)
(use-modules (gnu services databases)) (define-os-macros-for postgresql-role)
(use-modules (gnu services databases)) (define-os-macros-for postgresql)
(use-modules (gnu services databases)) (define-os-macros-for redis)
(use-modules (gnu services dbus)) (define-os-macros-for dbus-root)
(use-modules (gnu services dbus)) (define-os-macros-for polkit)
(use-modules (gnu services desktop)) (define-os-macros-for accountsservice)
(use-modules (gnu services desktop)) (define-os-macros-for bluetooth)
(use-modules (gnu services desktop)) (define-os-macros-for colord)
(use-modules (gnu services desktop)) (define-os-macros-for cups-pk-helper)
(use-modules (gnu services desktop)) (define-os-macros-for elogind)
(use-modules (gnu services desktop)) (define-os-macros-for enlightenment-desktop)
(use-modules (gnu services desktop)) (define-os-macros-for geoclue)
(use-modules (gnu services desktop)) (define-os-macros-for gnome-desktop)
(use-modules (gnu services desktop)) (define-os-macros-for gnome-keyring)
(use-modules (gnu services desktop)) (define-os-macros-for inputattach)
(use-modules (gnu services desktop)) (define-os-macros-for lxqt-desktop)
(use-modules (gnu services desktop)) (define-os-macros-for mate-desktop)
(use-modules (gnu services desktop)) (define-os-macros-for sane)
(use-modules (gnu services desktop)) (define-os-macros-for seatd)
(use-modules (gnu services desktop)) (define-os-macros-for sugar-desktop)
(use-modules (gnu services desktop)) (define-os-macros-for udisks)
(use-modules (gnu services desktop)) (define-os-macros-for upower)
(use-modules (gnu services desktop)) (define-os-macros-for x11-socket-directory)
(use-modules (gnu services desktop)) (define-os-macros-for xfce-desktop)
(use-modules (gnu services dict)) (define-os-macros-for dicod)
(use-modules (gnu services dns)) (define-os-macros-for ddclient)
(use-modules (gnu services dns)) (define-os-macros-for dnsmasq)
(use-modules (gnu services dns)) (define-os-macros-for knot-resolver)
(use-modules (gnu services dns)) (define-os-macros-for knot)
(use-modules (gnu services docker)) (define-os-macros-for docker)
(use-modules (gnu services docker)) (define-os-macros-for singularity)
(use-modules (gnu services file-sharing)) (define-os-macros-for transmission-daemon)
(use-modules (gnu services games)) (define-os-macros-for joycond)
(use-modules (gnu services games)) (define-os-macros-for wesnothd)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-cleaner)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-confd)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-kvmd)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-luxid)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-metad)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-mond)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-noded)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-rapi)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-watcher)
(use-modules (gnu services ganeti)) (define-os-macros-for ganeti-wconfd)
(use-modules (gnu services getmail)) (define-os-macros-for getmail)
(use-modules (gnu services guix)) (define-os-macros-for guix-build-coordinator-agent)
(use-modules (gnu services guix)) (define-os-macros-for guix-build-coordinator-queue-builds)
(use-modules (gnu services guix)) (define-os-macros-for guix-build-coordinator)
(use-modules (gnu services guix)) (define-os-macros-for guix-data)
(use-modules (gnu services guix)) (define-os-macros-for nar-herder)
(use-modules (gnu services hurd)) (define-os-macros-for hurd-console)
(use-modules (gnu services hurd)) (define-os-macros-for hurd-getty)
(use-modules (gnu services kerberos)) (define-os-macros-for krb5)
(use-modules (gnu services kerberos)) (define-os-macros-for pam-krb5)
(use-modules (gnu services ldap)) (define-os-macros-for directory-server)
(use-modules (gnu services lightdm)) (define-os-macros-for lightdm)
(use-modules (gnu services linux)) (define-os-macros-for earlyoom)
(use-modules (gnu services linux)) (define-os-macros-for fstrim)
(use-modules (gnu services linux)) (define-os-macros-for kernel-module-loader)
(use-modules (gnu services linux)) (define-os-macros-for rasdaemon)
(use-modules (gnu services linux)) (define-os-macros-for zram-device)
(use-modules (gnu services lirc)) (define-os-macros-for lirc)
(use-modules (gnu services mail)) (define-os-macros-for dovecot)
(use-modules (gnu services mail)) (define-os-macros-for exim)
(use-modules (gnu services mail)) (define-os-macros-for imap4d)
(use-modules (gnu services mail)) (define-os-macros-for mail-aliases)
(use-modules (gnu services mail)) (define-os-macros-for opensmtpd)
(use-modules (gnu services mail)) (define-os-macros-for radicale)
(use-modules (gnu services mcron)) (define-os-macros-for mcron)
(use-modules (gnu services messaging)) (define-os-macros-for bitlbee)
(use-modules (gnu services messaging)) (define-os-macros-for prosody)
(use-modules (gnu services messaging)) (define-os-macros-for quassel)
(use-modules (gnu services monitoring)) (define-os-macros-for darkstat)
(use-modules (gnu services monitoring)) (define-os-macros-for prometheus-node-exporter)
(use-modules (gnu services monitoring)) (define-os-macros-for vnstat)
(use-modules (gnu services monitoring)) (define-os-macros-for zabbix-agent)
(use-modules (gnu services monitoring)) (define-os-macros-for zabbix-front-end)
(use-modules (gnu services monitoring)) (define-os-macros-for zabbix-server)
(use-modules (gnu services networking)) (define-os-macros-for block-facebook-hosts)
(use-modules (gnu services networking)) (define-os-macros-for connman)
(use-modules (gnu services networking)) (define-os-macros-for dhcp-client)
(use-modules (gnu services networking)) (define-os-macros-for dhcpd)
(use-modules (gnu services networking)) (define-os-macros-for hostapd)
(use-modules (gnu services networking)) (define-os-macros-for ipfs)
(use-modules (gnu services networking)) (define-os-macros-for iptables)
(use-modules (gnu services networking)) (define-os-macros-for keepalived)
(use-modules (gnu services networking)) (define-os-macros-for modem-manager)
(use-modules (gnu services networking)) (define-os-macros-for network-manager)
(use-modules (gnu services networking)) (define-os-macros-for nftables)
(use-modules (gnu services networking)) (define-os-macros-for ntp)
(use-modules (gnu services networking)) (define-os-macros-for opendht)
(use-modules (gnu services networking)) (define-os-macros-for openntpd)
(use-modules (gnu services networking)) (define-os-macros-for openvswitch)
(use-modules (gnu services networking)) (define-os-macros-for pagekite)
(use-modules (gnu services networking)) (define-os-macros-for simulated-wifi)
(use-modules (gnu services networking)) (define-os-macros-for tor)
(use-modules (gnu services networking)) (define-os-macros-for usb-modeswitch)
(use-modules (gnu services networking)) (define-os-macros-for wpa-supplicant)
(use-modules (gnu services networking)) (define-os-macros-for yggdrasil)
(use-modules (gnu services networking)) (define-os-macros-for inetd)
(use-modules (gnu services nfs)) (define-os-macros-for gss)
(use-modules (gnu services nfs)) (define-os-macros-for idmap)
(use-modules (gnu services nfs)) (define-os-macros-for nfs)
(use-modules (gnu services nfs)) (define-os-macros-for pipefs)
(use-modules (gnu services nfs)) (define-os-macros-for rpcbind)
(use-modules (gnu services nix)) (define-os-macros-for nix)
(use-modules (gnu services pam-mount)) (define-os-macros-for pam-mount)
(use-modules (gnu services pm)) (define-os-macros-for thermald)
(use-modules (gnu services pm)) (define-os-macros-for tlp)
(use-modules (gnu services rsync)) (define-os-macros-for rsync)
(use-modules (gnu services samba)) (define-os-macros-for samba)
(use-modules (gnu services samba)) (define-os-macros-for wsdd)
(use-modules (gnu services science)) (define-os-macros-for rshiny)
(use-modules (gnu services sddm)) (define-os-macros-for sddm)
(use-modules (gnu services security-token)) (define-os-macros-for pcscd)
(use-modules (gnu services security)) (define-os-macros-for fail2ban)
(use-modules (gnu services shepherd)) (define-os-macros-for shepherd-root)
(use-modules (gnu services shepherd)) (define-os-macros-for user-processes)
(use-modules (gnu services shepherd)) (define-os-macros-for shepherd)
(use-modules (gnu services sound)) (define-os-macros-for alsa)
(use-modules (gnu services sound)) (define-os-macros-for ladspa)
(use-modules (gnu services sound)) (define-os-macros-for pulseaudio)
(use-modules (gnu services spice)) (define-os-macros-for spice-vdagent)
(use-modules (gnu services ssh)) (define-os-macros-for autossh)
(use-modules (gnu services ssh)) (define-os-macros-for dropbear)
(use-modules (gnu services ssh)) (define-os-macros-for lsh)
(use-modules (gnu services ssh)) (define-os-macros-for openssh)
(use-modules (gnu services ssh)) (define-os-macros-for webssh)
(use-modules (gnu services syncthing)) (define-os-macros-for syncthing)
(use-modules (gnu services sysctl)) (define-os-macros-for sysctl)
(use-modules (gnu services telephony)) (define-os-macros-for jami)
(use-modules (gnu services telephony)) (define-os-macros-for mumble-server)
(use-modules (gnu services version-control)) (define-os-macros-for git-daemon)
(use-modules (gnu services version-control)) (define-os-macros-for gitile)
(use-modules (gnu services version-control)) (define-os-macros-for gitolite)
(use-modules (gnu services virtualization)) (define-os-macros-for hurd-vm)
(use-modules (gnu services virtualization)) (define-os-macros-for libvirt)
(use-modules (gnu services virtualization)) (define-os-macros-for qemu-binfmt)
(use-modules (gnu services virtualization)) (define-os-macros-for qemu-guest-agent)
(use-modules (gnu services virtualization)) (define-os-macros-for secret)
(use-modules (gnu services virtualization)) (define-os-macros-for virtlog)
(use-modules (gnu services vnc)) (define-os-macros-for xvnc)
(use-modules (gnu services vpn)) (define-os-macros-for openvpn-client)
(use-modules (gnu services vpn)) (define-os-macros-for openvpn-server)
(use-modules (gnu services vpn)) (define-os-macros-for strongswan)
(use-modules (gnu services vpn)) (define-os-macros-for wireguard)
(use-modules (gnu services vpn)) (define-os-macros-for bitmask)
(use-modules (gnu services web)) (define-os-macros-for agate)
(use-modules (gnu services web)) (define-os-macros-for anonip)
(use-modules (gnu services web)) (define-os-macros-for fcgiwrap)
(use-modules (gnu services web)) (define-os-macros-for gmnisrv)
(use-modules (gnu services web)) (define-os-macros-for hpcguix-web)
(use-modules (gnu services web)) (define-os-macros-for httpd)
(use-modules (gnu services web)) (define-os-macros-for mumi)
(use-modules (gnu services web)) (define-os-macros-for nginx)
(use-modules (gnu services web)) (define-os-macros-for patchwork)
(use-modules (gnu services web)) (define-os-macros-for php-fpm)
(use-modules (gnu services web)) (define-os-macros-for tailon)
(use-modules (gnu services web)) (define-os-macros-for varnish)
(use-modules (gnu services xorg)) (define-os-macros-for dconf)
(use-modules (gnu services xorg)) (define-os-macros-for gdm)
(use-modules (gnu services xorg)) (define-os-macros-for localed)
(use-modules (gnu services xorg)) (define-os-macros-for screen-locker)
(use-modules (gnu services xorg)) (define-os-macros-for slim)
(use-modules (gnu services xorg)) (define-os-macros-for xorg-server)
(use-modules (gnu system install)) (define-os-macros-for configuration-template)
(use-modules (gnu system install)) (define-os-macros-for cow-store)
(use-modules (gnu system install)) (define-os-macros-for documentation)
(use-modules (gnu system install)) (define-os-macros-for uvesafb)
(use-modules (gnu system linux-container)) (define-os-macros-for dummy-networking)
(use-modules (gnu system mapped-devices)) (define-os-macros-for device-mapping)
(use-modules (gnu system pam)) (define-os-macros-for pam-root)
(use-modules (gnu system pam)) (define-os-macros-for session-environment)
(use-modules (gnu system shadow)) (define-os-macros-for account)
(use-modules (gnu tests)) (define-os-macros-for marionette)
(use-modules (guix scripts publish)) (define-os-macros-for publish)


Edouard Klein <edk@beaver-labs.com> writes:

> Dear Guixers,
>
> For my clients and my own use, I use a layer on top of operating-system
> declarations in which I create functions that take an os as a first
> argument, and return an os.
>
> With the help of the handy -> macro, I can chain them, thus allowing an
> easy stacking of multiple "roles" or "functionalities" on the same
> machine, and an easy transfer of said functionality to another machine:
> just cut and paste one line from one declaration to the other.
>
> I have written a tutorial for my clients here:
>
> https://guix-hosting.com/docs/tuto/getting-started.html
>
> that gives an example of what an os configuration may look like:
>
> #+begin_src scheme
> (->
>  (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
>  (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
>  (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
>  (add-services my-db))
> #+end_src
>
> The code of the function is on my channel:
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm
>
> After a few months of experience, and positive feedback from my clients,
> my question to you guys is: would you be interested in mainlining this,
> or should I keep my development efforts separate in my channel ?
>
> I do think this API is easier than manipulating services, and although
> extendable services are awesome and a very nifty piece of engineering,
> they require quite a good knowledge of scheme and take a while to be
> used to, while this new API, while way less powerful, lowers the barrier
> to entry for newcomers.
>
> They are an easy way to maintain a declarative whole operating system
> configuration, with a syntax similar enough to docker and ansible that
> sysadmins familiar with it can quickly get up and running, thus exposing
> more people to Guix.
>
> What do you think ?
>
> Cheers,
>
> Edouard.

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Freitag, dem 24.11.2023 um 22:43 +0100 schrieb Edouard Klein:
> Dear Guixers,
> 
> Here is a quick status update on my proposition to expose composable
> functions to change operating-system declarations.
> 
> Thank you all for the feedback you gave me :) It's very nice to not
> be talking in the void.
> 
> After Liliana opined that these functions should not put too much
> burden on the maintainers, a position with which I wholeheartedly
> agree, here is what I came up with:
> 
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/functional-services.scm
> 
> (file copied at the end of this email to make the list self-content
> instead of relying on external services)
> 
> That's a macro-writing macro called "define-os-macros-for" that, for
> any service (for example nginx), defines five forms. They all
> evaluate to a modified os:
> 
> (.nginx os)  ;; Add nginx in its default configuration
> (.nginx os toto titi tutu... )  ;; Add nginx, with toto... given as
> arguments to nginx-configuration
> (+nginx os toto titi tutu...)  ;; Extend an existing nginx service,
> giving
> toto... as arguments to simple-service
> (~nginx os toto...)  ;; Edit an existing nginx service, passing
> toto... as
> arguments to modify-service
> (-nginx os)  ;; Removes nginx
> 
> You can see all of them used in:
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm?ref_type=heads
> 
> e.g. the mkdir-p function is a (+activation ...),
> all web server functions use (+nginx...),
> mumble is (.mumble... ),
> os/git uses (~openssh... ).
> 
> I've use (-foo) forms while testing, but not in production. The
> others are currently running in prod on guix-hosting.com and the-
> dam.org.
The naming is a little confusing and imho not clearly helpful.  For
example, why (.nginx os) instead of the triple

(service+ OS SERVICE [CONF])
(service- OS SERVICE)
(modify-service OS SERVICE UPDATE)

Of course, you could also define triples (add-SERVICE OS [CONF]), 
(remove-SERVICE OS) and (modify-SERVICE OS UPDATE), which would fall in
line with the macro approach, but keep names a little more readable. 
(If you want to use your names, you can rename them in #:use-modules).

> -----------------functional-services.scm----------------
> (define-module (beaver functional-services)
>    #:use-module (gnu system)
>    #:export (*-append))
> 
> (define (syntax->string s)
>   "Shorthand to convert a piece of syntax to a string"
>   (symbol->string (syntax->datum s)))
Or (compose symbol->string syntax->datum).

> (define (*->string s)
>   "Convert into a string any symbol-like type we may encounter in a
> macro"
>   (cond
>    [(string? s) s]
>    [(symbol? s) (symbol->string s)]
>    [else (syntax->string s)]))  ;; assume syntax
There should not be a need to define this.  Try to use exact types.

> (define* (*-append #:rest args)
>   "Return a symbol which is a concatenation of the given symbol-like
> args"
>   (string->symbol (apply string-append (map *->string args))))
As above.

> (define-syntax define-os-macros-for
>   ;; This macro, called like e.g. (define-os-macros-for foo) will
> define four
>   ;; forms, .foo, +foo, ~foo and -foo
>   ;;
>   ;; These forms take an operating-system as their first argument,
> and
>   ;; evaluate to an operating-system, which allows one to compose
> them
>   ;; (e.g. (foo (bar os))).
>   (lambda (x)
>     (syntax-case x ()
>       [(_ foo)
>        (with-syntax
>         ([dot-foo           (datum->syntax x (*-append "." #'foo))]
>          [plus-foo          (datum->syntax x (*-append "+" #'foo))]
>          [tilde-foo         (datum->syntax x (*-append "~" #'foo))]
>          [minus-foo         (datum->syntax x (*-append "-" #'foo))]
>          [foo-service-type  (datum->syntax x (*-append #'foo "-
> service-type"))]
>          [foo-configuration (datum->syntax x (*-append #'foo "-
> configuration"))])
You're repeating datum->syntax here a bit too much for my liking.

This patch is also missing the most important part of beaverlabs to
make this work: the -> threading macro.

Cheers

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> (service+ OS SERVICE [CONF])
> (service- OS SERVICE)
> (modify-service OS SERVICE UPDATE)


what would the benefit of generating multiple macros for each service compared to the above functional API (with 3-4 elements altogether)?

i could be missing something here, but it seems to be precious little to me while it costs some extra complexity.

if i were to add a syntactic abstraction for this, i'd generate a full DSL in the form of a (modify-operating-system OS [fictional DSL to describe desired changes]).

but i don't think the extra complexity justifies any macrology here.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
By the time a man realises that his father was right, he has a son who thinks he’s wrong.

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Michal Atlas]

Hello guix,

I'll chime in because I've been playing around with this kind of thing 
in my channel [1]
for a bit now and perhaps some of the ideas will be deemed useful.

> (service+ OS SERVICE [CONF])
> (service- OS SERVICE)
> (modify-service OS SERVICE UPDATE)


I found that defining a functional API like this is about as convenient 
as the original version, more examples: [2].

Another small change I tried was to not have -> be a macro but rather a 
simple fold.

I took what the dot-macro form expands to and made it a curried function 
called services
(but remove and modify could be equivalently defined),
which takes a service and then returns a lambda that takes an OS and 
modifies it
just as the beaver-labs macros do with that extra service.


(define ((services . new) os)
   (operating-system
     (inherit os)
     (user-services
       (append new (operating-system-user-services os)))))

(define (-> system . services)
   (fold (cut <> <>) system services))


(-> os (services (service openssh-service-type)) ...) ; yields os with 
added openssh

((services (service openssh-service-type)) os) ; equivalent


The one macro that is indispensable for terse configurations is still 
some sort of service+ (I use &s as the name),
which appends -service-type, surrounds the body in a -configuration [3],
and in my case calls `services` on it for good measure to get the os 
modifying lambda straight from the service+ clause.


(-> os
   (&s openssh)
   (&s guix-publish
     (advertise? #t))
   ...)


This ends up save for 2 extra characters being syntactically almost 
identical to the original version,
just passes around lambdas instead of getting manipulated by nested macros.

Them being functions they're now easier to work with with in more 
traditional Scheme ways,
for example compose works, and we can wrap them in other functions that 
for example apply them only in some circumstances.
Or combine transformations into new ones. The definitions for which end 
up being remarkably short and generalizable.


(-> os

   (compose (os/hostname "the-dam.org") (&s gpm)) ; freely composable 
with other functions of the same sort

   (%services-with-arguments ...) ; indifferent to being composed on the 
spot even in the middle of threading

   (if-host "hydra" (&s openssh)) ; only adds openssh if the host is 
named hydra, simple function

            ; since the second argument is just a function that may or 
may not be applied by the new lambda

   ((mapped-file-systems <some-devices> ...)

     <more-devices-that-depend-on-former> ...))) ; not threading by 
macro allows more complex structures without confusion



Then again none of this has had proper field testing, the fact that it 
has less gotchas is just a personal opinion.


Cheers, and all the best


[1]: 
https://git.sr.ht/~michal_atlas/guix-channel/tree/a31b68b46da60002383e2793eba88b99fc5c2382/item/atlas/combinators.scm 
(modified from the original beaver-labs version).

[2]: 
https://git.sr.ht/~michal_atlas/dotfiles/tree/8c78f53139ae176ff0a4cab82ad0fb64bce6989b/item/atlas/config/system/services.scm#L56 


[3]: 
https://git.sr.ht/~michal_atlas/guix-channel/tree/master/item/atlas/utils/services.scm#L53 

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Thank you Liliana and Attila for the swift and actionable feedback :)

Below is a revised proposition.

Here is a minimal working example of an os declaration:
------------------mwe.scm---------------
(use-modules
 (beaver system)
 (beaver functional-services)
 (gnu packages version-control)
 (gnu services web)
 (gnu services telephony)
 (gnu services ssh)
 (gnu services base)
 (guix gexp))

(-> (minimal-ovh "osef")
    (instantiate nginx)
    (instantiate mumble-server
                 (welcome-text "coucou")
                 (port 64738))
    (extend openssh `(("alice" ,(local-file "/home/edouard/.ssh/id_rsa.pub"))))
    (modify openssh
            (password-authentication? #f)
            (allow-empty-passwords? #t))
    (remove guix))
-------------------------------------------------------

To see the value of this syntactic sugar, try to replicate this MWE with
the standard syntax. It's not horrendous, but it *is* off-putting to
many newcomers to git, whereas this sugary piece is more readable for
them (sample size of 1, p=0.00000005).

Here is the revised functional-services.scm, not yet commited and
pushed, and only lightly tested in local containers, but not in
production:

Advice and comments welcome :)


------------functional-services.scm--------------


(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:use-module (gnu services)
   #:export (instantiate extend modify remove))

(define syntax->string (compose symbol->string syntax->datum))

(define (service-configuration stx service)
  "Return the syntax one can use to refer to xxx-configuration for the given
service"
  (datum->syntax stx (string->symbol
                      (string-append
                       (syntax->string service)
                       "-configuration"))))

(define (service-type stx service)
  "Return the syntax one can use to refer to xxx-service-type for the given
service"
  (datum->syntax stx (string->symbol
                      (string-append
                       (syntax->string service)
                       "-service-type"))))

(define-syntax instantiate
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)  ;; It is wrapped in a lamba to make sure os is
               ;; evaluated once only. It it wasn't in a labmda, whatever
               ;; form os is in the calling code would be repeated
               ;; multiple times, and so if the form was e.g. (some-func
               ;; os), then some-func would be called multiple times,
               ;; which may not be desirable.
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (service service-type)
                   (operating-system-user-services x)))))
             os)))]
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)]
         [service-configuration (service-configuration stx #'service-name)])
        #'(begin
            ((lambda (x)  ;; Wrapping in a lambda for the same reasons as above
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (service service-type
                                 (service-configuration forms ...))
                   (operating-system-user-services x)))))
             os)))])))

(define-syntax extend
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (simple-service (format  #f "A ~a extension" (syntax->string #'service-name))
                                   service-type
                                   forms ...)
                   (operating-system-user-services x)))))
             os)))])))

(define-syntax modify
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)]
         [service-configuration (service-configuration stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (modify-services (operating-system-user-services x)
                    (service-type
                     config =>
                     (service-configuration
                      (inherit config)
                      forms ...))))))
             os)))])))

(define-syntax remove
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (modify-services (operating-system-user-services x)
                    (delete service-type)))))
             os)))])))


-------------------------


Attila Lendvai <attila@lendvai.name> writes:

>> (service+ OS SERVICE [CONF])
>> (service- OS SERVICE)
>> (modify-service OS SERVICE UPDATE)
>
>
> what would the benefit of generating multiple macros for each service compared to the above functional API (with 3-4 elements altogether)?
>
> i could be missing something here, but it seems to be precious little to me while it costs some extra complexity.
>
> if i were to add a syntactic abstraction for this, i'd generate a full DSL in the form of a (modify-operating-system OS [fictional DSL to describe desired changes]).
>
> but i don't think the extra complexity justifies any macrology here.
>
> --
> • attila lendvai
> • PGP: 963F 5D5F 45C7 DFCD 0A39

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Sonntag, dem 26.11.2023 um 17:49 +0100 schrieb Edouard Klein:
> Thank you Liliana and Attila for the swift and actionable feedback :)
> 
> Below is a revised proposition.
> 
> Here is a minimal working example of an os declaration:
> ------------------mwe.scm---------------
> (use-modules
>  (beaver system)
>  (beaver functional-services)
>  (gnu packages version-control)
>  (gnu services web)
>  (gnu services telephony)
>  (gnu services ssh)
>  (gnu services base)
>  (guix gexp))
> 
> (-> (minimal-ovh "osef")
>     (instantiate nginx)
I do wish you spelled out service.  Also, instantiate takes as much
characters to type as add-service.
>     (instantiate mumble-server
>                  (welcome-text "coucou")
>                  (port 64738))
>     (extend openssh `(("alice" ,(local-file
> "/home/edouard/.ssh/id_rsa.pub"))))
>     (modify openssh
>             (password-authentication? #f)
>             (allow-empty-passwords? #t))
>     (remove guix))
> -------------------------------------------------------
> 
> To see the value of this syntactic sugar, try to replicate this MWE
> with the standard syntax. It's not horrendous, but it *is* off-
> putting to many newcomers to git, whereas this sugary piece is more
> readable for them (sample size of 1, p=0.00000005).
Well, that'd be 
  (let ((base (minimal-ovh "osef")))
    (operating-system
      (inherit base)
      (services
        (cons*
         (service nginx-service-type)
         (service mumble-service-type
                  (mumble-configuration
                   (welcome-text "couocu")
                   (port 64738)))
         (service openssh-service-type
                  (openssh-configuation
                   (password-authentication? #f)
                   (allow-empty-passwords? #t)
                   (authorized-keys <your-keys)))
         (operating-system-user-services base))
or
  (-> (minimal-ovh "osef")
      (lambda (base) …))

On that note, we also have extend-openssh-authorized-keys for the use
with modify-services.

> Here is the revised functional-services.scm, not yet commited and
> pushed, and only lightly tested in local containers, but not in
> production:
> 
> Advice and comments welcome :)
> 
> 
> ------------functional-services.scm--------------
> 
> 
> (define-module (beaver functional-services)
>    #:use-module (gnu system)
>    #:use-module (gnu services)
>    #:export (instantiate extend modify remove))
> 
> (define syntax->string (compose symbol->string syntax->datum))
> 
> (define (service-configuration stx service)
>   "Return the syntax one can use to refer to xxx-configuration for
> the given
> service"
>   (datum->syntax stx (string->symbol
>                       (string-append
>                        (syntax->string service)
>                        "-configuration"))))
> 
> (define (service-type stx service)
>   "Return the syntax one can use to refer to xxx-service-type for the
> given
> service"
>   (datum->syntax stx (string->symbol
>                       (string-append
>                        (syntax->string service)
>                        "-service-type"))))
> 
> (define-syntax instantiate
>   (lambda (stx)
>     (syntax-case stx ()
>       [(_ os service-name)
>        (with-syntax
>         ([service-type (service-type stx #'service-name)])
>         #'(begin
>             ((lambda (x)  ;; It is wrapped in a lamba to make sure os
> is
>                ;; evaluated once only. It it wasn't in a labmda,
> whatever
>                ;; form os is in the calling code would be repeated
>                ;; multiple times, and so if the form was e.g. (some-
> func
>                ;; os), then some-func would be called multiple times,
>                ;; which may not be desirable.
Isn't it also wrapped in a lambda, because -> is a threading macro that
takes functions rather than syntax?


Cheers

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
>>     (instantiate nginx)
> I do wish you spelled out service.  Also, instantiate takes as much
> characters to type as add-service.
>

Done, see below. I was worried about the paronymy between add-service
and add-services which already exists. I defer to you and your
experience on this, naming things is hard.

>> To see the value of this syntactic sugar, try to replicate this MWE
>> with the standard syntax. It's not horrendous, but it *is* off-
>> putting to many newcomers to git, whereas this sugary piece is more
>> readable for them (sample size of 1, p=0.00000005).
> Well, that'd be
> ...

I tried:

(let ((base (minimal-ovh "osef")))
  (operating-system
    (inherit base)
    (services
      (cons*
       (service nginx-service-type)
       (service mumble-server-service-type
                (mumble-server-configuration
                 (welcome-text "couocu")
                 (port 64738)))
       (service openssh-service-type
                (openssh-configuration
                 (password-authentication? #f)
                 (allow-empty-passwords? #t)
                 (authorized-keys `(("alice" ,(local-file "/home/edouard/.ssh/id_rsa.pub"))))))
       (operating-system-user-services base)))))


I admit that this is as readable as the sweet version, but:

- Openssh is already defined in  (minimal-ovh "osef"), so the build
  fails with: 'guix system: error: service 'ssh-daemon' provided more
  than once'
- you forgot  the removal of guix-service, which admitedly is not used much in practice
  anyway

The problem with those two is that they break the nice structure of just
adding services, and now one has to first remove an element from
(operating-system-user-services base), then edit one of the element of
the resulting list, then add to elements to it. It is probably possible
to write it in a readable way, maybe less so than the sweet version, but
not so much as to justify adding the new macros to guix.

However the readability is not the main selling point, the writeability
is. Please do not discount how hard it is to write that kind of stuff
when scheme's not your main language. It is possible people here forgot
how hard this is for beginners, especially those like me whose brain is
deformed from years using algol-derived syntaxes.

I think we are losing a lot of mindshare because of the hard step of
learning both guile and guix, and the kind of syntactic sugar I
suggest may help bridge the gap long enough for newcomers to ease into
the syntax, after they get a taste of guix' capabilities.

Another experiment: with the sweet syntax, you can easily extend
services, without having to define a -record-type, etc. Just define a
function. I can write more at length about that if you want.

> On that note, we also have extend-openssh-authorized-keys for the use
> with modify-services.
>
I see it now in the source, but I was unaware of its existence.




Thanks for the swift again review :)

Cheers,

Edouard.


-------------mwe.scm

(use-modules
 (beaver system)
 (beaver functional-services)
 (gnu packages version-control)
 (gnu services web)
 (gnu services telephony)
 (gnu services ssh)
 (gnu services base)
 (guix gexp))

(-> (minimal-ovh "osef")
    (add-service nginx)
    (add-service mumble-server
                 (welcome-text "coucou")
                 (port 64738))
    (extend-service openssh `(("alice" ,(local-file "/home/edouard/.ssh/id_rsa.pub"))))
    (modify-service openssh
                    (password-authentication? #f)
                    (allow-empty-passwords? #t))
    (remove-service guix))

----------------------functional-services.scm

(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:use-module (gnu services)
   #:export (add-service extend-service modify-service remove-service))

(define syntax->string (compose symbol->string syntax->datum))

(define (service-configuration stx service)
  "Return the syntax one can use to refer to xxx-configuration for the given
service"
  (datum->syntax stx (string->symbol
                      (string-append
                       (syntax->string service)
                       "-configuration"))))

(define (service-type stx service)
  "Return the syntax one can use to refer to xxx-service-type for the given
service"
  (datum->syntax stx (string->symbol
                      (string-append
                       (syntax->string service)
                       "-service-type"))))

(define-syntax add-service
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)  ;; It is wrapped in a lamba to make sure os is
               ;; evaluated once only. It it wasn't in a labmda, whatever
               ;; form os is in the calling code would be repeated
               ;; multiple times, and so if the form was e.g. (some-func
               ;; os), then some-func would be called multiple times,
               ;; which may not be desirable.
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (service service-type)
                   (operating-system-user-services x)))))
             os)))]
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)]
         [service-configuration (service-configuration stx #'service-name)])
        #'(begin
            ((lambda (x)  ;; Wrapping in a lambda for the same reasons as above
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (service service-type
                                 (service-configuration forms ...))
                   (operating-system-user-services x)))))
             os)))])))

(define-syntax extend-service
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (cons
                   (simple-service (format  #f "A ~a extension" (syntax->string #'service-name))
                                   service-type
                                   forms ...)
                   (operating-system-user-services x)))))
             os)))])))

(define-syntax modify-service
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)]
         [service-configuration (service-configuration stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (modify-services (operating-system-user-services x)
                    (service-type
                     config =>
                     (service-configuration
                      (inherit config)
                      forms ...))))))
             os)))])))

(define-syntax remove-service
  (lambda (stx)
    (syntax-case stx ()
      [(_ os service-name forms ...)
       (with-syntax
        ([service-type (service-type stx #'service-name)])
        #'(begin
            ((lambda (x)
               (operating-system
                 (inherit x)
                 (services
                  (modify-services (operating-system-user-services x)
                    (delete service-type)))))
             os)))])))

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Sonntag, dem 26.11.2023 um 21:46 +0100 schrieb Edouard Klein:
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> > >     (instantiate nginx)
> > I do wish you spelled out service.  Also, instantiate takes as much
> > characters to type as add-service.
> > 
> 
> Done, see below. I was worried about the paronymy between add-service
> and add-services which already exists. I defer to you and your
> experience on this, naming things is hard.
That's not that much of a deal in scheme, see let and let* for example.

> > > To see the value of this syntactic sugar, try to replicate this
> > > MWE
> > > with the standard syntax. It's not horrendous, but it *is* off-
> > > putting to many newcomers to git, whereas this sugary piece is
> > > more
> > > readable for them (sample size of 1, p=0.00000005).
> > Well, that'd be
> > ...
> 
> I tried:
> 
> (let ((base (minimal-ovh "osef")))
>   (operating-system
>     (inherit base)
>     (services
>       (cons*
>        (service nginx-service-type)
>        (service mumble-server-service-type
>                 (mumble-server-configuration
>                  (welcome-text "couocu")
>                  (port 64738)))
>        (service openssh-service-type
>                 (openssh-configuration
>                  (password-authentication? #f)
>                  (allow-empty-passwords? #t)
>                  (authorized-keys `(("alice" ,(local-file
> "/home/edouard/.ssh/id_rsa.pub"))))))
>        (operating-system-user-services base)))))
> 
> 
> I admit that this is as readable as the sweet version, but:
> 
> - Openssh is already defined in  (minimal-ovh "osef"), so the build
>   fails with: 'guix system: error: service 'ssh-daemon' provided more
>   than once'
> - you forgot the removal of guix-service, which admitedly is not used
> much in practice
>   anyway
Nice catch.

> The problem with those two is that they break the nice structure of
> just adding services, and now one has to first remove an element from
> (operating-system-user-services base), then edit one of the element
> of the resulting list, then add to elements to it. It is probably
> possible to write it in a readable way, maybe less so than the sweet
> version, but not so much as to justify adding the new macros to guix.
Fair enough, but you could still implement that as an (extended)
modify-services.  We haven't reached the level of complexity where we'd
touch multiple fields, which is when operating-system really becomes a
pain to work with.  (The ssh-user from beaverlabs is one such example,
that IIRC is also used extensively on the-dam.)

> However the readability is not the main selling point, the
> writeability is. Please do not discount how hard it is to write that
> kind of stuff when scheme's not your main language. It is possible
> people here forgot how hard this is for beginners, especially those
> like me whose brain is deformed from years using algol-derived
> syntaxes.
> 
> I think we are losing a lot of mindshare because of the hard step of
> learning both guile and guix, and the kind of syntactic sugar I
> suggest may help bridge the gap long enough for newcomers to ease
> into the syntax, after they get a taste of guix' capabilities.
Fair point, and I'm not really opposed to making things more
readable/writable.  The problem comes with groking the additional
syntax.  It is not something the manuals would prepare you for and
probably also encounters weird corner cases like services using
something else than SERVICE-configuration for its configuration data.

> Another experiment: with the sweet syntax, you can easily extend
> services, without having to define a -record-type, etc. Just define a
> function. I can write more at length about that if you want.
That's again just simple-service doing its job tho :)

In essence, what I'm trying to get here is something that'd let us
implement the more complicated part of OS config fiddling with few
lines of code.  I think hyper-focusing on syntax to make services
"nicer" might be the wrong approach here: You could greatly reduce the
complexity by making them procedures instead of syntax and still keep
most of the configuration readable to a great extent.

Maybe we should start with making modify-services more powerful to also
cover the "adding services" case: modify-inputs already does that, so
it's weird that modify-services doesn't.  We should also think on how
to best wrap this in a lambda then; defining a new syntax for most
operations doesn't seem all that useful and scalable to me.

Cheers
> 

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> lines of code. I think hyper-focusing on syntax to make services
> "nicer" might be the wrong approach here: You could greatly reduce the
> complexity by making them procedures instead of syntax and still keep
> most of the configuration readable to a great extent.


i agree. in my experience, it's good practice in general to try to make a functional API as convenient as possible, and if that is still too verbose or cumbersome, only then add a thin layer of syntactic abstractions that expand to code that uses this functional API.

such code is much easier to work with, especially when debugging stuff interactively (i.e. it's possible to recompile a function that will then affect every place in the code where the macrology is used).

the downside of generating countless macros is that they won't show up in backtraces, and they cannot be found when grep'ping the codebase, and as such make the codebase much less approachable. the size of their expansion can also become an issue if they are used often enough.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“They muddy the water, to make it seem deep.”
	— Friedrich Nietzsche (1844–1900)

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Attila,

On Wed, Nov 29 2023, Attila Lendvai wrote:

> i agree. in my experience, it's good practice in general to try to
> make a functional API as convenient as possible, and if that is still
> too verbose or cumbersome, only then add a thin layer of syntactic
> abstractions that expand to code that uses this functional API.
>
> [...]
>
> the downside of generating countless macros is that they won't show up
> in backtraces, and they cannot be found when grep'ping the codebase,
> and as such make the codebase much less approachable.

Reading your words really helped me feel that I'm not alone. You more or
less summarized my feelings about the Guix codebase, which I have been
reading now for over a year. Guile's syntax features make the code more
symbolic and less approachable to newcomers.

Of course, syntax features are helpful and necessary in some places.

Any large project like GNU Guix, however, must continue to engage a
cadre of skilled maintainers. Between Guile records and syntax features,
code can get hairy pretty fast. Some files seem to have received more
hurried attention recently. In the long run, it's not good for Guix to
make code hard to read.

Please forgive me, everyone, for speaking up. I did not follow the
discussion here at all. Attila's commentary merely resonated with me as
I tried to deal with my own shortcomings in contributing to the Guix
codebase.

Eventually, I hope to make meaningful suggestions to the bootloader
interaction with system profiles, and to the automatic "wrapping" of
executables, which is currently done by hand.

The changes Edouard and Michal may well be very valuable. Please do not
allow my comments to reflect on their well-meant ideas and hard work, or
on any other contributions to the software we love.

Thank you all for contributing to GNU Guix!

Kind regards
Felix

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> > the downside of generating countless macros is that they won't show up
> > in backtraces, and they cannot be found when grep'ping the codebase,
> > and as such make the codebase much less approachable.
> 
> 
> Reading your words really helped me feel that I'm not alone. You more or
> less summarized my feelings about the Guix codebase, which I have been
> reading now for over a year. Guile's syntax features make the code more
> symbolic and less approachable to newcomers.


just FTR, i don't think that the guix codebase is too bad in this regard.

here i just wanted to remind people to the not so obvious cost of syntactic abstractions that should be accounted for when making decisions. introducing macros that generate macros is rarely justified.

in general, it's *very* valuable when stuff can be grep'ped -- and not only for newcomers. after enough time has passed i can feel like a newcomer to my own codebase... :) modulo the protocols that i keep while writing code.

e.g. define.*whatever is a grep that i regularly employ. the pattern here is that although there are countless ways to define countless different stuff, there's a convention to stick to the define.*[name] pattern.

intuitive, unique (i.e. grep'pable) names are also key to facilitate code approachability, especially for abstractions that are scattered around in many source files. in some situations i even copy-paste names of abstractions into comments for any future grep'ping to pick it up.

a negative example for this in the guix codebase is the use of 'service' to describe two rather different abstractions: a component of an OS vs. a deamon process run by shepherd. for a while it caused me quite some confusion.

</rant>

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The State is that organization in society which attempts to maintain a monopoly of the use of force and violence in a given territorial area; in particular, it is the only organization in society that obtains its revenue not by voluntary contribution or payment for services rendered but by coercion.”
	— Murray N. Rothbard (1926–1995), 'Anatomy of the State' (1974)

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Michal Atlas]

Hello,

> just FTR, i don't think that the guix codebase is too bad in this regard.
It's not bad for the slightly initiated, which does still take time, and I think that front would perhaps benefit from some attention.


My main take away from this was that what I liked mostly about the BeaverLabs syntax was how short and simple minimalist systems became and that larger intentions could be represented by short invocations. The thing is services already allow that. There's os/git a beautiful function in BeaverLabs, which calls a set of other beaverlabs functions but it could equivalently be a service which extends other services. Perhaps this implies that what is wished for is a set of convenience services that combine sets of higher-level features, and making the operating-system record itself friendlier to specifying very little.


I did mention observations about the functional approach here, but after a while of using it, I myself in the end failed to see its merits apart from the slightly shorter service declaration which I personally recovered by a trivial
(&s #NAME# #BODY# ...) => (service #NAME#-service-type (#NAME#-configuration #BODY# ...))
replacement, without the need for all the extra effort.


I agree with the points made by others about grepability and approachability. Perhaps though BeaverLabs service syntax could deserve a mention as an alternative syntax somewhere in the cookbook or as an appendix in the manual/blogpost to show how malleable guix is.

Cheers

Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

On Thu, Nov 30 2023, Attila Lendvai wrote:

> the use of 'service' to describe two rather different abstractions: a
> component of an OS vs. a deamon process run by shepherd.

Indeed, the use of 'service' in much of Guix appears to be a grand
misnomer. It probably occurred because the meaning expanded over time.

It's like we are looking back in time at the Big Bang. Our "services"
are the microwave echoes of Guix's initial, creative spark!

Please consider a recent, helpful reply to help-guix. [1] Carlo
mentioned the term "service" eleven times, but none of them referred to
what I believe most readers of this message would call a service in
other contexts. What's a newbie on help-guix to think?

Should Guix services instead be called "features"?

Those "features" are central to any operating system definition. Other
choices like "provider" may not fully capture our collective uses
throughout the code and the documentation. I am especially thinking
about 'modify-features' and '%base-features'.

Kind regards
Felix

[1] https://lists.gnu.org/archive/html/help-guix/2024-01/msg00213.html

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Liliana Marie Prikler]

Am Donnerstag, dem 01.02.2024 um 05:29 -0800 schrieb Felix Lechner:
> On Thu, Nov 30 2023, Attila Lendvai wrote:
> 
> > the use of 'service' to describe two rather different abstractions:
> > a component of an OS vs. a deamon process run by shepherd.
> 
> Indeed, the use of 'service' in much of Guix appears to be a grand
> misnomer. It probably occurred because the meaning expanded over
> time.
I'm pretty sure that "etc-service-type" was not something that expanded
the meaning of "service" over time, but rather something that had been
there for most of the time already.  And if you think about it,
symlinking stuff to /etc *is* a service.

> It's like we are looking back in time at the Big Bang. Our "services"
> are the microwave echoes of Guix's initial, creative spark!
> 
> Please consider a recent, helpful reply to help-guix. [1] Carlo
> mentioned the term "service" eleven times, but none of them referred
> to what I believe most readers of this message would call a service
> in other contexts. What's a newbie on help-guix to think?
You might be experiencing an effect where repeating a word often enough
makes it appear meaningless, but not only is the reply helpful, it also
highlights the exact point you're criticizing: Typically, for stuff
like configuring clamav (which I think many would consider a service),
you'd have a dedicated clamav-service-type, which of course itself
extends more low-level services.  The mere fact that we haven't yet
gotten around to implement that doesn't mean that our terminology is
somehow off, it just means that there's something missing that
something is missing.

> Should Guix services instead be called "features"?
> 
> Those "features" are central to any operating system definition.
> Other choices like "provider" may not fully capture our collective
> uses throughout the code and the documentation. I am especially
> thinking about 'modify-features' and '%base-features'.
I think it's better to do descriptive linguistics over prescriptive
linguistics.  The use of `guix shell' prevails over `guix environment'
not because one of them uses a "more correct" term, but rather because
one of them provides a nicer interface that requires less typing (among
other benefits like reduced headaches, etc.)

As for the term features, that's a word that can mean literally
anything in between and around packages, services, etc.  Typically,
packages or services implement features, for instance evolution
implements "reading mail" and "writing mail", whereas (a) gdm(-service)
implements "logging into your account".  I personally don't think your
proposed relabeling would be worthwhile, especially if we have more
important features (like syntactic sugar) to discuss.

Cheers

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Attila Lendvai]

> there for most of the time already. And if you think about it,
> symlinking stuff to /etc is a service.

i've arrived to guix after 3+ decades of programming, most of that in opensource environments, unix-like OS'es, and more than a decade using linux as my primary OS and lisp as my goto language.

it could be me, of course, but it took me months of tinkering until i understood the guix service vs shepherd service nomenclature. and i still need to focus when i'm dealing with foo-service-type and shepherd services at the same time.

this nomenclature was an obstacle to understanding, because the naming suggests something that was misleading me.

for an average unix user a service is a process that is running in the backgroud, doing stuff mostly without any user interaction. you can try to argue this away, but i'm afraid that this is the state of things.

and if you care whether your words (code) is communicating what you want to be understood by your audience, then you must consider their model of reality.

which reminds me of:

“Programs must be written for people to read, and only incidentally for machines to execute.”
	— Abelson & Sussman, SICP, preface to the first edition

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“As a rule, whatever we don’t deal with in our lives, we pass on to our children. Our unfinished emotional business becomes theirs. As a therapist said to me, "Children swim in their parents’ unconscious like fish swim in the sea."”
	— Gábor Máté (1944–), 'In the Realm of Hungry Ghosts' (2008)

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Liliana Marie Prikler]

Am Donnerstag, dem 01.02.2024 um 20:30 +0000 schrieb Attila Lendvai:
> 
> for an average unix user a service is a process that is running in
> the backgroud, doing stuff mostly without any user interaction. you
> can try to argue this away, but i'm afraid that this is the state of
> things.
Which is exactly what etc-service-type does.  It symlinks stuff to /etc
without user interaction.


Cheers

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Attila Lendvai]

> Am Donnerstag, dem 01.02.2024 um 20:30 +0000 schrieb Attila Lendvai:
> 
> > for an average unix user a service is a process that is running in
> > the backgroud, doing stuff mostly without any user interaction. you
> > can try to argue this away, but i'm afraid that this is the state of
> > things.
> 
> Which is exactly what etc-service-type does. It symlinks stuff to /etc
> without user interaction.


we can spend our life honing in on a satisfying definition, but let it be enough that what is commonly understood as a service has an active component (see 'run' in my definition); i.e. it has a temporal dimension.

but honestly? it felt silly to even provide a definition in my mail. we either live in a different universe, or you're just focused on justify the status quo. whichever is the case, we have reached a dead end, because essentially, this is aesthetics.

but anyway, i gave my feedback, and as i don't have the authority to lobby for renaming core guix abstractions, i'm out.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Until you figure out that [manipulation] is going on, we're all going to be run like rats through a maze. Culture is an intelligence test, and when you pass that test you don't give a shit about culture.”
	— Terence McKenna (1946–2000)

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Ricardo Wurmus]

> for an average unix user a service is a process that is running in the
> backgroud, doing stuff mostly without any user interaction. you can
> try to argue this away, but i'm afraid that this is the state of
> things.

I don’t think it’s a good idea to aim to satisfy some presumed “average
unix user”, because such a user would not be familiar with many concepts
introduced by Guix (e.g. “guix shell” or “guix system”).

The manual defines system services by referencing users’ expectations:

--8<---------------cut here---------------start------------->8---
11.18.1 Service Composition
---------------------------

Here we define a “service” as, broadly, something that extends the
functionality of the operating system.  Often a service is a process—a
“daemon”—started when the system boots: a secure shell server, a Web
server, the Guix build daemon, etc.  Sometimes a service is a daemon
whose execution can be triggered by another daemon—e.g., an FTP server
started by ‘inetd’ or a D-Bus service activated by ‘dbus-daemon’.
Occasionally, a service does not map to a daemon.  For instance, the
“account” service collects user accounts and makes sure they exist when
the system runs; the “udev” service collects device management rules and
makes them available to the eudev daemon; the ‘/etc’ service populates
the ‘/etc’ directory of the system.
--8<---------------cut here---------------end--------------->8---

Shepherd takes care of monitoring daemons and the like, but services
provided by the system (in the sense of system facilities) don’t have to
be daemons.

-- 
Ricardo

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Attila Lendvai]

> > for an average unix user a service is a process that is running in the
> > backgroud, doing stuff mostly without any user interaction. you can
> > try to argue this away, but i'm afraid that this is the state of
> > things.
> 
> 
> I don’t think it’s a good idea to aim to satisfy some presumed “average
> unix user”, because such a user would not be familiar with many concepts
> introduced by Guix (e.g. “guix shell” or “guix system”).


the primary argument was that two, very different abstractions share the same name, and in shared contexts.

it's just icing on the cake that one of the abstractions is nothing like what most users understand by the name 'service'.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“If you love somebody, let them go, for if they return, they were always yours. If they don’t, they never were.”
	— Khalil Gibran (1883–1931)

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

On 2024-02-02, Attila Lendvai wrote:
>> > for an average unix user a service is a process that is running in the
>> > backgroud, doing stuff mostly without any user interaction. you can
>> > try to argue this away, but i'm afraid that this is the state of
>> > things.
>> 
>> 
>> I don’t think it’s a good idea to aim to satisfy some presumed “average
>> unix user”, because such a user would not be familiar with many concepts
>> introduced by Guix (e.g. “guix shell” or “guix system”).
>
>
> the primary argument was that two, very different abstractions share the same name, and in shared contexts.
>
> it's just icing on the cake that one of the abstractions is nothing like what most users understand by the name 'service'.

In the systemd realm, there are different types of services, I think one
is called "one-shot" which is effectively quite similar to the types of
services guix has... they do something once, and there is no running
daemon. So, for better or worse, guix is not so far from one of the most
widespread and commonly used systems here...

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Attila Lendvai]

> In the systemd realm, there are different types of services, I think one
> is called "one-shot" which is effectively quite similar to the types of
> services guix has... they do something once, and there is no running
> daemon. So, for better or worse, guix is not so far from one of the most
> widespread and commonly used systems here...


executed at each boot vs. executed when compiling the system (i.e. at different stages, as in 'staged computation').

it's a bit like using the same word to describe macros and functions in lisp: yes, deep down they are both just functions, but they are different enough to warrant a different name to facilitate a more efficient human comprehension.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The nation that will insist on drawing a broad line of demarcation between the fighting man and the thinking man is liable to find its fighting done by fools and its thinking done by cowards.”
	— Sir William Francis Butler (1838–1910)

Re: Introducing Guix "Features"!

[Carlo Zancanaro]

On Thu, Feb 01 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:
> On Thu, Nov 30 2023, Attila Lendvai wrote:
>> the use of 'service' to describe two rather different abstractions: a
>> component of an OS vs. a deamon process run by shepherd.
>
> Indeed, the use of 'service' in much of Guix appears to be a grand
> misnomer. It probably occurred because the meaning expanded over time.

We can actually point to a specific moment when the meaning was
expanded. It was done in Guix 0.9.0:
https://guix.gnu.org/en/blog/2015/gnu-guix-090-released/

Here is a blog post introducing services back in 2015, which has a
heading "Services Beyond Daemons" which discusses the change to broaden
the term to encompass what Guix now calls a service:
https://guix.gnu.org/en/blog/2015/service-composition-in-guixsd/

There is also a talk that Ludovic gave at FOSDEM 2017 where he discusses
the ideas of Guix services (I can't remember the content of this, but it
seemed like it might be relevant):
https://archive.fosdem.org/2017/schedule/event/composingsystemservicesinguixsd/

This doesn't seem like accidental concept creep. This was an intentional
decision to use the word "service" to mean something specific.

I say this to note that any argument to change the term should first
understand why this decision was made (see: Chesterton's Fence).

Carlo

Re: Introducing Guix "Features"! (Was: Syntactic Diabetes)

[Edouard Klein]

Weighting in here, as small as my weight may be:

- re-using 'service' is IMHO a bad idea, it is a loaded term and the
  expectation of a new reader is that a service is a SysV-init service:
  it can be started, status-ed, stopped, restarted, and that's it. It
  maps to a daemon running in the background. Basically a shepherd
  service.

- The fact that systemd service maps in functionality to guix services
  should be a huge redflag that the name is not good. Systemd sucks for
  many reasons, and using confusing and incosistent language is one of
  them.

- I agree that 'features' is a worse name, way too generic.

- Descriptive linguistics would consider the broader and
  historical use of the term in the UNIX crowd from which guix users are
  drawn, instead of the in-group use, which, despite being admitedly
  well documented and rooted in the history of the project, is
  overloading a close-but-not-exactly-matching term.
  I can not emphasize enough how much those subtle unexpected problems
  make adopting guix very hard.
  I pushed through because I understand that the project is
  fundamentally sound and worthwhile, but for a few years I have
  made interns and colleague work with guix and saw their motivation die
  by a thousand cuts because of confusing stuff like this.
  Finding a better term would be a worthwhile endeavour, for ease of
  adoption.
  Maybe we should dedicate a session for it in the next guix days ?

- At os-declaration time, ALL guix services can be added, extended,
  modified, and removed. Discovering this and the difficult-to-write
  syntax that goes along with notably the extension, led me to the
  syntactic sugar we were initially discussing.

- At os-reconfiguration time, SOME guix services will do their thing

- At os-boot time, SOME guix services will do their thing

- At os-running time, SOME (the shepherd kind) guix services can be
  stopped, started, statused, doced, custom-actioned, and restared

- Understanding these behaviours may help use find an naming ontology
  that would foster understanding and adoption.



Note that I don't disagree that service is a good name in *isolation*, I
just want to emphasize, as Attila did, that ignoring beginner's
expecations (however obsolete they may seem to be) is detrimental to the
project's adoption.


Cheers,

Edouard.

Felix Lechner <felix.lechner@lease-up.com> writes:

> On Thu, Nov 30 2023, Attila Lendvai wrote:
>
>> the use of 'service' to describe two rather different abstractions: a
>> component of an OS vs. a deamon process run by shepherd.
>
> Indeed, the use of 'service' in much of Guix appears to be a grand
> misnomer. It probably occurred because the meaning expanded over time.
>
> It's like we are looking back in time at the Big Bang. Our "services"
> are the microwave echoes of Guix's initial, creative spark!
>
> Please consider a recent, helpful reply to help-guix. [1] Carlo
> mentioned the term "service" eleven times, but none of them referred to
> what I believe most readers of this message would call a service in
> other contexts. What's a newbie on help-guix to think?
>
> Should Guix services instead be called "features"?
>
> Those "features" are central to any operating system definition. Other
> choices like "provider" may not fully capture our collective uses
> throughout the code and the documentation. I am especially thinking
> about 'modify-features' and '%base-features'.
>
> Kind regards
> Felix
>
> [1] https://lists.gnu.org/archive/html/help-guix/2024-01/msg00213.html

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Ludovic Courtès]

Hello!

Edouard Klein <edou@rdklein.fr> skribis:

> Thank you Liliana and Attila for the swift and actionable feedback :)
>
> Below is a revised proposition.
>
> Here is a minimal working example of an os declaration:
> ------------------mwe.scm---------------
> (use-modules
>  (beaver system)
>  (beaver functional-services)
>  (gnu packages version-control)
>  (gnu services web)
>  (gnu services telephony)
>  (gnu services ssh)
>  (gnu services base)
>  (guix gexp))
>
> (-> (minimal-ovh "osef")
>     (instantiate nginx)
>     (instantiate mumble-server
>                  (welcome-text "coucou")
>                  (port 64738))
>     (extend openssh `(("alice" ,(local-file "/home/edouard/.ssh/id_rsa.pub"))))
>     (modify openssh
>             (password-authentication? #f)
>             (allow-empty-passwords? #t))
>     (remove guix))
> -------------------------------------------------------
>
> To see the value of this syntactic sugar, try to replicate this MWE with
> the standard syntax. It's not horrendous, but it *is* off-putting to
> many newcomers to git, whereas this sugary piece is more readable for
> them (sample size of 1, p=0.00000005).

Glad you ran a user study.  :-)

This is looking more and more like:

  (define-syntax ->
    (syntax-rules ()
      ((_ os rules ...)
       (operating-system
         (inherit os)
         (services (modify-services (operating-system-user-services os)
                     rules ...))))))

One thing that always makes me hesitate due to the longstanding hygienic
macro tradition in Scheme is non-hyienic introduction of identifiers: in
the example you gave above, the identifiers ‘openssh-service-type’ and
‘openssh-configuration’ are automatically derived from ‘openssh’.  This
could lead to surprises and makes grepping harder.  But I don’t know,
maybe that’s the price to pay?

Maybe one conclusion we can draw from this is that configuration records
and service types should be more closely tied to one another.

>>> (service+ OS SERVICE [CONF])
>>> (service- OS SERVICE)

This one looked really revolutionary, but it’s inspiring too!

Thanks,
Ludo’.