Idea: Function composition to declare operating-system

Idea: Function composition to declare operating-system

[Théo Maxime Tyburn]

Hi guix!

I experimented on a functional approach to the operating
system declaration. My goal was to be able to pass an operating-system
instance to a composition of functions, modifying it one after another
and returning the desired operating-system object. I find this approach
more convenient because it allows for better segmentation of the system
features. I achieved to make it work for the system declaration I
usually use and I’d like to share it with you.

--BEGIN USE_CASE
For example to add jackd to my system I need to add the "realtime"
group, add some users to this group and add a pam-limits-service. If I
want to remove this functionality from my system using the declarative
approach I have to look down my config file for places where I added
these things. Usually I partially solve this problem by putting comments
to signal the purpose of each code block in the system declaration.

But wouldn’t it be better if I just had a function `add-jackd` that takes an
operating-system instance and returns the os with the extra functionalities ?
--END USE_CASE

So that was the purpose of the experimentation. It didn’t turn out to be
too complicated to implement. At least for my use case, I just needed to add two helper
functions to extend users and services fields. The rest is handled directly by
record inheritance and by accessing the fields of the input operating-system.

The final declaration looks like this:
--8<---------------cut here---------------start------------->8---
((apply compose (reverse os-functions)) minimal-os)
--8<---------------cut here---------------end--------------->8---


Where `minimal-os` is roughly an (operation-system) declaration with at
least all the required fields. `os-functions` is a list of procedures
taking an operating-system as input and returning an operating-system.

An example for one such function that adds a label:
--8<---------------cut here---------------start------------->8---
(define (label-os os)
  (operating-system
   (inherit os)
   (label (string-append (operating-system-host-name os) ":"
						 "refactor-functional-os" ;; my custom tag
						 " " (operating-system-default-label this-operating-system)))))
--8<---------------cut here---------------end--------------->8---

The most problematic thing is handling services. For some reason I
ignore, there is a set of services that always gets added
again to the list of services when you call (operating-system ...) with a
services field with more than one service. So you can’t just do like
above and
--8<---------------cut here---------------start------------->8---
(define (add-some-service os)
  (operating-system
   (inherit os)
   (services (cons (simple-service ...)
				   (operating-system-services os)))))
--8<---------------cut here---------------end--------------->8---
because some services that need to be unique would come more than once
in the final operating-system object.

So I just added a simple function to filter them out
--8<---------------cut here---------------start------------->8---
(define* (extend-operating-system-services os services #:key (drop '()) (keep '()))
  (append (filter (lambda (service)
					(not (member (service-type-name (service-kind service))
								 (filter (lambda (s) (not (member s keep)))
										 (append drop %fixed-system-service-types)))))
				  (operating-system-services os))
		  services))
--8<---------------cut here---------------end--------------->8---
and also force keeping or dropping of some services if needed. The list
of services that gets duplicated seems to be this one:
--8<---------------cut here---------------start------------->8---
(define %fixed-system-service-types
  '(account activate boot cleanup etc file-systems firmware fstab guix host-name linux-bare-metal linux-builder pam profile root-file-system session-environment setuid-program shepherd-root system user-processes))
--8<---------------cut here---------------end--------------->8---
I generated the list by just checking which services get duplicated, so I am not
very sure about it. There surely is a better way to get it. 

Anyway I can now define a function adding desktop functionalities:
--8<---------------cut here---------------start------------->8---
(define (x-os os)
  (operating-system
   (inherit os)
   (services
	(extend-operating-system-services
	 os
	 (list
	  ;; slim display manager
	  (service slim-service-type
			   (slim-configuration
				(display ":0")
				(vt "vt7")
				(theme %default-slim-theme)
				(theme-name %default-slim-theme-name)
				(xorg-configuration
				 (xorg-configuration
				  (keyboard-layout (operating-system-keyboard-layout os)))))))
	 #:drop '(gdm)))
   (packages (cons*
			  ;; window managers
			  i3-wm python-py3status
			  emacs-nc-exwm-xdg
			  (operating-system-packages os)
			  ))))
--8<---------------cut here---------------end--------------->8---

Of course there is room for some macros to make this more elegant, but
this is the rough idea.

In a way it feels like treating the operating-system like a service
you can extend. Maybe it would even make sense to implement this as a
service ? Not sure about that.

It seems it would also be reasonable to have something like an
operating-system-configuration record and a way to compose some before
putting them into an operating-system record (it seems to be the
approach rde `features` are based on). But I felt too lazy to copy all the
fields from the operating-system record definition. There might be a
way to get all the fields programatically and define a
record/configuration though.

Anyway, what do you think about this functionality? Have you already experimented with similar things?
Did I reinvent the wheel? Is there a better approach?

Anyway that was a very fun hacking session :)

Happy Hacking!

Théo

Re: Idea: Function composition to declare operating-system

[muradm]

[-- Attachment #1: Type: text/plain, Size: 5009 bytes --]


Hi,

I had similar problem popping up periodically.
So here are my 10 cents..

Théo Maxime Tyburn <theo.tyburn@gmail.com> writes:

> Hi guix!
>

[...]

>
> --BEGIN USE_CASE
> For example to add jackd to my system I need to add the 
> "realtime"
> group, add some users to this group and add a 
> pam-limits-service. If I
> want to remove this functionality from my system using the 
> declarative
> approach I have to look down my config file for places where I 
> added
> these things. Usually I partially solve this problem by putting 
> comments
> to signal the purpose of each code block in the system 
> declaration.
>
> But wouldn’t it be better if I just had a function `add-jackd` 
> that takes an
> operating-system instance and returns the os with the extra 
> functionalities ?
> --END USE_CASE

To clarify, do you ask that in the end of the day, some where
in (gnu services ...) there should be exported `add-jackd`
function? If so, I beleive that this will increase cross
dependency between things, thus decreasing flexibility.

Imagine `add-jackd` maintainer should always keep track on
what is being added into guix, that potentially may cause
conflict with jackd and/or require adjustments in `add-jackd`
function implementation.

Also, IMHO, implementation of `add-jackd` would be very
much opinionated.

>
> So that was the purpose of the experimentation. It didn’t turn 
> out to be
> too complicated to implement. At least for my use case, I just 
> needed to add two helper
> functions to extend users and services fields. The rest is 
> handled directly by
> record inheritance and by accessing the fields of the input 
> operating-system.
>
> The final declaration looks like this:
>
> ((apply compose (reverse os-functions)) minimal-os)
>

[...]

>
> (define* (extend-operating-system-services os services #:key 
> (drop '()) (keep '()))
>   (append (filter (lambda (service)
> 					(not (member (service-type-name (service-kind 
> service))
> 								 (filter (lambda (s) (not (member 
> s keep)))
> 										 (append drop 
> %fixed-system-service-types)))))
> 				  (operating-system-services os))
> 		  services))
>

I suppose this could be useful to have it in guix toolbox,
although I would prefer to have (required '(account activate ...))
or (required %fixed-system-service-types) optional argument,
instead of refering to global constant 
%fixed-system-service-types,
which might not satisfy everyone requirements.

> and also force keeping or dropping of some services if needed. 
> The list
> of services that gets duplicated seems to be this one:
>
> (define %fixed-system-service-types
>   '(account activate boot cleanup etc file-systems firmware 
>   fstab guix host-name linux-bare-metal linux-builder pam 
>   profile root-file-system session-environment setuid-program 
>   shepherd-root system user-processes))
>
> I generated the list by just checking which services get 
> duplicated, so I am not
> very sure about it. There surely is a better way to get it.
>
> Anyway I can now define a function adding desktop 
> functionalities:
>
> (define (x-os os)
>   (operating-system
>    (inherit os)
>    (services
> 	(extend-operating-system-services
> 	 os
> 	 (list
> 	  ;; slim display manager
> 	  (service slim-service-type
> 			   (slim-configuration
> 				(display ":0")
> 				(vt "vt7")
> 				(theme %default-slim-theme)
> 				(theme-name %default-slim-theme-name)
> 				(xorg-configuration
> 				 (xorg-configuration
> 				  (keyboard-layout 
> (operating-system-keyboard-layout os)))))))
> 	 #:drop '(gdm)))
>    (packages (cons*
> 			  ;; window managers
> 			  i3-wm python-py3status
> 			  emacs-nc-exwm-xdg
> 			  (operating-system-packages os)
> 			  ))))
>
> Of course there is room for some macros to make this more 
> elegant, but
> this is the rough idea.
>
> In a way it feels like treating the operating-system like a 
> service
> you can extend. Maybe it would even make sense to implement this 
> as a
> service ? Not sure about that.
>
> It seems it would also be reasonable to have something like an
> operating-system-configuration record and a way to compose some 
> before
> putting them into an operating-system record (it seems to be the
> approach rde `features` are based on). But I felt too lazy to 
> copy all the
> fields from the operating-system record definition. There might 
> be a
> way to get all the fields programatically and define a
> record/configuration though.
>
> Anyway, what do you think about this functionality? Have you 
> already experimented with similar things?
> Did I reinvent the wheel? Is there a better approach?

Did you try using (modify-services ...)? Basically, you can
achieve similar goal within services list only with it.

> Anyway that was a very fun hacking session :)
>
> Happy Hacking!
>
> Théo

Thanks in advance,
muradm

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Idea: Function composition to declare operating-system

[Théo Maxime Tyburn]

Hi muradm!

muradm <mail@muradm.net> writes:

[...]

>> --BEGIN USE_CASE
>> For example to add jackd to my system I need to add the "realtime"
>> group, add some users to this group and add a pam-limits-service. If
>> I
>> want to remove this functionality from my system using the
>> declarative
>> approach I have to look down my config file for places where I added
>> these things. Usually I partially solve this problem by putting
>> comments
>> to signal the purpose of each code block in the system declaration.
>>
>> But wouldn’t it be better if I just had a function `add-jackd` that
>> takes an
>> operating-system instance and returns the os with the extra
>> functionalities ?
>> --END USE_CASE
>
> To clarify, do you ask that in the end of the day, some where
> in (gnu services ...) there should be exported `add-jackd`
> function? If so, I beleive that this will increase cross
> dependency between things, thus decreasing flexibility.
>
> Imagine `add-jackd` maintainer should always keep track on
> what is being added into guix, that potentially may cause
> conflict with jackd and/or require adjustments in `add-jackd`
> function implementation.
>
> Also, IMHO, implementation of `add-jackd` would be very
> much opinionated.

Actually I was thinking that the functions that build up the system like
`add-jackd` would be written by users. It is still user
configuration, just packaged in a function. So no one would "have to"
maintain it. But this could still provide an easy way to share
independent parts of a system configs. Especially for features requiring a
complex setup, a canonical configuration could be something you would want to maintain. But
it would be up to the maintainer.

Assuming tough, someone wants to maintain `add-jackd` under (gnu
services ...), it should not be more maintenance effort than the effort
each and every user of jackd has to put so that it’s system
configuration doesn’t break. So I don’t see anything against it. But I
might be wrong.

>>
>> So that was the purpose of the experimentation. It didn’t turn out
>> to be
>> too complicated to implement. At least for my use case, I just
>> needed to add two helper
>> functions to extend users and services fields. The rest is handled
>> directly by
>> record inheritance and by accessing the fields of the input
>> operating-system.
>>
>> The final declaration looks like this:
>>
>> ((apply compose (reverse os-functions)) minimal-os)
>>
>
> [...]
>
>>
>> (define* (extend-operating-system-services os services #:key (drop
>> '()) (keep '()))
>>   (append (filter (lambda (service)
>> 					(not (member (service-type-name (service-kind
>> service))
>> 								 (filter (lambda (s) (not (member s
>> keep)))
>> 										 (append drop
>> %fixed-system-service-types)))))
>> 				  (operating-system-services os))
>> 		  services))
>>
>
> I suppose this could be useful to have it in guix toolbox,
> although I would prefer to have (required '(account activate ...))
> or (required %fixed-system-service-types) optional argument,
> instead of refering to global constant %fixed-system-service-types,
> which might not satisfy everyone requirements.

I agree, better make this optional. If anything, only this this kind of
helper function would be added to guix so that users can define their
system conveniently using this function composition approach. 

>> and also force keeping or dropping of some services if needed. The
>> list
>> of services that gets duplicated seems to be this one:
>>
>> (define %fixed-system-service-types
>>   '(account activate boot cleanup etc file-systems firmware   fstab
>> guix host-name linux-bare-metal linux-builder pam   profile
>> root-file-system session-environment setuid-program   shepherd-root
>> system user-processes))
>>
>> I generated the list by just checking which services get duplicated,
>> so I am not
>> very sure about it. There surely is a better way to get it.
>>
>> Anyway I can now define a function adding desktop functionalities:
>>
>> (define (x-os os)
>>   (operating-system
>>    (inherit os)
>>    (services
>> 	(extend-operating-system-services
>> 	 os
>> 	 (list
>> 	  ;; slim display manager
>> 	  (service slim-service-type
>> 			   (slim-configuration
>> 				(display ":0")
>> 				(vt "vt7")
>> 				(theme %default-slim-theme)
>> 				(theme-name %default-slim-theme-name)
>> 				(xorg-configuration
>> 				 (xorg-configuration
>> 				  (keyboard-layout (operating-system-keyboard-layout
>> os)))))))
>> 	 #:drop '(gdm)))
>>    (packages (cons*
>> 			  ;; window managers
>> 			  i3-wm python-py3status
>> 			  emacs-nc-exwm-xdg
>> 			  (operating-system-packages os)
>> 			  ))))
>>
>> Of course there is room for some macros to make this more elegant,
>> but
>> this is the rough idea.
>>
>> In a way it feels like treating the operating-system like a service
>> you can extend. Maybe it would even make sense to implement this as
>> a
>> service ? Not sure about that.
>>
>> It seems it would also be reasonable to have something like an
>> operating-system-configuration record and a way to compose some
>> before
>> putting them into an operating-system record (it seems to be the
>> approach rde `features` are based on). But I felt too lazy to copy
>> all the
>> fields from the operating-system record definition. There might be a
>> way to get all the fields programatically and define a
>> record/configuration though.
>>
>> Anyway, what do you think about this functionality? Have you already
>> experimented with similar things?
>> Did I reinvent the wheel? Is there a better approach?
>
> Did you try using (modify-services ...)? Basically, you can
> achieve similar goal within services list only with it.

Yes you are right, that would also do the trick. There is actually no
real need for the helper function defined above. One would just have to
pass the list of service-types that gets duplicated to (delete TYPE) in
modify-services. I am not aware of a way of deleting a list of
service-types easily using modify-services without defining a macro
though, so it seems you need some sort of helper function anyway.

> Thanks in advance,
> muradm

Thanks for the return,

Théo

Re: Idea: Function composition to declare operating-system

[Hartmut Goebel]

Am 29.08.22 um 17:14 schrieb Théo Maxime Tyburn:
> Anyway, what do you think about this functionality? Have you already experimented with similar things?
> Did I reinvent the wheel? Is there a better approach?

I really like the idea!

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Idea: Function composition to declare operating-system

[Ludovic Courtès]

Hi Théo,

Théo Maxime Tyburn <theo.tyburn@gmail.com> skribis:

> I experimented on a functional approach to the operating
> system declaration. My goal was to be able to pass an operating-system
> instance to a composition of functions, modifying it one after another
> and returning the desired operating-system object. I find this approach
> more convenient because it allows for better segmentation of the system
> features. I achieved to make it work for the system declaration I
> usually use and I’d like to share it with you.

Neat!  In some cases, having “transformation” functions is clearer
indeed.  There’s a couple of them in the code, such as
‘virtualized-operating-system’, that make it easy to adapt an OS config
to a specific use case.

Thanks for sharing!

Ludo’.

Idea: Function composition to declare operating-system

[Nathan Dehnel]

Very cool!