[PATCHES] Get elogind-service working as intended

[PATCHES] Get elogind-service working as intended

[Mark H Weaver]

[-- Attachment #1: Type: text/plain, Size: 338 bytes --]

These patches are needed to get Andy's elogind-service working as
intended, e.g. to allow things like suspend-on-lid-close and various
hot keys to work.

I'm not really happy with any of these, so if someone wants to solve
these problems in a better way, please don't hesitate to do so :)

Comments and suggestions welcome.

      Mark



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: [PATCH 1/3] file-systems: Add %systemd-file-systems; add it to %base-file-systems --]
[-- Type: text/x-patch, Size: 2136 bytes --]

From 34d9f68b886fe590ff0efcd33d54e5d2f487ff70 Mon Sep 17 00:00:00 2001
From: Mark H Weaver <mhw@netris.org>
Date: Thu, 3 Sep 2015 16:58:08 -0400
Subject: [PATCH 1/3] file-systems: Add %systemd-file-systems; add it to
 %base-file-systems.

* gnu/system/file-systems.scm (%systemd-file-systems): New variable.
  (%base-file-systems): Add %systemd-file-systems.
---
 gnu/system/file-systems.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
index 003eb44..d03dcff 100644
--- a/gnu/system/file-systems.scm
+++ b/gnu/system/file-systems.scm
@@ -50,6 +50,7 @@
             %devtmpfs-file-system
             %immutable-store
             %control-groups
+            %systemd-file-systems
 
             %base-file-systems
             %container-file-systems
@@ -258,6 +259,26 @@ UUID representation."
                '("cpuset" "cpu" "cpuacct" "memory" "devices" "freezer"
                  "blkio" "perf_event" "hugetlb")))))
 
+(define %systemd-file-systems
+  ;; We don't use systemd, but these file systems are needed for elogind,
+  ;; which was extracted from systemd.
+  (list (file-system
+          (device "systemd")
+          (mount-point "/run/systemd")
+          (type "tmpfs")
+          (check? #f)
+          (flags '(no-suid no-dev no-exec))
+          (options "mode=0755")
+          (create-mount-point? #t))
+        (file-system
+          (device "systemd")
+          (mount-point "/run/user")
+          (type "tmpfs")
+          (check? #f)
+          (flags '(no-suid no-dev no-exec))
+          (options "mode=0755")
+          (create-mount-point? #t))))
+
 (define %base-file-systems
   ;; List of basic file systems to be mounted.  Note that /proc and /sys are
   ;; currently mounted by the initrd.
@@ -265,6 +286,7 @@ UUID representation."
                 %pseudo-terminal-file-system
                 %shared-memory-file-system
                 %immutable-store)
+          %systemd-file-systems
           %control-groups))
 
 ;; File systems for Linux containers differ from %base-file-systems in that
-- 
2.4.3


[-- Attachment #3: [PATCH 2/3] services: Add udev rules for (upower colord elogind) to %desktop-services --]
[-- Type: text/x-patch, Size: 3681 bytes --]

From 59921d5bd704b07d25104b7a1921380892f60a80 Mon Sep 17 00:00:00 2001
From: Mark H Weaver <mhw@netris.org>
Date: Thu, 3 Sep 2015 17:12:38 -0400
Subject: [PATCH 2/3] services: Add udev rules for (upower colord elogind) to
 %desktop-services.

* gnu/services/desktop.scm (%desktop-services): Replace the 'udev-service'
  from %base-services.
* gnu/services/base.scm (%base-services): Add a comment to keep them in sync.
---
 gnu/services/base.scm    |  4 ++++
 gnu/services/desktop.scm | 25 ++++++++++++++++++++-----
 2 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 888e446..7f37b3d 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2013, 2014, 2015 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2015 Alex Kost <alezost@gmail.com>
+;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -884,6 +885,9 @@ This is the GNU operating system, welcome!\n\n")))
           ;; The LVM2 rules are needed as soon as LVM2 or the device-mapper is
           ;; used, so enable them by default.  The FUSE and ALSA rules are
           ;; less critical, but handy.
+          ;;
+          ;; XXX Keep this in sync with the 'udev-service' call in
+          ;; %desktop-services.
           (udev-service #:rules (list lvm2 fuse alsa-utils crda)))))
 
 ;;; base.scm ends here
diff --git a/gnu/services/desktop.scm b/gnu/services/desktop.scm
index 1bf3db0..6017449 100644
--- a/gnu/services/desktop.scm
+++ b/gnu/services/desktop.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2015 Andy Wingo <wingo@igalia.com>
+;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -32,6 +33,8 @@
   #:use-module (gnu packages avahi)
   #:use-module (gnu packages wicd)
   #:use-module (gnu packages polkit)
+  #:use-module ((gnu packages linux)
+                #:select (lvm2 fuse alsa-utils crda))
   #:use-module (guix monads)
   #:use-module (guix records)
   #:use-module (guix store)
@@ -593,12 +596,24 @@ when they log out."
          (ntp-service)
 
          (map (lambda (mservice)
-                ;; Provide an nscd ready to use nss-mdns.
                 (mlet %store-monad ((service mservice))
-                  (if (memq 'nscd (service-provision service))
-                      (nscd-service (nscd-configuration)
-                                    #:name-services (list nss-mdns))
-                      mservice)))
+                  (cond
+                   ;; Provide an nscd ready to use nss-mdns.
+                   ((memq 'nscd (service-provision service))
+                    (nscd-service (nscd-configuration)
+                                  #:name-services (list nss-mdns)))
+
+                   ;; Add more rules to udev-service.
+                   ;;
+                   ;; XXX Keep this in sync with the 'udev-service' call in
+                   ;; %base-services.  Here we intend only to add 'upower',
+                   ;; 'colord', and 'elogind'.
+                   ((memq 'udev (service-provision service))
+                    (udev-service #:rules
+                                  (list lvm2 fuse alsa-utils crda
+                                        upower colord elogind)))
+
+                   (else mservice))))
               %base-services)))
 
 ;;; desktop.scm ends here
-- 
2.4.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: [PATCH 3/3] services: elogind-configuration: Disable hibernation key by default --]
[-- Type: text/x-patch, Size: 1441 bytes --]

From 00b8791d7cea8f9702a004377756caefa243b23e Mon Sep 17 00:00:00 2001
From: Mark H Weaver <mhw@netris.org>
Date: Thu, 3 Sep 2015 17:16:26 -0400
Subject: [PATCH 3/3] services: elogind-configuration: Disable hibernation key
 by default.

* gnu/services/desktop.scm (<elogind-configuration>): Change the default value
  of the 'handle-hibernate-key' field to 'ignore'.
---
 gnu/services/desktop.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/gnu/services/desktop.scm b/gnu/services/desktop.scm
index 6017449..b91bdd8 100644
--- a/gnu/services/desktop.scm
+++ b/gnu/services/desktop.scm
@@ -439,7 +439,11 @@ the system if the user is logged in locally."
   (handle-suspend-key              elogind-handle-suspend-key
                                    (default 'suspend))
   (handle-hibernate-key            elogind-handle-hibernate-key
-                                   (default 'hibernate))
+                                   ;; (default 'hibernate)
+                                   ;; XXX Ignore it for now, since we don't
+                                   ;; yet handle resume-from-hibernation in
+                                   ;; our initrd.
+                                   (default 'ignore))
   (handle-lid-switch               elogind-handle-lid-switch
                                    (default 'suspend))
   (handle-lid-switch-docked        elogind-handle-lid-switch-docked
-- 
2.4.3

Re: [PATCHES] Get elogind-service working as intended

[Andy Wingo]

On Thu 03 Sep 2015 23:23, Mark H Weaver <mhw@netris.org> writes:

> These patches are needed to get Andy's elogind-service working as
> intended, e.g. to allow things like suspend-on-lid-close and various
> hot keys to work.
>
> I'm not really happy with any of these, so if someone wants to solve
> these problems in a better way, please don't hesitate to do so :)
>
> Comments and suggestions welcome.

Thanks for doing this!  I suggest we apply, just to get things working,
then work on expanding <service> to have more fields.  I can do that
later this morning.  I note also that there is still more work to do
related to PAM.

The problem is that there are services whose configuration should depend
on the set of services.  For example udev, which should add rules
corresponding to the colord service, etc.

Now one option would be to depend on on the set of services but on the
set of packages in the system install.  I agree with Mark that this is a
bad idea because you don't want to have to audit a package before adding
it to the system install.  On the other hand if privileges like
installing a udev rule are only accessible from services, that's easier
to audit because there are fewer services.

So then, for example, the elogind service would indicate that it
installs udev rules, dbus interfaces, and polkit rules from the elogind
package.  The udev service would then query all services, adding rules
from services that indicate that they extend the udev configuration.  To
do this I propose a "finalize-configuration" field in services.  First,
the services are collected into a list, as they are now.  Next, Guix
does something like this:

  (define (finalize-services services)
    (map (lambda (service)
           ((service-finalize service) service services))
         services))

The default service-finalize will be (lambda (service services)
service).  WDYT?

Cheers,

Andy

Service refactoring

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

Andy Wingo <wingo@igalia.com> skribis:

> So then, for example, the elogind service would indicate that it
> installs udev rules, dbus interfaces, and polkit rules from the elogind
> package.  The udev service would then query all services, adding rules
> from services that indicate that they extend the udev configuration.  To
> do this I propose a "finalize-configuration" field in services.  First,
> the services are collected into a list, as they are now.  Next, Guix
> does something like this:
>
>   (define (finalize-services services)
>     (map (lambda (service)
>            ((service-finalize service) service services))
>          services))
>
> The default service-finalize will be (lambda (service services)
> service).  WDYT?

Yes, something like that.

An important characteristic is that service types extend each other:
dbus extends dmd (by adding stuff into dmd.conf), polkit extends dbus
(through .service files), elogind extends polkit (through policy files)
and dbus and udev, and so on.

Service types and their “extends” relations form a DAG like this:


[-- Attachment #2: the DAG --]
[-- Type: image/png, Size: 49482 bytes --]

[-- Attachment #3: Type: text/plain, Size: 633 bytes --]


I think we want to define a new API to capture this.  Service types will
declare which other service types they extend and how; service types
that can be extended will declare how extensions affect their
parameters.  (We need to distinguish between “service type” and “service
instance” because there can be several instances of a given type.)

Building the system works in two steps: first we propagate the
extensions down to the sinks of the DAG (such as dmd), then we
instantiate those.

Thoughts?

I’ll try to refine this and come up with a proof-of-concept for
discussion soonish.

Thanks,
Ludo’.

Re: Service refactoring

[Andy Wingo]

On Sun 06 Sep 2015 23:23, ludo@gnu.org (Ludovic Courtès) writes:

> Service types and their “extends” relations form a DAG

I am not sure how much ordering matters.  The reason is that the
"extends" relations actually proceed from packages associated with a
service, not the service itself.  It's enough to know the set of
services and their associated extends; ordering does not appear to be
important.  Of course we could do a topological sort on services for
some other reason, but we don't actually need to for these purposes.

I think a two-phase configuration can work: one, in which we specify
services like this:

  (operating-system
   ...
   (services SERVICES))

and a second in which the services are "finalized".  Finalization is a
monadic procedure of type SERVICE SERVICES -> SERVICE.  Finalization is
where e.g. udev would grovel the SERVICES to collect all udev extends.

Andy

Re: Service refactoring

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> On Sun 06 Sep 2015 23:23, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Service types and their “extends” relations form a DAG
>
> I am not sure how much ordering matters.  The reason is that the
> "extends" relations actually proceed from packages associated with a
> service, not the service itself.  It's enough to know the set of
> services and their associated extends; ordering does not appear to be
> important.  Of course we could do a topological sort on services for
> some other reason, but we don't actually need to for these purposes.

My intent was more to make relations among services explicit, rather
than specify some ordering.

> I think a two-phase configuration can work: one, in which we specify
> services like this:
>
>   (operating-system
>    ...
>    (services SERVICES))
>
> and a second in which the services are "finalized".  Finalization is a
> monadic procedure of type SERVICE SERVICES -> SERVICE.  Finalization is
> where e.g. udev would grovel the SERVICES to collect all udev extends.

It would work, yes.  But I think there’s a couple of issues worth
addressing.

An issue is that each finalization procedure is passed more information
than strictly needed.  Thus, any service can potentially influence any
other service’s configuration, which makes it harder to reason about
service composition.

Another problem is that it’s not really extensible: we’ll have to keep
adding new fields to <service> every time we think of a new way a
service needs to extend another service.  We could use an alist instead
of those record fields, but then that would make the thing sloppy (typos
would go undetected, etc.)

There’s also the assumption that each service that the user specifies
maps to a dmd service, which is not always the case (D-Bus services,
Apache modules, etc.)

Lastly, without making the “extends” relations explicit, it’s easy to
end up specifying an extension that actually extends nothing.  For
instance, you use a service that has a non-empty ‘dbus-services’ field
but forget to use the D-Bus service; Guix has no way to tell that
something’s missing.

What I suggested would address these by constraining things.  What
remains to be seen is if this can be implemented without making things
too complex.  I’ll try to experiment with this.

Thanks,
Ludo’.

Re: Service refactoring

[Andy Wingo]

Hi!  Some thoughts but I am interested to see your results, your
approach sounds good.

On Tue 08 Sep 2015 12:12, ludo@gnu.org (Ludovic Courtès) writes:

> An issue is that each finalization procedure is passed more information
> than strictly needed.  Thus, any service can potentially influence any
> other service’s configuration, which makes it harder to reason about
> service composition.

Good point.

> Another problem is that it’s not really extensible: we’ll have to keep
> adding new fields to <service> every time we think of a new way a
> service needs to extend another service.  We could use an alist instead
> of those record fields, but then that would make the thing sloppy (typos
> would go undetected, etc.)

On the scale of badness, this to me is bad but not so bad.  How many of
these kinds of services will there be?  I am guessing like 6 or so (pam,
udev, dbus, polkit, file systems, dunno).  It's manageable.

If you extend services as first-class values -- is this your proposal?
-- that gets more precise, but then you have the problem of -- well
let's say that desktop.scm defines dbus-service, which is a factory
procedure.  What would elogind-service capture?  Would you have to pass
in the value of the instantiated dbus service in your (operating-system
(services _))?

> There’s also the assumption that each service that the user specifies
> maps to a dmd service, which is not always the case (D-Bus services,
> Apache modules, etc.)

Good point...

> Lastly, without making the “extends” relations explicit, it’s easy to
> end up specifying an extension that actually extends nothing.  For
> instance, you use a service that has a non-empty ‘dbus-services’ field
> but forget to use the D-Bus service; Guix has no way to tell that
> something’s missing.

This is only the case if you have an open extension set.  If the
extension set is closed, like the set of <service> fields is closed, we
can add custom logic.

> What I suggested would address these by constraining things.  What
> remains to be seen is if this can be implemented without making things
> too complex.  I’ll try to experiment with this.

Good luck :)  Looking forward to see what you come up with.

Cheers,

Andy

Re: Service refactoring

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@igalia.com> skribis:
>
>> So then, for example, the elogind service would indicate that it
>> installs udev rules, dbus interfaces, and polkit rules from the elogind
>> package.  The udev service would then query all services, adding rules
>> from services that indicate that they extend the udev configuration.  To
>> do this I propose a "finalize-configuration" field in services.  First,
>> the services are collected into a list, as they are now.  Next, Guix
>> does something like this:
>>
>>   (define (finalize-services services)
>>     (map (lambda (service)
>>            ((service-finalize service) service services))
>>          services))
>>
>> The default service-finalize will be (lambda (service services)
>> service).  WDYT?
>
> Yes, something like that.
>
> An important characteristic is that service types extend each other:
> dbus extends dmd (by adding stuff into dmd.conf), polkit extends dbus
> (through .service files), elogind extends polkit (through policy files)
> and dbus and udev, and so on.
>
> Service types and their “extends” relations form a DAG like this:

I think that Ludovic's proposal is excellent, and I'm sympathetic to his
desire to minimize the amount of information services have about each
other, but I have one concern:

Will we ever encounter a case where two services need to be told about
each other?  More generally, the "needs to know about" graph might
contain cycles.  Even if there's not one today, we may encounter one
tomorrow.  I don't see a compelling reason to restrict this structure to
be a DAG, and such a restriction may cause problems for us later.

What do you think?

       Mark

Re: [PATCHES] Get elogind-service working as intended

[Ludovic Courtès]

I just noticed I had this message in my draft folder, and it seems to
still be applicable, so here we go.  Sorry for the mess!

Mark H Weaver <mhw@netris.org> skribis:

> From 34d9f68b886fe590ff0efcd33d54e5d2f487ff70 Mon Sep 17 00:00:00 2001
> From: Mark H Weaver <mhw@netris.org>
> Date: Thu, 3 Sep 2015 16:58:08 -0400
> Subject: [PATCH 1/3] file-systems: Add %systemd-file-systems; add it to
>  %base-file-systems.
>
> * gnu/system/file-systems.scm (%systemd-file-systems): New variable.
>   (%base-file-systems): Add %systemd-file-systems.

[...]

> +(define %systemd-file-systems

I would call the variable ‘%elogind-file-systems’ for clarity, but
otherwise LGTM.

> From 59921d5bd704b07d25104b7a1921380892f60a80 Mon Sep 17 00:00:00 2001
> From: Mark H Weaver <mhw@netris.org>
> Date: Thu, 3 Sep 2015 17:12:38 -0400
> Subject: [PATCH 2/3] services: Add udev rules for (upower colord elogind) to
>  %desktop-services.
>
> * gnu/services/desktop.scm (%desktop-services): Replace the 'udev-service'
>   from %base-services.
> * gnu/services/base.scm (%base-services): Add a comment to keep them in sync.

OK.

> From 00b8791d7cea8f9702a004377756caefa243b23e Mon Sep 17 00:00:00 2001
> From: Mark H Weaver <mhw@netris.org>
> Date: Thu, 3 Sep 2015 17:16:26 -0400
> Subject: [PATCH 3/3] services: elogind-configuration: Disable hibernation key
>  by default.
>
> * gnu/services/desktop.scm (<elogind-configuration>): Change the default value
>   of the 'handle-hibernate-key' field to 'ignore'.

OK.

Thanks!

Ludo’.

Re: Service refactoring

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Will we ever encounter a case where two services need to be told about
> each other?  More generally, the "needs to know about" graph might
> contain cycles.  Even if there's not one today, we may encounter one
> tomorrow.  I don't see a compelling reason to restrict this structure to
> be a DAG, and such a restriction may cause problems for us later.

I can’t think of any pair of services that “extend” each other, but I’ll
keep that in mind.

Thanks,
Ludo’.

Re: Service refactoring

[Ludovic Courtès]

The ‘wip-service-refactor’ currently does the easy part of the service
refactoring, which essentially turns the service procedures into regular
procedures (rather than monadic procedures.)

I think it’s important because it allows users to ‘map’ over
%base-services, for instance, possibly replacing some of the entries,
without actually touching the store (whereas currently, one would first
have to “bind” each item of %base-services, possibly affecting the
store.)

It also makes it easier for newcomers to fiddle with services in their
OS config file.

To do so, the branch adds a few “file-like objects” that really are just
counterparts of existing monadic procedures:

  • ‘computed-file’ returns an object with the same meaning as a
    ‘gexp->derivation’ call;

  • ‘program-file’ is the declarative counterpart of ‘gexp->script’.

These objects can be used anywhere in a gexp, which is probably going to
be useful in other places.

--8<---------------cut here---------------start------------->8---
+ 56d3d68 system: Make service procedures non-monadic.
+ 807d34a gexp: Add 'mixed-text-file'.
+ 03d2914 gexp: Add 'program-file'.
+ 82ad249 gexp: Add 'computed-file'.
+ 7dfa966 services: 'mingetty-service' no longer takes monadic values.
--8<---------------cut here---------------end--------------->8---

More to come!

Ludo’.

Re: Service refactoring

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

Currently that ’wip-service-refactor’ contains a first implementation of
the “service DAG”.  So far I only “converted” services in (gnu services
base); modulo the yet-to-be-converted services, it looks like this:


[-- Attachment #2: the service hierarchy --]
[-- Type: image/png, Size: 73921 bytes --]

[-- Attachment #3: Type: text/plain, Size: 2791 bytes --]


“Service” here has a much broader meaning than before.

There’s a service for PAM services, one for user accounts, one for dmd
services, one for /etc entries, one for activation snippets, and finally
the “boot” service.

The PAM root service collects PAM services produced by lshd, mingetty,
etc. and turns them into ‘pam.d’ entry for /etc; the /etc service takes
such entries and turns them into a gexp that calls ‘activate-etc’.
Likewise, the guix-daemon service passes user accounts and groups to the
account service, which turns that into an activation snippet.

The nice thing is that it allows us to express things that were not
possible before, and increases separation of concerns.  For instance,
the ‘account’ service takes care of everything related to user accounts
in a single place, whereas before this would be entangled in (gnu
system); ditto for PAM services.

The API defines <service-type>, <service-extension>, and <service>.  The
“service type” defines how services of this type are “extended” and/or
how services of this type extend other services:

  (define dmd-root-service-type
    (service-type
     (name 'dmd-root)
     ;; Extending the root dmd service (aka. PID 1) happens by concatenating the
     ;; list of <dmd-service> provided by the extensions.
     (extend concatenate)
     (extensions (list (service-extension (target boot-service-type)
                                          (compute dmd-boot-gexp))))))

  (define guix-service-type
    (service-type
     (name 'guix)
     (extensions
      (list (service-extension (target dmd-root-service-type)
                               (compute guix-dmd-service))
            (service-extension (target account-service-type)
                               (compute guix-accounts))
            (service-extension (target activation-service-type)
                               (compute guix-activation))))))

The service procedures used in OS declarations can usually remain
unchanged, written like this:

  (define* (guix-service #:optional (config %default-guix-configuration))
    (service (type guix-service-type)
             (parameters config)))

There can be several services of a given type.  However, there must be
only one service of a type that can be extended–for instance, there can
be only one service of type ‘dmd-root-service-type’ or
‘etc-service-type’.

The ‘fold-services’ procedure is passed a list of services, propagates
extensions, and returns the root service (typically ‘%boot-service’)
with its ‘parameters’ field changed accordingly.

I’m quite happy with the result, but comments are welcome!  I’ll convert
some more services to see how it goes.

Ludo’.

Re: Service refactoring

[Andy Wingo]

On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:

> I’m quite happy with the result, but comments are welcome!  I’ll convert
> some more services to see how it goes.

Neat!  Sounds great.  One question: there are some services like colord
or geoclue that don't need to be managed by DMD, but are just declared
as services so that their users are created, there /var/foo directories
are created, etc.  The can be started by D-Bus as needed.  Does the new
design support services of this kind?

A

Re: Service refactoring

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>
>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>> some more services to see how it goes.
>
> Neat!  Sounds great.  One question: there are some services like colord
> or geoclue that don't need to be managed by DMD, but are just declared
> as services so that their users are created, there /var/foo directories
> are created, etc.  The can be started by D-Bus as needed.  Does the new
> design support services of this kind?

Yes it does.

‘colord’ will “extend” the D-Bus service by giving it its package object
(thus its .service files), it will extend the activation service by
providing it #~(mkdir "/var/foo"), and it will extend the account
service by passing it its user accounts/groups.

We’re getting there.  :-)

Ludo’.

Re: Service refactoring

[Thompson, David]

On Mon, Sep 21, 2015 at 12:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Andy Wingo <wingo@igalia.com> skribis:
>
>> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>>> some more services to see how it goes.
>>
>> Neat!  Sounds great.  One question: there are some services like colord
>> or geoclue that don't need to be managed by DMD, but are just declared
>> as services so that their users are created, there /var/foo directories
>> are created, etc.  The can be started by D-Bus as needed.  Does the new
>> design support services of this kind?
>
> Yes it does.
>
> ‘colord’ will “extend” the D-Bus service by giving it its package object
> (thus its .service files), it will extend the activation service by
> providing it #~(mkdir "/var/foo"), and it will extend the account
> service by passing it its user accounts/groups.

I'm picturing an enhanced 'nginx-service' that is able to accept site
configuration files from other services (say, a MediaGoblin service)
and it sounds like a great feature.  Now, is the following scenario
possible?: 'mediagoblin-service' can extend both 'apache-service' and
'nginx-service', and when the system configuration is realized, the
web service the user has chosen is extended.

This is really great stuff.  :)

- Dave

Re: Service refactoring

[Ludovic Courtès]

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Mon, Sep 21, 2015 at 12:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Andy Wingo <wingo@igalia.com> skribis:
>>
>>> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>>>
>>>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>>>> some more services to see how it goes.
>>>
>>> Neat!  Sounds great.  One question: there are some services like colord
>>> or geoclue that don't need to be managed by DMD, but are just declared
>>> as services so that their users are created, there /var/foo directories
>>> are created, etc.  The can be started by D-Bus as needed.  Does the new
>>> design support services of this kind?
>>
>> Yes it does.
>>
>> ‘colord’ will “extend” the D-Bus service by giving it its package object
>> (thus its .service files), it will extend the activation service by
>> providing it #~(mkdir "/var/foo"), and it will extend the account
>> service by passing it its user accounts/groups.
>
> I'm picturing an enhanced 'nginx-service' that is able to accept site
> configuration files from other services (say, a MediaGoblin service)
> and it sounds like a great feature.

Yes, that’s the idea.

> Now, is the following scenario possible?: 'mediagoblin-service' can
> extend both 'apache-service' and 'nginx-service', and when the system
> configuration is realized, the web service the user has chosen is
> extended.

If it makes sense, there could be a single ‘web-service-type’ that both
Apache httpd and NGINX would implement.  The MediaGoblin service would
extend them by providing something like a vhost specification I suppose?

Now, perhaps we would need separate ‘httpd-service-type’ and
‘nginx-service-type’ if httpd and NGINX happen to be extensible in
different ways.  I don’t really know these tools, but it would be good
to see how we would do it.  WDYT?

Ludo’.

Re: Service refactoring

[宋文武]

2015-09-24 15:41 GMT+08:00 Ludovic Courtès <ludo@gnu.org>:
> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
>> On Mon, Sep 21, 2015 at 12:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Andy Wingo <wingo@igalia.com> skribis:
>>>
>>>> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>>>>
>>>>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>>>>> some more services to see how it goes.
>>>>
>>>> Neat!  Sounds great.  One question: there are some services like colord
>>>> or geoclue that don't need to be managed by DMD, but are just declared
>>>> as services so that their users are created, there /var/foo directories
>>>> are created, etc.  The can be started by D-Bus as needed.  Does the new
>>>> design support services of this kind?
>>>
>>> Yes it does.
Great news! I don't miss NixOS's modules anymore :-)

I think the difference is that we don't seperate options and implementations
like NixOS did,  'service-type' have both in one place.
So, our services is highly tie to gexps and dmd (that's ok).
>>> [...]

Re: Service refactoring

[Ludovic Courtès]

宋文武 <iyzsong@gmail.com> skribis:

> 2015-09-24 15:41 GMT+08:00 Ludovic Courtès <ludo@gnu.org>:
>> "Thompson, David" <dthompson2@worcester.edu> skribis:
>>
>>> On Mon, Sep 21, 2015 at 12:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>>> Andy Wingo <wingo@igalia.com> skribis:
>>>>
>>>>> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>>>>>
>>>>>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>>>>>> some more services to see how it goes.
>>>>>
>>>>> Neat!  Sounds great.  One question: there are some services like colord
>>>>> or geoclue that don't need to be managed by DMD, but are just declared
>>>>> as services so that their users are created, there /var/foo directories
>>>>> are created, etc.  The can be started by D-Bus as needed.  Does the new
>>>>> design support services of this kind?
>>>>
>>>> Yes it does.
> Great news! I don't miss NixOS's modules anymore :-)

Heheh, NixOS modules are a very good competitor.  ;-)

I think this model handles multiple service instances better than NixOS
modules.  NixOS services can be enabled/disabled, for instance with
‘services.httpd.enable = true’ or ‘services.openssh.enable = true’, but
I don’t think there’s a general mechanism to have multiple instances of
httpd or sshd.

Also in NixOS any service can do ‘foo.bar = baz’ and change an unrelated
OS setting, which is not possible here.

I find the fixed-point approach of NixOS modules very elegant, but it
also complicates things: It’s easy to enter an infinite recursion, and
one has to carefully use ‘mkIf’ and similar constructs where it matters,
which is non-obvious.

> I think the difference is that we don't seperate options and implementations
> like NixOS did,  'service-type' have both in one place.

‘service-type’ is sort-of an interface: it describes how services are
composed, but not what they do.

> So, our services is highly tie to gexps and dmd (that's ok).

The whole thing is rather tied to gexps, but it’s not tied to dmd.  dmd
is just a service that has nothing special, and (gnu system) does not
embed any knowledge about dmd.

The only service type that’s known to (gnu system) is
‘boot-service-type’.

Thanks for your feedback!

Ludo’.

Re: Service refactoring

[Christopher Allan Webber]

Ludovic Courtès writes:

> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
>> On Mon, Sep 21, 2015 at 12:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Andy Wingo <wingo@igalia.com> skribis:
>>>
>>>> On Sun 20 Sep 2015 15:42, ludo@gnu.org (Ludovic Courtès) writes:
>>>>
>>>>> I’m quite happy with the result, but comments are welcome!  I’ll convert
>>>>> some more services to see how it goes.
>>>>
>>>> Neat!  Sounds great.  One question: there are some services like colord
>>>> or geoclue that don't need to be managed by DMD, but are just declared
>>>> as services so that their users are created, there /var/foo directories
>>>> are created, etc.  The can be started by D-Bus as needed.  Does the new
>>>> design support services of this kind?
>>>
>>> Yes it does.
>>>
>>> ‘colord’ will “extend” the D-Bus service by giving it its package object
>>> (thus its .service files), it will extend the activation service by
>>> providing it #~(mkdir "/var/foo"), and it will extend the account
>>> service by passing it its user accounts/groups.
>>
>> I'm picturing an enhanced 'nginx-service' that is able to accept site
>> configuration files from other services (say, a MediaGoblin service)
>> and it sounds like a great feature.
>
> Yes, that’s the idea.

You've already heard me say it elsewhere, but I'm suuuuuuper excited
about this.  I've already written about it on the pumpiverse, so for
posterity, here's that link:

  https://identi.ca/cwebber/note/vNnKxmgtQBOO2W82r0ZgwA

Glad to see mediagoblin-service being used as an example too :)

>> Now, is the following scenario possible?: 'mediagoblin-service' can
>> extend both 'apache-service' and 'nginx-service', and when the system
>> configuration is realized, the web service the user has chosen is
>> extended.
>
> If it makes sense, there could be a single ‘web-service-type’ that both
> Apache httpd and NGINX would implement.  The MediaGoblin service would
> extend them by providing something like a vhost specification I suppose?

I think that's right.

> Now, perhaps we would need separate ‘httpd-service-type’ and
> ‘nginx-service-type’ if httpd and NGINX happen to be extensible in
> different ways.  I don’t really know these tools, but it would be good
> to see how we would do it.  WDYT?
>
> Ludo’.

So, nginx and apache have pretty drastically different config file
types.  However, most of the things users want I think are compatible
with both.  We probably want a "minimal" service that just accepts
whatever config files for apache and nginx, and just lets the user
specify that whole manually.  We can have more advanced procedures for
building up a config file by translating our world-of-s-expressions or
whatever schemey types into the config file format... but given "just
how much" you can do with nginx and apache that is just so wildly
different between the two, we probably want something that provides an
abastract representation of what most web applications need, some sort
of "medium common denominator" (I don't think we need to shoot for
lowest here!).  That way we can have a general way of letting
applications specify what they need from their web server, but also
allow a sysadmin to go "full on manual" if they need to with the config
file formats.

Dave / Ludo: Does that fit with what you two are thinking as well?

 - cwebb

Re: Service refactoring

[Ludovic Courtès]

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> So, nginx and apache have pretty drastically different config file
> types.  However, most of the things users want I think are compatible
> with both.  We probably want a "minimal" service that just accepts
> whatever config files for apache and nginx, and just lets the user
> specify that whole manually.  We can have more advanced procedures for
> building up a config file by translating our world-of-s-expressions or
> whatever schemey types into the config file format... but given "just
> how much" you can do with nginx and apache that is just so wildly
> different between the two, we probably want something that provides an
> abastract representation of what most web applications need, some sort
> of "medium common denominator" (I don't think we need to shoot for
> lowest here!).  That way we can have a general way of letting
> applications specify what they need from their web server, but also
> allow a sysadmin to go "full on manual" if they need to with the config
> file formats.
>
> Dave / Ludo: Does that fit with what you two are thinking as well?

Yes, sounds good.  We just need to figure out what that common
denominator is.  :-)

Thanks,
Ludo’.

Re: Service refactoring

[Ludovic Courtès]

I’ve entered the polishing and documentation phase of
‘wip-service-refactor’, with the hope of merging it within a few days.

Currently the branch should have no loss of functionality compared to
‘master’.  You’re welcome to give it a try!

Recently I’ve removed the controversial ‘define-record-type†’ unhygienic
macro that I had introduced at a time when I was tired of typing.  ;-)
Overall the patch adds many lines, mostly because services now need to
have all their parameters in a single object.  So we have records like
<static-networking-configuration>, <geoclue-configuration>, and so on.

Ludo’.

Re: Service refactoring

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> I’ve entered the polishing and documentation phase of
> ‘wip-service-refactor’, with the hope of merging it within a few days.

Damn, it took longer than expected, but now it’s merged!

So now is the time to report bugs etc.  :-)

The documentation is pretty long, but still with missing bits here and
there, mostly so that readers have a chance to discover by themselves.
;-)

Ludo’.