Defining user services in Guix.

Defining user services in Guix.

[Mathieu Othacehe]

Hi Guix,

I just a wrote a small service for redshift¹. Redshift is a program
adjusting color temperature of the screen according to surroundings.

This program needs to communicate to X, so DISPLAY and XAUTHORITY must
be set correctly.

The only way I found to make my service work is something very hacky:

--8<---------------cut here---------------start------------->8---
(start #~(make-forkexec-constructor
          #$redshift-command
          #:user "mathieu"
          #:group "users"
          #:environment-variables
          '("DISPLAY=:0"
            "XAUTHORITY=/home/mathieu/.Xauthority")))
--8<---------------cut here---------------end--------------->8---

Is there a better way to define this kind of "user" service in Guix ?

Thanks,

Mathieu

¹: http://jonls.dk/redshift/

Re: Defining user services in Guix.

[Danny Milosavljevic]

Hi Mathieu,

personally, I'd just start it inside ~/.xinitrc, ~/.xsession, ~/.fluxbox/startup or similar.  That's a little old-school, though, but it's how I always did those things.  The disadvantage is that if redshift crashes, it's not being brought back up.

But you can start your own shepherd under your user account and have that shepherd manage user services - it's nice.

Create a file ${XDG_CONFIG_HOME-.config}/shepherd/init.scm and put your services there:

(use-modules (shepherd service))

(define redshift (make <service> #:provides '(redshift) #:start ...))
(register-services redshift) ; Note: if there has already been a (register-services) form, replace it

You can then launch your own shepherd:

mathieu@xxx $ shepherd

And manage it using herd:

mathieu@xxx $ herd start redshift

You can also load files at runtime using:

$ herd load shepherd ~/bla.scm

See also

$ info shepherd

Unfortunately, it still doesn't respawn redshift if it dies - it seems to require a pid file for that (#:start (make-forkexec-constructor ... #:pid-file "xxx.pid") and you have to specify #:respawn? #t.  The invoked program then has to create "xxx.pid").  I think it would be a nice addition to have it monitor the process that make-forkexec-constructor invoked - without any pid file.  Maybe it's even a bug...

To reproduce:

dannym@dayas ~/.config/shepherd$ cat init.scm 

(use-modules (shepherd service))

(define redshift
  (make <service>
        #:provides '(redshift)
        #:start (make-forkexec-constructor
                 '("/run/current-system/profile/bin/xterm") ; to make it more obvious
)
        #:stop (make-kill-destructor)
        #:respawn? #t))
(register-services redshift)

;; Services known to shepherd:
;; Add new services (defined using 'make <service>') to shepherd here by
;; providing them as arguments to 'register-services'.
;(register-services)

;; Send shepherd into the background
(action 'shepherd 'daemonize)

;; Services to start when shepherd starts:
;; Add the name of each service that should be started to the list
;; below passed to 'for-each'.
(for-each start '())

dannym@dayas ~/.config/shepherd$ shepherd
dannym@dayas ~/.config/shepherd$ herd start redshift
<xterm pops up>
<exit it>
<xterm doesn't pop up again... bug?>

At first I thought that was because a regular exit is not bad - so I did instead:
dannym@dayas ~/.config/shepherd$ herd start redshift
<xterm pops up>
dannym@dayas ~/.config/shepherd$ herd status redshift
<note pid>
dannym@dayas ~/.config/shepherd$ kill -9 <pid>
<xterm doesn't pop up again... bug?>

On the plus side, you have access to the correct DISPLAY and XAUTHORITY just fine there.

Re: Defining user services in Guix.

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> personally, I'd just start it inside ~/.xinitrc, ~/.xsession, ~/.fluxbox/startup or similar.  That's a little old-school, though, but it's how I always did those things.  The disadvantage is that if redshift crashes, it's not being brought back up.
>
> But you can start your own shepherd under your user account and have that shepherd manage user services - it's nice.

That’s exactly what I do.

Surely we could do better, like providing unprivileged users with a tool
akin to ‘guix system reconfigure’ but to manage their own services.
This came up several times in the past.

Ideas?

Ludo’.

Re: Defining user services in Guix.

[Mathieu Othacehe]

Hi guys,

>> But you can start your own shepherd under your user account and have that shepherd manage user services - it's nice.

Thank you Danny for the details procedure. I'll use that for now !

> Surely we could do better, like providing unprivileged users with a tool
> akin to ‘guix system reconfigure’ but to manage their own services.
> This came up several times in the past.

Well, I think it's too bad that user services can't be handled as easily
as the other services in Guix.

I would be eager to participate to some sort of guix system reconfigure
--user-services that would allow people to write and share their
unpriviledged services :).

Mathieu

Re: Defining user services in Guix.

[Mekeor Melire]

Mathieu Othacehe <m.othacehe@gmail.com> writes:

>>> But you can start your own shepherd under your user account and have that shepherd manage user services - it's nice.
>
> Thank you Danny for the details procedure. I'll use that for now !
>
>> Surely we could do better, like providing unprivileged users with a tool
>> akin to ‘guix system reconfigure’ but to manage their own services.
>> This came up several times in the past.
>
> Well, I think it's too bad that user services can't be handled as easily
> as the other services in Guix.
>
> I would be eager to participate to some sort of guix system reconfigure
> --user-services that would allow people to write and share their
> unpriviledged services :).

Btw, beside user-specific services, we also already discussed
user-specific aliases for the `guix ...` commands. So now, there are at
least two things a user-specific configuration file could contain. :)

One, the system.scm would contain

     (operating-system
        ; ...
        (services
            ; ...
        ))

And secondly, each user could have a user.scm e.g. like

    (user-configuration
        ; ...
        (aliases
            '(
                ("sysrec" "system reconfigure")
                ("pl"     "pull")
                ; ...
            ))
        (services
            (emacs-daemon-service)
            (dzen-service)
            ; ...
            ))

or similar :)

--
mekeor ~ EDD3 DFFA 76F6 11C0 145F 9A99 AC85 BAD8 A2F8 C868

Re: Defining user services in Guix.

[Ricardo Wurmus]

Mekeor Melire <mekeor.melire@gmail.com> writes:

> Mathieu Othacehe <m.othacehe@gmail.com> writes:
>
>>>> But you can start your own shepherd under your user account and have that shepherd manage user services - it's nice.
>>
>> Thank you Danny for the details procedure. I'll use that for now !
>>
>>> Surely we could do better, like providing unprivileged users with a tool
>>> akin to ‘guix system reconfigure’ but to manage their own services.
>>> This came up several times in the past.
>>
>> Well, I think it's too bad that user services can't be handled as easily
>> as the other services in Guix.
>>
>> I would be eager to participate to some sort of guix system reconfigure
>> --user-services that would allow people to write and share their
>> unpriviledged services :).
>
> Btw, beside user-specific services, we also already discussed
> user-specific aliases for the `guix ...` commands. So now, there are at
> least two things a user-specific configuration file could contain. :)
>
> One, the system.scm would contain
>
>      (operating-system
>         ; ...
>         (services
>             ; ...
>         ))
>
> And secondly, each user could have a user.scm e.g. like
>
>     (user-configuration
>         ; ...
>         (aliases
>             '(
>                 ("sysrec" "system reconfigure")
>                 ("pl"     "pull")
>                 ; ...
>             ))
>         (services
>             (emacs-daemon-service)
>             (dzen-service)
>             ; ...
>             ))
>
> or similar :)

While I’m not a fan of Guix-specific aliases, I think it’s not a bad
idea to use Guix for user configurations.  I’m tired of having to keep
track of a set of custom configuration files scattered all over the
place, but I’m not sure how to approach this without going overboard.

To stay on the topic of user services: I think they do have a place in
Guix and could be defined with the same mechanisms that we have for
system services.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Defining user services in Guix.

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Mekeor Melire <mekeor.melire@gmail.com> writes:

[...]

>> And secondly, each user could have a user.scm e.g. like
>>
>>     (user-configuration
>>         ; ...
>>         (aliases
>>             '(
>>                 ("sysrec" "system reconfigure")
>>                 ("pl"     "pull")
>>                 ; ...
>>             ))
>>         (services
>>             (emacs-daemon-service)
>>             (dzen-service)
>>             ; ...
>>             ))
>>
>> or similar :)
>
> While I’m not a fan of Guix-specific aliases, I think it’s not a bad
> idea to use Guix for user configurations.  I’m tired of having to keep
> track of a set of custom configuration files scattered all over the
> place, but I’m not sure how to approach this without going overboard.
>
> To stay on the topic of user services: I think they do have a place in
> Guix and could be defined with the same mechanisms that we have for
> system services.

Agreed.  A ‘guix user’ tool (or similar) could talk to the user’s
Shepherd instance to upgrade user services like
‘upgrade-shepherd-services’ does in (guix scripts system).

For non-Shepherd things, the benefits are less obvious to me.  For
instance, having ~/.bashrc be a simply to /gnu/store/…-bashrc is not
significantly better to having ~/.bashrc link to
~/directory-under-version-control/bashrc.

Anyway, food for thought!

Ludo’.

Re: Defining user services in Guix.

[Mathieu Othacehe]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]


Hi,

> Agreed.  A ‘guix user’ tool (or similar) could talk to the user’s
> Shepherd instance to upgrade user services like
> ‘upgrade-shepherd-services’ does in (guix scripts system).

With the attached patch, it is possibe to use (gnu services herd) to
interact with a user shepherd instance.

For instance :

--8<---------------cut here---------------start------------->8---
(use-modules (gnu services herd))
(parameterize
 ((%shepherd-socket-file "/home/mathieu/.config/shepherd/run/socket"))
 (current-services))
--8<---------------cut here---------------end--------------->8---

Which returns :

--8<---------------cut here---------------start------------->8---
$1 = (#<<live-service> provision: (redshift) requirement: () running: 3632> #<<live-service> provision: (root shepherd) requirement: () running: #t>)
--8<---------------cut here---------------end--------------->8---

Based on that, I'll try to define a guix user command to upgrade user
services if it is ok for everyone :).

Thanks,

Mathieu

[-- Attachment #2: 0001-services-herd-Make-shepherd-socket-file-a-parameter-.patch --]
[-- Type: text/x-patch, Size: 1598 bytes --]

From caf7d9f5a7875663623b24662c1c23b99e60c5ad Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <m.othacehe@gmail.com>
Date: Fri, 28 Apr 2017 17:09:53 +0200
Subject: [PATCH] services: herd: Make %shepherd-socket-file a parameter and
 export it.

* gnu/services/herd.scm (%shepherd-socket-file): Make it an exported
  parameter.
(open-connection): Adapt.
---
 gnu/services/herd.scm | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm
index 03bfbf1d7..f8d60a480 100644
--- a/gnu/services/herd.scm
+++ b/gnu/services/herd.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -23,7 +24,9 @@
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
-  #:export (shepherd-error?
+  #:export (%shepherd-socket-file
+
+            shepherd-error?
             service-not-found-error?
             service-not-found-error-service
             action-not-found-error?
@@ -58,9 +61,9 @@
 ;;; Code:
 
 (define %shepherd-socket-file
-  "/var/run/shepherd/socket")
+  (make-parameter "/var/run/shepherd/socket"))
 
-(define* (open-connection #:optional (file %shepherd-socket-file))
+(define* (open-connection #:optional (file (%shepherd-socket-file)))
   "Open a connection to the daemon, using the Unix-domain socket at FILE, and
 return the socket."
   ;; The protocol is sexp-based and UTF-8-encoded.
-- 
2.12.2

Re: Defining user services in Guix.

[Ludovic Courtès]

Hello Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

>> Agreed.  A ‘guix user’ tool (or similar) could talk to the user’s
>> Shepherd instance to upgrade user services like
>> ‘upgrade-shepherd-services’ does in (guix scripts system).
>
> With the attached patch, it is possibe to use (gnu services herd) to
> interact with a user shepherd instance.
>
> For instance :
>
> (use-modules (gnu services herd))
> (parameterize
>  ((%shepherd-socket-file "/home/mathieu/.config/shepherd/run/socket"))
>  (current-services))
>
>
> Which returns :
>
> $1 = (#<<live-service> provision: (redshift) requirement: () running: 3632> #<<live-service> provision: (root shepherd) requirement: () running: #t>)
>
> Based on that, I'll try to define a guix user command to upgrade user
> services if it is ok for everyone :).

Neat, sounds like a good plan!

Could you create an account on Savannah, add the OpenPGP public key
you’ll use to sign commits there, and let us know when you’re done with
a signed message?  Looks like we should be able to increase the
bandwidth.  :-)

Thanks,
Ludo’.

Re: Defining user services in Guix.

[Mathieu Othacehe]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]


Hi Ludo,

> Neat, sounds like a good plan!

Great !

> Could you create an account on Savannah, add the OpenPGP public key
> you’ll use to sign commits there, and let us know when you’re done with
> a signed message?  Looks like we should be able to increase the
> bandwidth.  :-)

Thanks :) I created the account "mothacehe" on Savannah and signed the
email with the uploaded OpenPGP key. 

Mathieu

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Defining user services in Guix.

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

>> Could you create an account on Savannah, add the OpenPGP public key
>> you’ll use to sign commits there, and let us know when you’re done with
>> a signed message?  Looks like we should be able to increase the
>> bandwidth.  :-)
>
> Thanks :) I created the account "mothacehe" on Savannah and signed the
> email with the uploaded OpenPGP key. 

Awesome.  I’ve added you to the project, welcome aboard!

Please read the “Commit Access” section in ‘HACKING’ and let us know if
you have any questions.

Happy hacking!  :-)

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Defining user services in Guix.

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Mathieu Othacehe <m.othacehe@gmail.com> skribis:
>
>>> Could you create an account on Savannah, add the OpenPGP public key
>>> you’ll use to sign commits there, and let us know when you’re done with
>>> a signed message?  Looks like we should be able to increase the
>>> bandwidth.  :-)
>>
>> Thanks :) I created the account "mothacehe" on Savannah and signed the
>> email with the uploaded OpenPGP key. 
>
> Awesome.  I’ve added you to the project, welcome aboard!
>
> Please read the “Commit Access” section in ‘HACKING’ and let us know if
> you have any questions.
>
> Happy hacking!  :-)
>
> Ludo’.

Yay!  Welcome Mathieu!

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Defining user services in Guix.

[Mathieu Othacehe]

>> Happy hacking!  :-)

>> Ludo’.
>
> Yay!  Welcome Mathieu!

Thanks Ricardo and Ludo !

Pushed as 5e03b122cf295d24e01861ca942505d47f9aa59e.

Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Danny Milosavljevic]

Hi,

I'm revisiting this topic because I want to have a dbus user bus to be available as "${XDG_RUNTIME_DIR}/bus".

> > With the attached patch, it is possibe to use (gnu services herd) to
> > interact with a user shepherd instance.

Nice!  It works!

But while testing it I noticed:

For a real user shepherd, it would be nice if when I logged in twice using the same user account (without logging out in-between - so resulting in two sessions of that user) it would still only have one shepherd instance for that user account in total - and if that instance (and remaining user processes for that matter) went away only when I logged out from *all* sessions of that user.

I thought I could fake that by just trying to invoke shepherd on each session start and have it fail on the second attempt - but apparently we will happily start an infinite number of shepherds for one user.  Is that on purpose?

The relevant place (in shepherd) is:

(define (open-server-socket file-name)
  "Open a socket at FILE-NAME, and listen for connections there."
  (with-fluids ((%default-port-encoding "UTF-8"))
    (let ((sock    (socket PF_UNIX SOCK_STREAM 0))
          (address (make-socket-address AF_UNIX file-name)))
      (false-if-exception (delete-file file-name)) ;  <===== [dannym: WTF.  Would it be better to try to connect first?]
      (bind sock address)
      (listen sock 10)
      sock)))

Probably not good.

Maybe better:

(define (server-present? file-name)
  "Open a socket at FILE-NAME, and connect to the server, if any.  Return #t if that worked."
  (with-fluids ((%default-port-encoding "UTF-8"))
    (let ((sock    (socket PF_UNIX SOCK_STREAM 0))
          (address (make-socket-address AF_UNIX file-name)))
      (false-if-exception (connect sock address))))) ; probably missing a "[catch] close".  How to do that best?

(define (open-server-socket file-name)
  "Open a socket at FILE-NAME, and listen for connections there."
  (if (server-present? file-name)
      (exit 0) ; User's shepherd is already running, so let it do its work.
  (with-fluids ((%default-port-encoding "UTF-8"))
    (let ((sock    (socket PF_UNIX SOCK_STREAM 0))
          (address (make-socket-address AF_UNIX file-name)))
      (false-if-exception (delete-file file-name)) ; we could get rid of that if we put the shepherd socket into /run/user/xxx (i.e. XDG_RUNTIME_DIR) instead, because that's on a tmpfs.  That's also where it's supposed to go according to the standard.
      (bind sock address)
      (listen sock 10)
      sock)))

elogind already sends a message to the dbus system bus whenever a user really stops or starts (all their sessions closed, one session opened, respectively):

p = "/org/freedesktop/login1/user/_"UID_FMT;
sd_bus_emit_signal(     u->manager->bus,
                        "/org/freedesktop/login1",
                        "org.freedesktop.login1.Manager",
                        new_user ? "UserNew" : "UserRemoved",
                        "uo", (uint32_t) u->uid, p);

I'm trying to find the right place to insert my "dbus-daemon" invocation for providing the user bus (note: not session bus)...

At first I thought of hard-coding the "dbus-daemon" invocation somewhere but then I thought the option with the most user freedom would be to only invoke the user shepherd [and nothing else] when the user logs in - and if he has a user service launching "dbus-daemon", good for him...

Should we make a system shepherd service that invokes the user shepherd service on behalf of users?  Would that be that safe?

Or should we just expect the user to put a (shepherd with fix) invocation into their HOME startup scripts like .xinitrc ?  Note that if we did that there's some session-specific stuff in the session's environment that shepherd will inherit.  Probably not that bad if invoked early enough.

Re: Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Mathieu Othacehe]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]


Hi Danny,

> Or should we just expect the user to put a (shepherd with fix)
> invocation into their HOME startup scripts like .xinitrc ?

I wrote a first draft of user services a month ago. The idea here is
that guix user -r user-manifest.scm generates a script that lauches a
user shepherd.

For instance with the following user-manifest.scm :

--8<---------------cut here---------------start------------->8---
(define (redshift-service config)
  (list (shepherd-service
         (documentation "Run redshift.")
         (provision '(redshift-test))
         (requirement '())
         (start #~(make-forkexec-constructor
                   (list (string-append #$redshift "/bin/redshift")
                         "-l" "48:2")))
         (stop  #~(make-kill-destructor)))))

(define redshift-service-type
  (service-type
   (name 'test-user)
   (extensions
    (list
     (service-extension shepherd-user-service-type
                        test-shepherd-service)))))

(user-configuration
 (services (list (service redshift-service-type #f))))
--8<---------------cut here---------------end--------------->8---

I get a script that lauches shepherd himself starting redshift.

The plan here was to add a symlink, (don't know where !), pointing to
the last generated shepherd script, and have the user start shepherd by
executing the script pointed by the symlink in his .xinitrc for
instance.

> Note that if we did that there's some session-specific stuff in the session's environment that shepherd will inherit.  Probably not that bad if invoked early enough.

Starting shepherd there ensures to have DISPLAY, XAUTHORITY and other
variables that user services may use (like redshift).

I attached my draft patch.

Thanks,

Mathieu

[-- Attachment #2: 0001-user-services.patch --]
[-- Type: text/x-patch, Size: 11488 bytes --]

From 1d02fd18b187bb5c8fae8413116a7608eb7e5088 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <m.othacehe@gmail.com>
Date: Mon, 1 May 2017 16:22:23 +0200
Subject: [PATCH] user services.

---
 Makefile.am               |   1 +
 gnu/services.scm          |   5 ++
 gnu/services/shepherd.scm |  70 +++++++++++++++++++++-----
 gnu/system.scm            |   9 ++++
 guix/scripts/user.scm     | 125 ++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 197 insertions(+), 13 deletions(-)
 create mode 100644 guix/scripts/user.scm

diff --git a/Makefile.am b/Makefile.am
index 8fe9e350c..7a87f548a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -165,6 +165,7 @@ MODULES =					\
   guix/scripts/publish.scm			\
   guix/scripts/edit.scm				\
   guix/scripts/size.scm				\
+  guix/scripts/user.scm				\
   guix/scripts/graph.scm			\
   guix/scripts/container.scm			\
   guix/scripts/container/exec.scm		\
diff --git a/gnu/services.scm b/gnu/services.scm
index 5c314748d..08b595a60 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -73,6 +73,7 @@
             ambiguous-target-service-error-target-type
 
             system-service-type
+            user-service-type
             boot-service-type
             cleanup-service-type
             activation-service-type
@@ -281,6 +282,10 @@ containing the given entries."
                 (compose identity)
                 (extend system-derivation)))
 
+(define user-service-type
+  (service-type (name 'user)
+                (extensions '())))
+
 (define (compute-boot-script _ mexps)
   (mlet %store-monad ((gexps (sequence %store-monad mexps)))
     (gexp->file "boot"
diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm
index 7281746ab..787d8b2b0 100644
--- a/gnu/services/shepherd.scm
+++ b/gnu/services/shepherd.scm
@@ -35,7 +35,11 @@
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
   #:export (shepherd-root-service-type
+            shepherd-user-service-type
+
             %shepherd-root-service
+            %shepherd-user-root-service
+
             shepherd-service-type
 
             shepherd-service
@@ -86,6 +90,14 @@
                 (execl #$(file-append shepherd "/bin/shepherd")
                        "shepherd" "--config" #$shepherd-conf)))))
 
+(define (shepherd-user-gexp _ services)
+  (mlet %store-monad ((shepherd-conf
+                       (shepherd-user-configuration-file services)))
+    (return #~(begin
+                ;; Start shepherd.
+                (execl #$(file-append shepherd "/bin/shepherd")
+                       "shepherd" "--config" #$shepherd-conf)))))
+
 (define shepherd-root-service-type
   (service-type
    (name 'shepherd-root)
@@ -98,11 +110,21 @@
                      (service-extension profile-service-type
                                         (const (list shepherd)))))))
 
+(define shepherd-user-service-type
+  (service-type
+   (name 'shepherd-user)
+   (compose concatenate)
+   (extend shepherd-user-gexp)
+   (extensions (list (service-extension user-service-type (const #t))))))
+
 (define %shepherd-root-service
   ;; The root shepherd service, aka. PID 1.  Its parameter is a list of
   ;; <shepherd-service> objects.
   (service shepherd-root-service-type '()))
 
+(define %shepherd-user-root-service
+  (service shepherd-user-service-type #f))
+
 (define-syntax-rule (shepherd-service-type service-name proc)
   "Return a <service-type> denoting a simple shepherd service--i.e., the type
 for a service that extends SHEPHERD-ROOT-SERVICE-TYPE and nothing else."
@@ -216,6 +238,22 @@ stored."
                       #:start #$(shepherd-service-start service)
                       #:stop #$(shepherd-service-stop service))))))
 
+(define (shepherd-start-services services)
+  #~(for-each
+     (lambda (service)
+       ;; In the Shepherd 0.3 the 'start' method can raise
+       ;; '&action-runtime-error' if it fails, so protect
+       ;; against it.  (XXX: 'action-runtime-error?' is not
+       ;; exported is 0.3, hence 'service-error?'.)
+       (guard (c ((service-error? c)
+                  (format (current-error-port)
+                          "failed to start service '~a'~%"
+                          service)))
+         (start service)))
+     '#$(append-map shepherd-service-provision
+                    (filter shepherd-service-auto-start?
+                            services))))
+
 (define (shepherd-configuration-file services)
   "Return the shepherd configuration file for SERVICES."
   (assert-valid-graph services)
@@ -238,19 +276,25 @@ stored."
               (setenv "PATH" "/run/current-system/profile/bin")
 
               (format #t "starting services...~%")
-              (for-each (lambda (service)
-                          ;; In the Shepherd 0.3 the 'start' method can raise
-                          ;; '&action-runtime-error' if it fails, so protect
-                          ;; against it.  (XXX: 'action-runtime-error?' is not
-                          ;; exported is 0.3, hence 'service-error?'.)
-                          (guard (c ((service-error? c)
-                                     (format (current-error-port)
-                                             "failed to start service '~a'~%"
-                                             service)))
-                            (start service)))
-                        '#$(append-map shepherd-service-provision
-                                       (filter shepherd-service-auto-start?
-                                               services)))))))
+              #$(shepherd-start-services services)))))
+
+    (gexp->file "shepherd.conf" config)))
+
+(define (shepherd-user-configuration-file services)
+  "Return the shepherd configuration file for SERVICES."
+  (assert-valid-graph services)
+
+  (mlet %store-monad ((files (mapm %store-monad
+                                   shepherd-service-file services)))
+    (define config
+      #~(begin
+          (use-modules (srfi srfi-34)
+                       (system repl error-handling))
+
+          ;; (action 'shepherd 'daemonize)
+
+          (apply register-services (map primitive-load '#$files))
+          #$(shepherd-start-services services)))
 
     (gexp->file "shepherd.conf" config)))
 
diff --git a/gnu/system.scm b/gnu/system.scm
index a35a416cb..dd69e31aa 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -65,6 +65,10 @@
   #:export (operating-system
             operating-system?
 
+            user-configuration
+            user-configuration?
+            user-configuration-services
+
             operating-system-bootloader
             operating-system-services
             operating-system-user-services
@@ -182,6 +186,11 @@
   (sudoers-file operating-system-sudoers-file     ; file-like
                 (default %sudoers-specification)))
 
+(define-record-type* <user-configuration> user-configuration
+  make-user-configuration
+  user-configuration?
+  (services user-configuration-services))
+
 \f
 ;;;
 ;;; Services.
diff --git a/guix/scripts/user.scm b/guix/scripts/user.scm
new file mode 100644
index 000000000..1ee3f9535
--- /dev/null
+++ b/guix/scripts/user.scm
@@ -0,0 +1,125 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts user)
+  #:use-module (gnu services)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system)
+  #:use-module (guix derivations)
+  #:use-module (guix records)
+  #:use-module (guix scripts)
+  #:use-module (guix scripts build)
+  #:use-module (guix store)
+  #:use-module (guix monads)
+  #:use-module (guix gexp)
+  #:use-module (guix ui)
+  #:use-module (guix utils)
+  #:use-module (ice-9 match)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-37)
+  #:export (guix-user))
+
+\f
+;;;
+;;; Command-line options.
+;;;
+
+(define %options
+  ;; Specifications of the command-line options.
+  (cons* (option '(#\h "help") #f #f
+                 (lambda args
+                   (show-help)
+                   (exit 0)))
+         (option '(#\r "reconfigure") #t #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'action `(reconfigure . ,arg) result)))
+         (option '(#\V "version") #f #f
+                 (lambda args
+                   (show-version-and-exit "guix user")))
+         %standard-build-options))
+
+(define %default-options
+  `((system . ,(%current-system))
+    (substitutes? . #t)
+    (graft? . #t)
+    (max-silent-time . 3600)
+    (verbosity . 0)))
+
+(define (show-help)
+  (display (G_ "Usage: guix user [OPTION]...
+Create a bundle of PACKAGE.\n"))
+  (display (G_ "
+  -r, --reconfigure-services=FILE  reconfigure services described in FILE"))
+  (newline)
+  (display (G_ "
+  -h, --help                       display this help and exit"))
+  (display (G_ "
+  -V, --version                    display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+\f
+;;;
+;;; User services.
+;;;
+
+(define (fold-user-services services)
+  (fold-services (cons* (service user-service-type #f)
+                        %shepherd-user-root-service
+                        services)
+                 #:target-type shepherd-user-service-type))
+
+(define (generate-sheperd-configuration services opts)
+  (mlet* %store-monad ((services -> (fold-user-services services))
+                       (shepherd-conf.drv (service-value services))
+                       (shepherd-launch (gexp->script "shepherd" shepherd-conf.drv))
+                       (drvs -> (list shepherd-launch)))
+    (mbegin %store-monad
+      (show-what-to-build* drvs
+                           #:use-substitutes?
+                           (assoc-ref opts 'substitutes?))
+      (built-derivations drvs)
+      (return (derivation->output-path shepherd-launch)))))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define %user-module
+  ;; Module in which the user configuration file is loaded.
+  (make-user-module '((gnu system)
+                      (gnu services))))
+
+(define (process-action store opts)
+  (let ((action (assoc-ref opts 'action)))
+    (match action
+      (('reconfigure . file)
+       (let* ((user-conf
+               (if file
+                   (load* file %user-module)
+                   (leave (G_ "no user configuration file specified~%"))))
+              (services (user-configuration-services user-conf)))
+         (format #t "~a\n" (run-with-store store
+                             (generate-sheperd-configuration services opts))))))))
+
+(define (guix-user . args)
+  (with-error-handling
+    (let ((opts  (parse-command-line args %options (list %default-options)))
+          (store (open-connection)))
+      (process-action store opts))))
-- 
2.13.1

Re: Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Ludovic Courtès]

Hello!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> I wrote a first draft of user services a month ago. The idea here is
> that guix user -r user-manifest.scm generates a script that lauches a
> user shepherd.
>
> For instance with the following user-manifest.scm :
>
> (define (redshift-service config)
>   (list (shepherd-service
>          (documentation "Run redshift.")
>          (provision '(redshift-test))
>          (requirement '())
>          (start #~(make-forkexec-constructor
>                    (list (string-append #$redshift "/bin/redshift")
>                          "-l" "48:2")))
>          (stop  #~(make-kill-destructor)))))
>
> (define redshift-service-type
>   (service-type
>    (name 'test-user)
>    (extensions
>     (list
>      (service-extension shepherd-user-service-type
>                         test-shepherd-service)))))
>
> (user-configuration
>  (services (list (service redshift-service-type #f))))
>
> I get a script that lauches shepherd himself starting redshift.

So you do “guix user user-manifest.scm” and it generates shepherd.conf
and spawns shepherd, right?  Sounds pretty cool!

> The plan here was to add a symlink, (don't know where !), pointing to
> the last generated shepherd script, and have the user start shepherd by
> executing the script pointed by the symlink in his .xinitrc for
> instance.

Maybe ~/.config/guix/services/start could be that symlink (and an
indirect GC root.)

Nice work!

Ludo’.

Re: Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Ludovic Courtès]

Hello!

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> For a real user shepherd, it would be nice if when I logged in twice using the same user account (without logging out in-between - so resulting in two sessions of that user) it would still only have one shepherd instance for that user account in total - and if that instance (and remaining user processes for that matter) went away only when I logged out from *all* sessions of that user.
>
> I thought I could fake that by just trying to invoke shepherd on each session start and have it fail on the second attempt - but apparently we will happily start an infinite number of shepherds for one user.  Is that on purpose?
>
> The relevant place (in shepherd) is:
>
> (define (open-server-socket file-name)
>   "Open a socket at FILE-NAME, and listen for connections there."
>   (with-fluids ((%default-port-encoding "UTF-8"))
>     (let ((sock    (socket PF_UNIX SOCK_STREAM 0))
>           (address (make-socket-address AF_UNIX file-name)))
>       (false-if-exception (delete-file file-name)) ;  <===== [dannym: WTF.  Would it be better to try to connect first?]
>       (bind sock address)
>       (listen sock 10)
>       sock)))
>
> Probably not good.

The ‘delete-file’ was to avoid EADDRINUSE but I wonder if it’s really a
good idea.

> Maybe better:
>
> (define (server-present? file-name)
>   "Open a socket at FILE-NAME, and connect to the server, if any.  Return #t if that worked."
>   (with-fluids ((%default-port-encoding "UTF-8"))
>     (let ((sock    (socket PF_UNIX SOCK_STREAM 0))
>           (address (make-socket-address AF_UNIX file-name)))
>       (false-if-exception (connect sock address))))) ; probably missing a "[catch] close".  How to do that best?

That’s a pretty good test.  Note that (gnu services herd) and (shepherd
comm) already provide an ‘open-connection’ procedure to do that.

From a shell script, you can also simply run “herd status root” and
check the exit status.

> I'm trying to find the right place to insert my "dbus-daemon" invocation for providing the user bus (note: not session bus)...

What’s the difference between a “user bus” and a “session bus”?  In
general, my understanding is that the user dbus-daemon (session bus?) is
started on demand.

> Should we make a system shepherd service that invokes the user shepherd service on behalf of users?  Would that be that safe?

The “guix user” approach that Mathieu shown is more flexible than having
to do something in the ‘operating-system’ declaration IMO.

> Or should we just expect the user to put a (shepherd with fix) invocation into their HOME startup scripts like .xinitrc ?  Note that if we did that there's some session-specific stuff in the session's environment that shepherd will inherit.  Probably not that bad if invoked early enough.

Currently that’s what I do: my ~/.xinitrc runs shepherd and that’s it.

HTH,
Ludo’.

Re: Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Danny Milosavljevic]

Hi Ludo,

On Tue, 13 Jun 2017 10:06:16 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> The ‘delete-file’ was to avoid EADDRINUSE but I wonder if it’s really a
> good idea.

Not a good idea.

We should just put the shepherd socket somewhere in /run/user/4711 where 4711 is the user id (the entire name is present in an environment variable called XDG_RUNTIME_DIR).  That's a new tmpfs created by the login process (elogind) on the first session of that user.  That way, this EADDRINUSE can never happen except when it should (see below).  Everything else is a weird workaround.

> What’s the difference between a “user bus” and a “session bus”?  In
> general, my understanding is that the user dbus-daemon (session bus?) is
> started on demand.

The user bus is started once per user.  The session bus is started once per session.

For example let's say you have a terminal server and log in as ludo on seat1 and also as ludo on seat2, then you have two session buses and one user bus. 

Filesystem-based services usually run per-user because almost no one creates extra directories for seats.  Many programmers don't even think about it and so their services are per-user whether they want to or not.

> > Or should we just expect the user to put a (shepherd with fix) invocation into their HOME startup scripts like .xinitrc ?  Note that if we did that there's some session-specific stuff in the session's environment that shepherd will inherit.  Probably not that bad if invoked early enough.  
> 
> Currently that’s what I do: my ~/.xinitrc runs shepherd and that’s it.

Yeah, me too.  That's how I found the bug... when I logged in to multiple seats suddenly I had a LOT of duplicate "user" services running :)

Re: Invoking user shepherd; Was: Re: Defining *user* services in Guix

[Ludovic Courtès]

Hello Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Tue, 13 Jun 2017 10:06:16 +0200
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> The ‘delete-file’ was to avoid EADDRINUSE but I wonder if it’s really a
>> good idea.
>
> Not a good idea.
>
> We should just put the shepherd socket somewhere in /run/user/4711 where 4711 is the user id (the entire name is present in an environment variable called XDG_RUNTIME_DIR).  That's a new tmpfs created by the login process (elogind) on the first session of that user.  That way, this EADDRINUSE can never happen except when it should (see below).  Everything else is a weird workaround.

Yes, that makes sense, we should do that.

>> What’s the difference between a “user bus” and a “session bus”?  In
>> general, my understanding is that the user dbus-daemon (session bus?) is
>> started on demand.
>
> The user bus is started once per user.  The session bus is started once per session.
>
> For example let's say you have a terminal server and log in as ludo on seat1 and also as ludo on seat2, then you have two session buses and one user bus. 
>
> Filesystem-based services usually run per-user because almost no one creates extra directories for seats.  Many programmers don't even think about it and so their services are per-user whether they want to or not.

I see.

Thanks for explaining, that makes a lot of sense to me!

Ludo’.

Re: Defining user services in Guix.

[Ludovic Courtès]

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> From caf7d9f5a7875663623b24662c1c23b99e60c5ad Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <m.othacehe@gmail.com>
> Date: Fri, 28 Apr 2017 17:09:53 +0200
> Subject: [PATCH] services: herd: Make %shepherd-socket-file a parameter and
>  export it.
>
> * gnu/services/herd.scm (%shepherd-socket-file): Make it an exported
>   parameter.
> (open-connection): Adapt.

LGTM, please push.   Thank you!

Ludo’.

Re: Defining user services in Guix.

[Carlo Zancanaro]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Sat, Apr 22 2017, Danny Milosavljevic wrote:
> Unfortunately, it still doesn't respawn redshift if it dies - it seems
> to require a pid file for that (#:start (make-forkexec-constructor ...
> #:pid-file "xxx.pid") and you have to specify #:respawn? #t. The
> invoked program then has to create "xxx.pid"). I think it would be a
> nice addition to have it monitor the process that
> make-forkexec-constructor invoked - without any pid file. Maybe it's
> even a bug...

I currently use shepherd to manage my local user daemons, and I have run
into this. There's an easy workaround, but there's definitely a bug
lurking here.

The problem is this:

> ;; Send shepherd into the background
> (action 'shepherd 'daemonize)

If you tell shepherd to turn itself into a daemon then it no longer
processes the SIGCHLD that it expects when the child dies.

I've also found that shepherd doesn't properly watch a process that uses
#:pid-file. I start my emacs with --daemon using shepherd and have it
create a pid file, but if emacs dies shepherd doesn't restart it for me.

Other than this, everything else about using shepherd to manage my user
services has worked great. I've been meaning to spend some time looking
into it, but I don't really know where to start in terms of debugging
and fixing this.

Carlo

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]