XWayland, /tmp/.X11-unix

XWayland, /tmp/.X11-unix

[Thorsten Wilms]

Hi!

If Weston is configured to support XWayland, weston-launch will fail with:
failed to bind to /tmp/.X11-unix/X0: No such file or directory

A simple `mkdir /tmp/.X11-unix` will fix that for the moment.

On #wayland, I have been told of that being handled by systemd tmpfiles 
configuration, on a per distribution basis. Apparently  /tmp/.X11-unix 
is an X-server thing, not specific to Weston.

Initially I thought creation of /tmp/.X11-unix should be tied to the 
xorg-server-xwayland package, but since it is more generic: which 
component should create that dir on Guix SD (based on what)?


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Ricardo Wurmus]

Thorsten Wilms <t_w_@freenet.de> writes:

> Initially I thought creation of /tmp/.X11-unix should be tied to the
> xorg-server-xwayland package, but since it is more generic: which
> component should create that dir on Guix SD (based on what)?

It is needed at run-time (because packages cannot create files outside
of their store prefix at build time), so it should be created by a
system service.  A service is not the same as a shepherd service; we
also have activation services that run once and only create a file or a
directory.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 22.03.2018 00:00, Ricardo Wurmus wrote:
> 
> Thorsten Wilms <t_w_@freenet.de> writes:
> 
>> Initially I thought creation of /tmp/.X11-unix should be tied to the
>> xorg-server-xwayland package, but since it is more generic: which
>> component should create that dir on Guix SD (based on what)?
> 
> It is needed at run-time (because packages cannot create files outside
> of their store prefix at build time), so it should be created by a
> system service.  A service is not the same as a shepherd service; we
> also have activation services that run once and only create a file or a
> directory.

After 2 to 3 hours of research, going through documentation and various 
.scm, I still don't even manage to write a service for my system 
configuration that just does a (mkdir-p "/tmp/.X11-uni"), so figuring 
out how to make that dependent on the actual need is way outside my 
capabilities.

Should I file a bug, and if so, against what?


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 22.03.2018 14:04, Thorsten Wilms wrote:
> On 22.03.2018 00:00, Ricardo Wurmus wrote:
>>
>> Thorsten Wilms <t_w_@freenet.de> writes:
>>
>>> Initially I thought creation of /tmp/.X11-unix should be tied to the
>>> xorg-server-xwayland package, but since it is more generic: which
>>> component should create that dir on Guix SD (based on what)?
>>
>> It is needed at run-time (because packages cannot create files outside
>> of their store prefix at build time), so it should be created by a
>> system service.  A service is not the same as a shepherd service; we
>> also have activation services that run once and only create a file or a
>> directory.

Revisiting, this wasn't too hard, actually:

Using (guix gexp) implied:

; Create /tmp/.X11-unix and make it writeable to, as required by 
weston-launch with XWayland enabled:
(define mkdir-x11-service
   (simple-service 'mkdir-x11
                   activation-service-type
                   #~(begin (let ((p "/tmp/.X11-unix"))
                               (mkdir-p p)
                               (chmod p #o777)))))

Or perhaps rather:

(define mkdir-x11-service
   (simple-service 'mkdir-x11
                   activation-service-type
                   #~(begin (use-modules (guix build utils))
                            (let ((p "/tmp/.X11-unix"))
                               (mkdir-p p)
                               (chmod p #o777)))))


I can't find anything that suggests a way to automatically add such a 
service to the operating system, if xorg-server-xwayland (or anything 
else that would use that dir) is installed.


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1893 bytes --]

Thorsten Wilms <t_w_@freenet.de> writes:

> On 22.03.2018 14:04, Thorsten Wilms wrote:
>> On 22.03.2018 00:00, Ricardo Wurmus wrote:
>>>
>>> Thorsten Wilms <t_w_@freenet.de> writes:
>>>
>>>> Initially I thought creation of /tmp/.X11-unix should be tied to the
>>>> xorg-server-xwayland package, but since it is more generic: which
>>>> component should create that dir on Guix SD (based on what)?
>>>
>>> It is needed at run-time (because packages cannot create files outside
>>> of their store prefix at build time), so it should be created by a
>>> system service.  A service is not the same as a shepherd service; we
>>> also have activation services that run once and only create a file or a
>>> directory.
>
> Revisiting, this wasn't too hard, actually:
>
> Using (guix gexp) implied:
>
> ; Create /tmp/.X11-unix and make it writeable to, as required by 
> weston-launch with XWayland enabled:
> (define mkdir-x11-service
>    (simple-service 'mkdir-x11
>                    activation-service-type
>                    #~(begin (let ((p "/tmp/.X11-unix"))
>                                (mkdir-p p)
>                                (chmod p #o777)))))
>
> Or perhaps rather:
>
> (define mkdir-x11-service
>    (simple-service 'mkdir-x11
>                    activation-service-type
>                    #~(begin (use-modules (guix build utils))
>                             (let ((p "/tmp/.X11-unix"))
>                                (mkdir-p p)
>                                (chmod p #o777)))))
>
>
> I can't find anything that suggests a way to automatically add such a 
> service to the operating system, if xorg-server-xwayland (or anything 
> else that would use that dir) is installed.

It could be done with a "profile hook" in (guix profiles).  Although for
the common case I suppose this will be done by a display manager?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 26.03.2018 11:33, Marius Bakke wrote:
> It could be done with a "profile hook" in (guix profiles).  Although for
> the common case I suppose this will be done by a display manager?

But then all X11-supporting display managers would have to care about it.

According to answers in
https://unix.stackexchange.com/questions/196677/what-is-tmp-x11-unix
/tmp/.X11-unix/ is the directory where any X11 server will create the 
unix domain socket X0.

One answer implies that is not necessarily the only way an X11 server 
may communicate with clients on the same machine.

`/gnu/store: grep -iRs X11-unix` only showed results caused by my own 
simple-service, matches in binary files and a comment in a 
slim-sigusr1.patch:
"The problem was that SLiM doesn't pay attention to SIGUSR1.  So in 
practice, if X starts slowly, then SLiM gets ECONNREFUSED a couple of 
time on /tmp/.X11-unix/X0, then goes on trying to connect to 
localhost:6000, where nobody answers; eventually, it times out and tries 
again on /tmp/.X11-unix/X0, and finally it shows up on the screen."

I would think that something must already take care of /tmp/.X11-unix/ 
for an operating-system configuration using plain X11?


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

Thorsten Wilms <t_w_@freenet.de> writes:

> On 26.03.2018 11:33, Marius Bakke wrote:
>> It could be done with a "profile hook" in (guix profiles).  Although for
>> the common case I suppose this will be done by a display manager?
>
> But then all X11-supporting display managers would have to care about it.
>
> According to answers in
> https://unix.stackexchange.com/questions/196677/what-is-tmp-x11-unix
> /tmp/.X11-unix/ is the directory where any X11 server will create the 
> unix domain socket X0.
>
> One answer implies that is not necessarily the only way an X11 server 
> may communicate with clients on the same machine.
>
> `/gnu/store: grep -iRs X11-unix` only showed results caused by my own 
> simple-service, matches in binary files and a comment in a 
> slim-sigusr1.patch:
> "The problem was that SLiM doesn't pay attention to SIGUSR1.  So in 
> practice, if X starts slowly, then SLiM gets ECONNREFUSED a couple of 
> time on /tmp/.X11-unix/X0, then goes on trying to connect to 
> localhost:6000, where nobody answers; eventually, it times out and tries 
> again on /tmp/.X11-unix/X0, and finally it shows up on the screen."
>
> I would think that something must already take care of /tmp/.X11-unix/ 
> for an operating-system configuration using plain X11?

Interesting.  I assumed SLiM created it, but could not find it with
'grep'.  Maybe libx11?

In any case it should be safe to add an activation script that creates
/tmp/.X11-unix on GuixSD.  I think it can be part of %desktop-services,
or maybe even %base-services.  Would you like to try it?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: XWayland, /tmp/.X11-unix

[Ludovic Courtès]

Hello!

Marius Bakke <mbakke@fastmail.com> skribis:

> Thorsten Wilms <t_w_@freenet.de> writes:
>
>> On 26.03.2018 11:33, Marius Bakke wrote:
>>> It could be done with a "profile hook" in (guix profiles).  Although for
>>> the common case I suppose this will be done by a display manager?
>>
>> But then all X11-supporting display managers would have to care about it.
>>
>> According to answers in
>> https://unix.stackexchange.com/questions/196677/what-is-tmp-x11-unix
>> /tmp/.X11-unix/ is the directory where any X11 server will create the 
>> unix domain socket X0.
>>
>> One answer implies that is not necessarily the only way an X11 server 
>> may communicate with clients on the same machine.
>>
>> `/gnu/store: grep -iRs X11-unix` only showed results caused by my own 
>> simple-service, matches in binary files and a comment in a 
>> slim-sigusr1.patch:
>> "The problem was that SLiM doesn't pay attention to SIGUSR1.  So in 
>> practice, if X starts slowly, then SLiM gets ECONNREFUSED a couple of 
>> time on /tmp/.X11-unix/X0, then goes on trying to connect to 
>> localhost:6000, where nobody answers; eventually, it times out and tries 
>> again on /tmp/.X11-unix/X0, and finally it shows up on the screen."
>>
>> I would think that something must already take care of /tmp/.X11-unix/ 
>> for an operating-system configuration using plain X11?
>
> Interesting.  I assumed SLiM created it, but could not find it with
> 'grep'.  Maybe libx11?

‘_xcb_open_unix’, called by ‘_xcb_open’ in libxcb, creates /tmp/.X11-unix.

Ideally Wayland’s client library would create it as well if it needs it?

> In any case it should be safe to add an activation script that creates
> /tmp/.X11-unix on GuixSD.  I think it can be part of %desktop-services,
> or maybe even %base-services.

Sounds like a good idea.

Ludo’.

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 26.03.2018 13:18, Marius Bakke wrote:
> In any case it should be safe to add an activation script that creates
> /tmp/.X11-unix on GuixSD.  I think it can be part of %desktop-services,
> or maybe even %base-services.  Would you like to try it?

I'm looking into it.


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 26.03.2018 13:18, Marius Bakke wrote:
> In any case it should be safe to add an activation script that creates
> /tmp/.X11-unix on GuixSD.  I think it can be part of %desktop-services,
> or maybe even %base-services.  Would you like to try it?

I would have like to send this (or similar) to the patch list:

---

 From 0af5028419db3adc8c7d8c4d51668c5077013ecc Mon Sep 17 00:00:00 2001
From: Thorsten Wilms <t_w_@freenet.de>
Date: Wed, 28 Mar 2018 19:59:11 +0200
Subject: [PATCH] gnu: Add x11-socket-dir-service

* gnu/services/desktop.scm (gnu): Export x11-socket-dir-service, define it,
   add it to %desktop-services list.
---
  gnu/services/desktop.scm | 19 +++++++++++++++++++
  1 file changed, 19 insertions(+)

diff --git a/gnu/services/desktop.scm b/gnu/services/desktop.scm
index 897252917..ef3a5457e 100644
--- a/gnu/services/desktop.scm
+++ b/gnu/services/desktop.scm
@@ -94,6 +94,8 @@
              xfce-desktop-service
              xfce-desktop-service-type

+            x11-socket-dir-service
+
              %desktop-services))

  ;;; Commentary:
@@ -882,6 +884,21 @@ with the administrator's password."

  \f
  ;;;
+;;; X11 socket directory service
+;;;
+
+(define x11-socket-dir-service
+  ;; Return a service that creates /tmp/.X11-unix. X11 servers, including
+  ;; XWayland, create their socket file there.
+  (simple-service 'x11-socket-dir
+                  activation-service-type
+                  #~(begin (use-modules (guix build utils))
+                           (let ((p "/tmp/.X11-unix"))
+                             (mkdir-p p)
+                             (chmod p #o776))))) ;; drwxrwxrwt
+
+\f
+;;;
  ;;; The default set of desktop services.
  ;;;

@@ -912,6 +929,8 @@ with the administrator's password."

           (ntp-service)

+         x11-socket-dir-service
+
           %base-services))

  ;;; desktop.scm ends here
-- 
2.11.0

---

I wanted to test it by using the changed guix and an operating-system 
that would include the x11-socket-dir-service. Things got messy, guix 
started to insist on building Python (and failing at it!). I cleaned up 
and started again:
$ git clone https://git.savannah.gnu.org/git/guix.git
$ cd guix
$ guix environment guix
$ ./bootstrap
$ ./configure --localstatedir=/var
$ make

# ./pre-inst-env guix-daemon --build-users-group=guixbuild
# ./pre-inst-env guix system reconfigure /etc/config.scm

=> since I had things to do away from the computer, I gave guix the 
chance to complete, but like 3 hours later it was still building and 
testing stuff, where prior to this mess, I would have expected none of 
that. Last package I recognised was Subversion.


Prior attempts included sessions like:

---

thorwil@charly ~/guix_env/guix$ sudo ./pre-inst-env guix-daemon 
--build-users-group=guixbuild
^Z
[1]+  Stopped                 sudo ./pre-inst-env guix-daemon 
--build-users-group=guixbuild
thorwil@charly ~/guix_env/guix$ bg
[1]+ sudo ./pre-inst-env guix-daemon --build-users-group=guixbuild &
thorwil@charly ~/guix_env/guix$ sudo ./pre-inst-env guix system 
reconfigure /etc/config.scm
guix: system: command not found

---

root@charly ~# cd /home/thorwil/guix_env/guix/
root@charly /home/thorwil/guix_env/guix# ./pre-inst-env guix system 
reconfigure /etc/config.scm
substitute: guix substitute: warning: ACL for archive imports seems to 
be uninitialized, substitutes may be unavailable
guix: offload: command not found
Try `guix --help' for more information.
guix system: error: build failed: unexpected EOF reading a line

---

root@charly /home/thorwil/guix_env/guix# guix environment guix
substitute: guix substitute: warning: ACL for archive imports seems to 
be uninitialized, substitutes may be unavailable
guix: offload: command not found
Try `guix --help' for more information.
guix environment: error: build failed: unexpected EOF reading a line


Reading that now, I guess "substitute: guix substitute: warning: ACL for 
archive imports seems to be uninitialized, substitutes may be 
unavailable" is the key?


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1616 bytes --]

Thorsten Wilms <t_w_@freenet.de> writes:

> I wanted to test it by using the changed guix and an operating-system 
> that would include the x11-socket-dir-service. Things got messy, guix 
> started to insist on building Python (and failing at it!). I cleaned up 
> and started again:
> $ git clone https://git.savannah.gnu.org/git/guix.git
> $ cd guix
> $ guix environment guix
> $ ./bootstrap
> $ ./configure --localstatedir=/var
> $ make
>
> # ./pre-inst-env guix-daemon --build-users-group=guixbuild
> # ./pre-inst-env guix system reconfigure /etc/config.scm
>
> => since I had things to do away from the computer, I gave guix the 
> chance to complete, but like 3 hours later it was still building and 
> testing stuff, where prior to this mess, I would have expected none of 
> that. Last package I recognised was Subversion.

[...]

> root@charly ~# cd /home/thorwil/guix_env/guix/
> root@charly /home/thorwil/guix_env/guix# ./pre-inst-env guix system 
> reconfigure /etc/config.scm
> substitute: guix substitute: warning: ACL for archive imports seems to 
> be uninitialized, substitutes may be unavailable
> guix: offload: command not found
> Try `guix --help' for more information.
> guix system: error: build failed: unexpected EOF reading a line

This could be because you are running the daemon from a git checkout.
I've never tried doing that, but you will probably have to configure
with "--sysconfdir=/etc" to pick up '/etc/guix/acl'.

Does it work if you use the "normal" daemon?  There's usually no need to
run the daemon from git unless you are hacking on it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 29.03.2018 19:02, Marius Bakke wrote:
> This could be because you are running the daemon from a git checkout.
> I've never tried doing that, but you will probably have to configure
> with "--sysconfdir=/etc" to pick up '/etc/guix/acl'.
> 
> Does it work if you use the "normal" daemon?  There's usually no need to
> run the daemon from git unless you are hacking on it.

Ah, I just went over
https://www.gnu.org/software/guix/manual/html_node/Running-Guix-Before-It-Is-Installed.html
without questioning the need or usefulness of running the daemon in this 
particular case. Still unfortunate that it defaults to building 
*everything*.

With only `./pre-inst-env guix system reconfigure /etc/config.scm` 
things work now.

Thanks!


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 9097 bytes --]

Thorsten Wilms <t_w_@freenet.de> writes:

> On 22.03.2018 00:00, Ricardo Wurmus wrote:
>>
>> Thorsten Wilms <t_w_@freenet.de> writes:
>>
>>> Initially I thought creation of /tmp/.X11-unix should be tied to the
>>> xorg-server-xwayland package, but since it is more generic: which
>>> component should create that dir on Guix SD (based on what)?
>>
>> It is needed at run-time (because packages cannot create files outside
>> of their store prefix at build time), so it should be created by a
>> system service.  A service is not the same as a shepherd service; we
>> also have activation services that run once and only create a file or a
>> directory.
>
> After 2 to 3 hours of research, going through documentation and
> various .scm, I still don't even manage to write a service for my
> system configuration that just does a (mkdir-p "/tmp/.X11-uni"), so
> figuring out how to make that dependent on the actual need is way
> outside my capabilities.

Take heart: it is not easy for me, either.  :-)

> Should I file a bug, and if so, against what?

If this is a problem that is preventing you from building something new,
then discussing it here is probably the right approach.  If it's
affecting an existing package or service in Guix, then you might want to
create a bug for it by emailing bug-guix@gnu.org.

By the way, to create a "service" that does what you need, you might
find it useful to look at an existing example service.  For example, the
procedure nscd-activation in gnu/services/base.scm does something
similar.  So does the procedure exim-activation in
gnu/services/mail.scm.

The manual does explain what is going on (see: (guix) Service
Reference), but I have read it multiple times and examined the source
multiple times, and I still have trouble remembering exactly how it all
fits together.

I'll try to explain how I understand it.  In Guix, a service is a
<service> object; sometimes this is also referred to as a service
"instance".  A <service> object contains two other objects: a "type" and
a "value".  The "type" is a <service-type> object and is sometimes
referred to as the service "kind" (because it represents what kind of
service it is).  The "value" is whatever that particular service type
requires, and sometimes it is also referred to as a service's
"parameter", "parameters", or "parameter value".  Usually the value is a
configuration object of some kind.  For example, the Guix service
expects its value to be a <guix-configuration> object.

By the way, these objects with angle brackets, like <service>, are
defined using the define-record-type* syntax (which itself is defined in
guix/records.scm).  They are similar to normal Scheme records (see:
(guile) Record Overview), if you're familiar with those.

Guix has an "activation service".  Its purpose, as described in the
manual (see: (guix) Service Types and Services), is to run a code
snippet "at activation time - e.g., when the service is booted".  The
basic contract for extending the "activation service" is that if you
define an extension of the activation service which specifies a
"compute" procedure foo, where foo is a procedure that returns a
G-Expression that takes some action (e.g., create the "/tmp/.X11-uni"
directory), then that action will be taken at activation time.  The
"foo" procedure will receive a single argument as input: the argument
will be the value of the service (your service!) that is extending the
activation service.

Confused?  I know I was when I first looked into this.  But stay with me
- together we can make it through!  The mechanism that implements the
extension logic can be found in the fold-services procedure in
gnu/services.scm.  Specifically, when a service like the Guix service
extends an extensible service like the activation service, we have the
following situation:

* A guix-service-type is defined (in gnu/services/base.scm).  It
  contains a <service-extension> for which the "target" is the
  activation-service-type and the "compute" is the guix-activation
  procedure.

* We add an instance of the Guix service to the %base-services list by
  calling the guix-service procedure.  This procedure creates an
  instance of the service and uses %default-guix-configuration as its
  value (which is a <guix-configuration> object).

* The activation-service-type is defined (in gnu/services.scm).  Its
  "compose" is the append procedure (see: (guile) Append/Reverse), and
  its "extend" is the second-argument procedure.

* We add an instance of the activation service to the essential-services
  list.  The activation service's initial value is #t.

When Guix builds all the system services, it will do the following:

* Guix will find a service of the activation-service-type.  This is the
  activation service.

* Guix will find a service of the guix-service-type.  This is the Guix
  service.

* Guix will recognize that the Guix service extends the activation
  service.

* Guix will pass the Guix service's value to the guix-service-type's
  "compute" procedure defined earlier.  This means that Guix will call
  the guix-activation procedure with a single argument: the
  %default-guix-configuration.  The guix-activation procedure returns an
  appropriate G-Expression that takes steps to activate Guix according
  to the provided configuration.

* If there are any other services that extend the
  activation-service-type, Guix will call their "compute" procedures in
  a similar fashion.

* Guix will collect all of the objects returned by the various "compute"
  procedures in a list.  The G-Expression returned by guix-activation
  will be one element of this list.  Guix will pass this list to the
  activation-service-type's "compose" procedure.  This means that it
  will call append with a single argument: the list.  This will actually
  return the same list, since appending a single list returns the same
  list.  (As explained elsewhere on the guix-devel email list, this
  should really be the "identity" procedure, not the "append" procedure,
  since it more accurately reflects the intent [1].)

* Guix will call the activation-service-type's "extend" procedure with
  two arguments: the first argument is the value of the activation
  service found earlier, and the second argument is the result of the
  "compose" procedure from the previous step.  This means that Guix will
  call the second-argument procedure with the value #t as its first
  argument, and the list of G-Expressions from above as its second
  argument.  Because the second-argument procedure simply returns the
  second argument, the value #t will be ignored, and the result will be
  the list of G-Expressions.

* Guix replaces the activation service with a new service instance that
  has the same type but a new value.  The new value is set to the result
  of calling the "extend" procedure.  This means that the activation
  service will be replaced with a new activation service that uses the
  list of G-Expressions from the previous step as its value.

Guix repeats this process recursively for all services defined in your
<operating-system> declaration.  In this way, even though the high-level
contract for how services extend one another is dictated by Guix, the
interpretation of the objects and procedures involved is largely
determined by the service that is being extended.  For example, Guix
promises the activation service that it will collect objects from
services that extend the activation service and then call the activation
service's "compose" and "extend" procedures as described above to create
a new, modified activation service.  However, the structure of those
objects and the behavior of those procedures are largely determined by
the activation-service-type itself.  A different type of service may
very well define a different contract.

To re-iterate what I wrote earlier, the activation-service-type's
contract is as follows: when you define a service that extends it (i.e.,
when you define a <service-type> that extends the
activation-service-type), your service will define a "compute" procedure
that returns a G-Expression that performs some action (e.g., create a
directory).  The activation service will then arrange to run that action
at activation time.  This means that if your service needs a
"/tmp/.X11-uni" directory to exist before it runs, you need to define a
"compute" procedure that accepts a single argument (which will be your
service's value) and returns a G-Expression that creates the directory.
Different extensible services have different expectations, so you would
need to implement different "compute" procedures to extend them.
Thankfully, even if an extensible service is not clearly documented, you
can usually tell what its contract is by looking at some services that
extend it already.

I hope that helps!

Footnotes: 
[1]  https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00307.html

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

Thank you, Chris, for taking the time to go into such detail! However, 
as mentioned later in the thread, I arrived at a solution days ago. It 
just took finding a close match among existing services and recalling 
some basics about Scheme evaluation :)

Your write-up did improve my idea of what is going on, though I guess I 
may have to reread it later on, to get more out of it.


On 29.03.2018 08:18, Chris Marusich wrote:
 > Thorsten Wilms <t_w_@freenet.de> writes:
 >
 >> On 22.03.2018 00:00, Ricardo Wurmus wrote:
 >>>
 >>> Thorsten Wilms <t_w_@freenet.de> writes:
 >>>
 >>>> Initially I thought creation of /tmp/.X11-unix should be tied to the
 >>>> xorg-server-xwayland package, but since it is more generic: which
 >>>> component should create that dir on Guix SD (based on what)?
 >>>
 >>> It is needed at run-time (because packages cannot create files outside
 >>> of their store prefix at build time), so it should be created by a
 >>> system service.  A service is not the same as a shepherd service; we
 >>> also have activation services that run once and only create a file or a
 >>> directory.
 >>
 >> After 2 to 3 hours of research, going through documentation and
 >> various .scm, I still don't even manage to write a service for my
 >> system configuration that just does a (mkdir-p "/tmp/.X11-uni"), so
 >> figuring out how to make that dependent on the actual need is way
 >> outside my capabilities.
 >
 > Take heart: it is not easy for me, either.  :-)
 >
 >> Should I file a bug, and if so, against what?
 >
 > If this is a problem that is preventing you from building something new,
 > then discussing it here is probably the right approach.  If it's
 > affecting an existing package or service in Guix, then you might want to
 > create a bug for it by emailing bug-guix@gnu.org.
 >
 > By the way, to create a "service" that does what you need, you might
 > find it useful to look at an existing example service.  For example, the
 > procedure nscd-activation in gnu/services/base.scm does something
 > similar.  So does the procedure exim-activation in
 > gnu/services/mail.scm.
 >
 > The manual does explain what is going on (see: (guix) Service
 > Reference), but I have read it multiple times and examined the source
 > multiple times, and I still have trouble remembering exactly how it all
 > fits together.
 >
 > I'll try to explain how I understand it.  In Guix, a service is a
 > <service> object; sometimes this is also referred to as a service
 > "instance".  A <service> object contains two other objects: a "type" and
 > a "value".  The "type" is a <service-type> object and is sometimes
 > referred to as the service "kind" (because it represents what kind of
 > service it is).  The "value" is whatever that particular service type
 > requires, and sometimes it is also referred to as a service's
 > "parameter", "parameters", or "parameter value".  Usually the value is a
 > configuration object of some kind.  For example, the Guix service
 > expects its value to be a <guix-configuration> object.
 >
 > By the way, these objects with angle brackets, like <service>, are
 > defined using the define-record-type* syntax (which itself is defined in
 > guix/records.scm).  They are similar to normal Scheme records (see:
 > (guile) Record Overview), if you're familiar with those.
 >
 > Guix has an "activation service".  Its purpose, as described in the
 > manual (see: (guix) Service Types and Services), is to run a code
 > snippet "at activation time - e.g., when the service is booted".  The
 > basic contract for extending the "activation service" is that if you
 > define an extension of the activation service which specifies a
 > "compute" procedure foo, where foo is a procedure that returns a
 > G-Expression that takes some action (e.g., create the "/tmp/.X11-uni"
 > directory), then that action will be taken at activation time.  The
 > "foo" procedure will receive a single argument as input: the argument
 > will be the value of the service (your service!) that is extending the
 > activation service.
 >
 > Confused?  I know I was when I first looked into this.  But stay with me
 > - together we can make it through!  The mechanism that implements the
 > extension logic can be found in the fold-services procedure in
 > gnu/services.scm.  Specifically, when a service like the Guix service
 > extends an extensible service like the activation service, we have the
 > following situation:
 >
 > * A guix-service-type is defined (in gnu/services/base.scm).  It
 >    contains a <service-extension> for which the "target" is the
 >    activation-service-type and the "compute" is the guix-activation
 >    procedure.
 >
 > * We add an instance of the Guix service to the %base-services list by
 >    calling the guix-service procedure.  This procedure creates an
 >    instance of the service and uses %default-guix-configuration as its
 >    value (which is a <guix-configuration> object).
 >
 > * The activation-service-type is defined (in gnu/services.scm).  Its
 >    "compose" is the append procedure (see: (guile) Append/Reverse), and
 >    its "extend" is the second-argument procedure.
 >
 > * We add an instance of the activation service to the essential-services
 >    list.  The activation service's initial value is #t.
 >
 > When Guix builds all the system services, it will do the following:
 >
 > * Guix will find a service of the activation-service-type.  This is the
 >    activation service.
 >
 > * Guix will find a service of the guix-service-type.  This is the Guix
 >    service.
 >
 > * Guix will recognize that the Guix service extends the activation
 >    service.
 >
 > * Guix will pass the Guix service's value to the guix-service-type's
 >    "compute" procedure defined earlier.  This means that Guix will call
 >    the guix-activation procedure with a single argument: the
 >    %default-guix-configuration.  The guix-activation procedure returns an
 >    appropriate G-Expression that takes steps to activate Guix according
 >    to the provided configuration.
 >
 > * If there are any other services that extend the
 >    activation-service-type, Guix will call their "compute" procedures in
 >    a similar fashion.
 >
 > * Guix will collect all of the objects returned by the various "compute"
 >    procedures in a list.  The G-Expression returned by guix-activation
 >    will be one element of this list.  Guix will pass this list to the
 >    activation-service-type's "compose" procedure.  This means that it
 >    will call append with a single argument: the list.  This will actually
 >    return the same list, since appending a single list returns the same
 >    list.  (As explained elsewhere on the guix-devel email list, this
 >    should really be the "identity" procedure, not the "append" procedure,
 >    since it more accurately reflects the intent [1].)
 >
 > * Guix will call the activation-service-type's "extend" procedure with
 >    two arguments: the first argument is the value of the activation
 >    service found earlier, and the second argument is the result of the
 >    "compose" procedure from the previous step.  This means that Guix will
 >    call the second-argument procedure with the value #t as its first
 >    argument, and the list of G-Expressions from above as its second
 >    argument.  Because the second-argument procedure simply returns the
 >    second argument, the value #t will be ignored, and the result will be
 >    the list of G-Expressions.
 >
 > * Guix replaces the activation service with a new service instance that
 >    has the same type but a new value.  The new value is set to the result
 >    of calling the "extend" procedure.  This means that the activation
 >    service will be replaced with a new activation service that uses the
 >    list of G-Expressions from the previous step as its value.
 >
 > Guix repeats this process recursively for all services defined in your
 > <operating-system> declaration.  In this way, even though the high-level
 > contract for how services extend one another is dictated by Guix, the
 > interpretation of the objects and procedures involved is largely
 > determined by the service that is being extended.  For example, Guix
 > promises the activation service that it will collect objects from
 > services that extend the activation service and then call the activation
 > service's "compose" and "extend" procedures as described above to create
 > a new, modified activation service.  However, the structure of those
 > objects and the behavior of those procedures are largely determined by
 > the activation-service-type itself.  A different type of service may
 > very well define a different contract.
 >
 > To re-iterate what I wrote earlier, the activation-service-type's
 > contract is as follows: when you define a service that extends it (i.e.,
 > when you define a <service-type> that extends the
 > activation-service-type), your service will define a "compute" procedure
 > that returns a G-Expression that performs some action (e.g., create a
 > directory).  The activation service will then arrange to run that action
 > at activation time.  This means that if your service needs a
 > "/tmp/.X11-uni" directory to exist before it runs, you need to define a
 > "compute" procedure that accepts a single argument (which will be your
 > service's value) and returns a G-Expression that creates the directory.
 > Different extensible services have different expectations, so you would
 > need to implement different "compute" procedures to extend them.
 > Thankfully, even if an extensible service is not clearly documented, you
 > can usually tell what its contract is by looking at some services that
 > extend it already.
 >
 > I hope that helps!
 >
 > Footnotes:
 > [1]  https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00307.html
 >

-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: XWayland, /tmp/.X11-unix

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1265 bytes --]

Thorsten Wilms <t_w_@freenet.de> writes:

> Thank you, Chris, for taking the time to go into such detail!

I found the service documentation to be a little confusing when I first
read it.  The slight incorrect statements (which I have now corrected
after confirming the intended behavior with Ludo) did not help clarify
matters.  My explanation was as much an attempt to help you as it was to
remind myself (yet again) how all this stuff works!

If anything is still unclear, please let me know and I will try to
explain more.  I hope to eventually go back and improve the services
documentation further in the manual, once I feel I have a strong enough
grasp of how it all works.

> However, as mentioned later in the thread, I arrived at a solution
> days ago. It just took finding a close match among existing services
> and recalling some basics about Scheme evaluation :)

I'm glad you were able to get it working!  Hopefully my explanation,
along with the manual, will help clarify any doubts you had about why it
works.

> Your write-up did improve my idea of what is going on, though I guess
> I may have to reread it later on, to get more out of it.

Excellent!  It's good to know I was able to help even a little.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Services, was: Re: XWayland, /tmp/.X11-unix

[Thorsten Wilms]

On 30.03.2018 06:34, Chris Marusich wrote:
> I hope to eventually go back and improve the services
> documentation further in the manual, once I feel I have a strong enough
> grasp of how it all works.

A few thoughts then, that I hope will be of use:

Initially, I had a hard time understanding the talk about "extending", 
as the first snippets I looked at seemed rather like adding items to 
todo-lists. The 2nd paragraph on 
https://www.gnu.org/software/guix/manual/html_node/Service-Composition.html 
makes it clear.

So it appears to me that the core of this is that there are daemons and 
procedures that attain and control capabilities. Since there may be any 
number of things-to-do dependent on operating-system configuration, 
requiring access to those capabilities, a DAG is constructed and 
flattened to arrive at daemon configuration and executable procedures.

One aspect that makes reading the docs hard are the multiple meanings of 
"service" and the rather unspecific "service type", I think. Makes one 
think of Smurfs. In fairness: it seems hard to avoid ... maybe:
Could we try to reserve a naked "service" to refer to daemons and 
procedures with effects on the operating system? The things that go into 
operating-system/services are ... service-building-blocks, latent 
services, service-parts, proto-services ... OK; not terribly satisfying.

It is mentioned that service types are the nodes in the services DAG. 
Thus they could have been called service-nodes, which would have avoided 
confusion with those other *types* of services: ... printing s., desktop 
s., database s. ... .

Here I notice that I don't quite get how service-types wrapped in 
services in a list of services map to a DAG.


I see 2 main uses cases regarding the documentation:
A. Trying to understand the whole system.
B. Trying to accomplish a specific task and looking for just the 
required information.

The given descriptive nature of the documentation so far is alright for 
A. To serve B, you'd have to work along questions like:
- Is what I'm trying to accomplish within the scope of services? What 
are their capabilities and limitations? (A case like adding files to 
/etc might not come to everyone's mind.)
- What is the nature of what I'm tying to do, regarding time of 
execution, capabilities, dependencies and permissions? Here an overview 
of what service-type allows what?, when?, would shine.


-- 
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/