[bug#30657] Add support for file-like objects to the Prosody configuration

[bug#30657] Add support for file-like objects to the Prosody configuration

[Clément Lassieur]

Hello,

These patches add support for file-like objects to the Prosody service
configuration.  The idea is to replace this mechanism (which is used
very often):

    (plain-file
      (with-output-to-string
         ...
         (format #t ...)
         ...))

with this one:

    (mixed-text-file
      (flatten
        (with-tokens-to-list
           ...
           (push-tokens ...)
           ...))

The point is that tokens don't have to be strings.

Comments are welcome!
Clément

[bug#30657] [PATCH 1/4] services: utils: Add 'push-tokens' and 'with-tokens-to-list'.

[Clément Lassieur]

* gnu/services/utils.scm: New file.
(push-tokens, with-tokens-to-list): New exported procedures.
(%tokens-stack): New parameter.
* gnu/local.mk: Add new file.
---
 gnu/local.mk           |  1 +
 gnu/services/utils.scm | 41 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+)
 create mode 100644 gnu/services/utils.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 68f3a20f6..07cc2229a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/ssh.scm				\
   %D%/services/sysctl.scm			\
   %D%/services/telephony.scm			\
+  %D%/services/utils.scm			\
   %D%/services/version-control.scm              \
   %D%/services/vpn.scm				\
   %D%/services/web.scm				\
diff --git a/gnu/services/utils.scm b/gnu/services/utils.scm
new file mode 100644
index 000000000..a8d7981b2
--- /dev/null
+++ b/gnu/services/utils.scm
@@ -0,0 +1,41 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services utils)
+  #:export (push-tokens
+            with-tokens-to-list))
+
+(define %tokens-stack (make-parameter #f))
+
+(define (push-tokens . tokens)
+  "Push TOKENS to %TOKENS-STACK so that they will be returned by
+WITH-TOKENS-TO-LIST."
+  (unless (%tokens-stack)
+    (error "not called within the dynamic extent of 'with-tokens-to-list'"))
+  (%tokens-stack (append (%tokens-stack) tokens)))
+
+(define (with-tokens-to-list thunk)
+  "Call THUNK and return the tokens that were pushed to %TOKENS-STACK as a
+list."
+  (parameterize ((%tokens-stack '()))
+    (thunk)
+    (%tokens-stack)))
+
+;;; Local Variables:
+;;; eval: (put 'with-tokens-to-list 'scheme-indent-function 0)
+;;; End:
-- 
2.16.2

[bug#30657] [PATCH 2/4] gexp: Add 'file-like?'.

[Clément Lassieur]

* guix/gexp.scm (file-like?): New exported procedure.
---
 guix/gexp.scm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/guix/gexp.scm b/guix/gexp.scm
index f005c4d29..9a30579a6 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -70,6 +71,8 @@
             file-append-base
             file-append-suffix
 
+            file-like?
+
             load-path-expression
             gexp-modules
 
@@ -437,6 +440,14 @@ SUFFIX."
                           (base   (expand base lowered output)))
                      (string-append base (string-concatenate suffix)))))))
 
+(define (file-like? object)
+  (or (local-file? object)
+      (plain-file? object)
+      (computed-file? object)
+      (program-file? object)
+      (scheme-file? object)
+      (file-append? object)))
+
 \f
 ;;;
 ;;; Inputs & outputs.
-- 
2.16.2

[bug#30657] [PATCH 3/4] services: utils: move 'flatten' from (gnu services web).

[Clément Lassieur]

* gnu/services/web.scm (flatten): Move it...
* gnu/services/utils.scm (flatten): ...to here, and export it.
---
 gnu/services/utils.scm | 12 +++++++++++-
 gnu/services/web.scm   |  9 +--------
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/gnu/services/utils.scm b/gnu/services/utils.scm
index a8d7981b2..4b7c9c219 100644
--- a/gnu/services/utils.scm
+++ b/gnu/services/utils.scm
@@ -17,8 +17,10 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu services utils)
+  #:use-module (srfi srfi-1)
   #:export (push-tokens
-            with-tokens-to-list))
+            with-tokens-to-list
+            flatten))
 
 (define %tokens-stack (make-parameter #f))
 
@@ -36,6 +38,14 @@ list."
     (thunk)
     (%tokens-stack)))
 
+(define (flatten . lst)
+  "Return a list that recursively concatenates all sub-lists of LST."
+  (define (flatten1 head out)
+    (if (list? head)
+        (fold-right flatten1 out head)
+        (cons head out)))
+  (fold-right flatten1 '() lst))
+
 ;;; Local Variables:
 ;;; eval: (put 'with-tokens-to-list 'scheme-indent-function 0)
 ;;; End:
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index beda481b0..38af66367 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -29,6 +29,7 @@
   #:use-module (gnu packages admin)
   #:use-module (gnu packages web)
   #:use-module (gnu packages php)
+  #:use-module (gnu services utils)
   #:use-module (guix records)
   #:use-module (guix gexp)
   #:use-module ((guix utils) #:select (version-major))
@@ -499,14 +500,6 @@ of index files."
         (nginx-upstream-configuration-servers upstream))
    "    }\n"))
 
-(define (flatten . lst)
-  "Return a list that recursively concatenates all sub-lists of LST."
-  (define (flatten1 head out)
-    (if (list? head)
-        (fold-right flatten1 out head)
-        (cons head out)))
-  (fold-right flatten1 '() lst))
-
 (define (default-nginx-config config)
   (match-record config
                 <nginx-configuration>
-- 
2.16.2

[bug#30657] [PATCH 4/4] services: messaging: Prosody config supports file-like objects.

[Clément Lassieur]

* doc/guix.texi (Messaging Services): Update accordingly.
* gnu/services/messaging.scm (serialize-field, enclose-quotes,
serialize-raw-content, serialize-ssl-configuration,
serialize-virtualhost-configuration, serialize-int-component-configuration,
serialize-ext-component-configuration, prosody-activation): Replace 'format'
with 'push-token' and 'with-output-to-string' with 'with-tokens-to-list'.
(prosody-activation): Replace 'plain-file' with 'mixed-text-file' and flatten
the tokens.
(serialize-non-negative-integer, serialize-non-negative-integer-list): Convert
numbers to strings.
(file-object?, serialize-file-object, file-object-list?,
serialize-file-object-list): New procedures.
(ssl-configuration)[capath, cafile], (prosody-configuration)[plugin-paths,
groups-file]: Replace 'file-name' with 'file-object'.
---
 doc/guix.texi              | 13 +++++++----
 gnu/services/messaging.scm | 58 ++++++++++++++++++++++++++++------------------
 2 files changed, 45 insertions(+), 26 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 24db16761..3bb544e62 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14210,6 +14210,9 @@ There is also a way to specify the configuration as a string, if you
 have an old @code{prosody.cfg.lua} file that you want to port over from
 some other system; see the end for more details.
 
+The @code{file-object} type designates either a file-like object
+(@pxref{G-Expressions, file-like objects}) or a file name.
+
 @c The following documentation was initially generated by
 @c (generate-documentation) in (gnu services messaging).  Manually maintained
 @c documentation is better, so we shouldn't hesitate to edit below as
@@ -14230,7 +14233,7 @@ Location of the Prosody data storage directory.  See
 Defaults to @samp{"/var/lib/prosody"}.
 @end deftypevr
 
-@deftypevr {@code{prosody-configuration} parameter} file-name-list plugin-paths
+@deftypevr {@code{prosody-configuration} parameter} file-object-list plugin-paths
 Additional plugin directories.  They are searched in all the specified
 paths in order.  See @url{https://prosody.im/doc/plugins_directory}.
 Defaults to @samp{()}.
@@ -14271,7 +14274,7 @@ should you want to disable them then add them to this list.
 Defaults to @samp{()}.
 @end deftypevr
 
-@deftypevr {@code{prosody-configuration} parameter} file-name groups-file
+@deftypevr {@code{prosody-configuration} parameter} file-object groups-file
 Path to a text file where the shared groups are defined.  If this path is
 empty then @samp{mod_groups} does nothing.  See
 @url{https://prosody.im/doc/modules/mod_groups}.
@@ -14304,13 +14307,13 @@ Path to your private key file.
 Path to your certificate file.
 @end deftypevr
 
-@deftypevr {@code{ssl-configuration} parameter} file-name capath
+@deftypevr {@code{ssl-configuration} parameter} file-object capath
 Path to directory containing root certificates that you wish Prosody to
 trust when verifying the certificates of remote servers.
 Defaults to @samp{"/etc/ssl/certs"}.
 @end deftypevr
 
-@deftypevr {@code{ssl-configuration} parameter} maybe-file-name cafile
+@deftypevr {@code{ssl-configuration} parameter} maybe-file-object cafile
 Path to a file containing root certificates that you wish Prosody to trust.
 Similar to @code{capath} but with all certificates concatenated together.
 @end deftypevr
@@ -14570,6 +14573,8 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@c end of Prosody auto-generated documentation
+
 @subsubheading BitlBee Service
 
 @cindex IRC (Internet Relay Chat)
diff --git a/gnu/services/messaging.scm b/gnu/services/messaging.scm
index 427e2121f..316eebecb 100644
--- a/gnu/services/messaging.scm
+++ b/gnu/services/messaging.scm
@@ -24,6 +24,7 @@
   #:use-module (gnu services)
   #:use-module (gnu services shepherd)
   #:use-module (gnu services configuration)
+  #:use-module (gnu services utils)
   #:use-module (gnu system shadow)
   #:use-module (guix gexp)
   #:use-module (guix modules)
@@ -115,16 +116,16 @@
                  "_")))
 
 (define (serialize-field field-name val)
-  (format #t "~a = ~a;\n" (uglify-field-name field-name) val))
+  (push-tokens (format #f "~a = " (uglify-field-name field-name)) val ";\n"))
 (define (serialize-field-list field-name val)
   (serialize-field field-name
-                   (with-output-to-string
+                   (with-tokens-to-list
                      (lambda ()
-                       (format #t "{\n")
+                       (push-tokens "{\n")
                        (for-each (lambda (x)
-                                   (format #t "~a;\n" x))
+                                   (push-tokens x ";\n"))
                                  val)
-                       (format #t "}")))))
+                       (push-tokens "}")))))
 
 (define (serialize-boolean field-name val)
   (serialize-field field-name (if val "true" "false")))
@@ -140,17 +141,17 @@
 (define (non-negative-integer? val)
   (and (exact-integer? val) (not (negative? val))))
 (define (serialize-non-negative-integer field-name val)
-  (serialize-field field-name val))
+  (serialize-field field-name (number->string val)))
 (define-maybe non-negative-integer)
 
 (define (non-negative-integer-list? val)
   (and (list? val) (and-map non-negative-integer? val)))
 (define (serialize-non-negative-integer-list field-name val)
-  (serialize-field-list field-name val))
+  (serialize-field-list field-name (map number->string val)))
 (define-maybe non-negative-integer-list)
 
 (define (enclose-quotes s)
-  (format #f "\"~a\"" s))
+  (list "\"" s "\""))
 (define (serialize-string field-name val)
   (serialize-field field-name (enclose-quotes val)))
 (define-maybe string)
@@ -183,10 +184,22 @@
   (serialize-string-list field-name val))
 (define-maybe file-name)
 
+(define (file-object? val)
+  (or (file-like? val) (file-name? val)))
+(define (serialize-file-object field-name val)
+  (serialize-string field-name val))
+(define-maybe file-object)
+
+(define (file-object-list? val)
+  (and (list? val) (and-map file-object? val)))
+(define (serialize-file-object-list field-name val)
+  (serialize-string-list field-name val))
+(define-maybe file-object)
+
 (define (raw-content? val)
   (not (eq? val 'disabled)))
 (define (serialize-raw-content field-name val)
-  (format #t "~a" val))
+  (push-tokens val))
 (define-maybe raw-content)
 
 (define-configuration mod-muc-configuration
@@ -224,12 +237,12 @@ just joined the room."))
    "Path to your certificate file.")
 
   (capath
-   (file-name "/etc/ssl/certs")
+   (file-object "/etc/ssl/certs")
    "Path to directory containing root certificates that you wish Prosody to
 trust when verifying the certificates of remote servers.")
 
   (cafile
-   (maybe-file-name 'disabled)
+   (maybe-file-object 'disabled)
    "Path to a file containing root certificates that you wish Prosody to trust.
 Similar to @code{capath} but with all certificates concatenated together.")
 
@@ -273,9 +286,9 @@ can create such a file with:
    (maybe-string 'disabled)
    "Password for encrypted private keys."))
 (define (serialize-ssl-configuration field-name val)
-  (format #t "ssl = {\n")
+  (push-tokens "ssl = {\n")
   (serialize-configuration val ssl-configuration-fields)
-  (format #t "};\n"))
+  (push-tokens "};\n"))
 (define-maybe ssl-configuration)
 
 (define %default-modules-enabled
@@ -331,7 +344,7 @@ can create such a file with:
      global)
 
     (plugin-paths
-     (file-name-list '())
+     (file-object-list '())
      "Additional plugin directories.  They are searched in all the specified
 paths in order.  See @url{https://prosody.im/doc/plugins_directory}."
      global)
@@ -372,7 +385,7 @@ should you want to disable them then add them to this list."
      common)
 
     (groups-file
-     (file-name "/var/lib/prosody/sharedgroups.txt")
+     (file-object "/var/lib/prosody/sharedgroups.txt")
      "Path to a text file where the shared groups are defined.  If this path is
 empty then @samp{mod_groups} does nothing.  See
 @url{https://prosody.im/doc/modules/mod_groups}."
@@ -566,7 +579,7 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
                '(domain))))
   (let ((domain (virtualhost-configuration-domain config))
         (rest (filter rest? virtualhost-configuration-fields)))
-    (format #t "VirtualHost \"~a\"\n" domain)
+    (push-tokens (format #f "VirtualHost \"~a\"\n" domain))
     (serialize-configuration config rest)))
 
 ;; Serialize Component line first.
@@ -577,7 +590,7 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
   (let ((hostname (int-component-configuration-hostname config))
         (plugin (int-component-configuration-plugin config))
         (rest (filter rest? int-component-configuration-fields)))
-    (format #t "Component \"~a\" \"~a\"\n" hostname plugin)
+    (push-tokens (format #f "Component \"~a\" \"~a\"\n" hostname plugin))
     (serialize-configuration config rest)))
 
 ;; Serialize Component line first.
@@ -587,7 +600,7 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
                '(hostname))))
   (let ((hostname (ext-component-configuration-hostname config))
         (rest (filter rest? ext-component-configuration-fields)))
-    (format #t "Component \"~a\"\n" hostname)
+    (push-tokens (format #f "Component \"~a\"\n" hostname))
     (serialize-configuration config rest)))
 
 ;; Serialize virtualhosts and components last.
@@ -646,13 +659,14 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
          (default-certs-dir "/etc/prosody/certs")
          (data-path (prosody-configuration-data-path config))
          (pidfile-dir (dirname (prosody-configuration-pidfile config)))
-         (config-str
+         (config-tokens
           (if (opaque-prosody-configuration? config)
-              (opaque-prosody-configuration-prosody.cfg.lua config)
-              (with-output-to-string
+              (list (opaque-prosody-configuration-prosody.cfg.lua config))
+              (with-tokens-to-list
                 (lambda ()
                   (serialize-prosody-configuration config)))))
-         (config-file (plain-file "prosody.cfg.lua" config-str)))
+         (config-file
+          (apply mixed-text-file "prosody.cfg.lua" (flatten config-tokens))))
     #~(begin
         (use-modules (guix build utils))
         (define %user (getpw "prosody"))
-- 
2.16.2

[bug#30657] [PATCH 2/4] gexp: Add 'file-like?'.

[Ludovic Courtès]

Hello!

Clément Lassieur <clement@lassieur.org> skribis:

> * guix/gexp.scm (file-like?): New exported procedure.
> ---
>  guix/gexp.scm | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/guix/gexp.scm b/guix/gexp.scm
> index f005c4d29..9a30579a6 100644
> --- a/guix/gexp.scm
> +++ b/guix/gexp.scm
> @@ -1,5 +1,6 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright © 2014, 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -70,6 +71,8 @@
>              file-append-base
>              file-append-suffix
>  
> +            file-like?
> +
>              load-path-expression
>              gexp-modules
>  
> @@ -437,6 +440,14 @@ SUFFIX."
>                            (base   (expand base lowered output)))
>                       (string-append base (string-concatenate suffix)))))))
>  
> +(define (file-like? object)
> +  (or (local-file? object)
> +      (plain-file? object)
> +      (computed-file? object)
> +      (program-file? object)
> +      (scheme-file? object)
> +      (file-append? object)))

This procedure would miss new types added with ‘define-gexp-compiler’.

In fact I think you can simply write:

  (define file-like? (@@ (guix gexp) lookup-compiler))

Does that make sense?

Ludo’.

[bug#30657] Add support for file-like objects to the Prosody configuration

[Ludovic Courtès]

Hello,

Clément Lassieur <clement@lassieur.org> skribis:

> These patches add support for file-like objects to the Prosody service
> configuration.  The idea is to replace this mechanism (which is used
> very often):
>
>     (plain-file
>       (with-output-to-string
>          ...
>          (format #t ...)
>          ...))
>
> with this one:
>
>     (mixed-text-file
>       (flatten
>         (with-tokens-to-list
>            ...
>            (push-tokens ...)
>            ...))
>
> The point is that tokens don't have to be strings.

That’s a great improvement.

I’m wondering if we could somehow avoid building the token list in this
imperative manner.

We could arrange to procedure a string-value gexp, i.e.:

  #~(string-append #$a #$b #$c)

What if each ‘serialize-*’ procedure produce a string-valued gexp that
can be directly spliced in this big #~(string-append …)?

That would require changing ‘serialize-*’ to return a value (currently
their return value is ignored.)  WDYT?  Doable?  Pointless?

Thanks,
Ludo’.

[bug#30657] [PATCH] services: messaging: Prosody config supports file-like objects.

[Clément Lassieur]

* doc/guix.texi (Messaging Services): Update accordingly.
* gnu/services/configuration.scm (serialize-configuration,
serialize-maybe-stem, serialize-package): Return strings or string-valued
gexps (these procedures were only used for their side-effects).
(file-like?): New exported procedure.
* gnu/services/messaging.scm (serialize-field, serialize-field-list,
enclose-quotes, serialize-raw-content, serialize-ssl-configuration,
serialize-virtualhost-configuration-list,
serialize-int-component-configuration-list,
serialize-ext-component-configuration-list,
serialize-virtualhost-configuration, serialize-int-component-configuration,
serialize-ext-component-configuration, serialize-prosody-configuration):
Return strings or string-valued gexps and stop printing.
(prosody-activation): Use SERIALIZE-PROSODY-CONFIGURATION's return value with
MIXED-TEXT-FILE instead of using its output with PLAIN-FILE.
(serialize-non-negative-integer, serialize-non-negative-integer-list): Convert
numbers to strings.
(file-object?, serialize-file-object, file-object-list?,
serialize-file-object-list): New procedures.
(ssl-configuration)[capath, cafile], (prosody-configuration)[plugin-paths,
groups-file]: Replace FILE-NAME with FILE-OBJECT.
---
 doc/guix.texi                  |  13 +++--
 gnu/services/configuration.scm |  23 +++++----
 gnu/services/messaging.scm     | 111 ++++++++++++++++++++++++-----------------
 3 files changed, 87 insertions(+), 60 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 24db16761..3bb544e62 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14210,6 +14210,9 @@ There is also a way to specify the configuration as a string, if you
 have an old @code{prosody.cfg.lua} file that you want to port over from
 some other system; see the end for more details.
 
+The @code{file-object} type designates either a file-like object
+(@pxref{G-Expressions, file-like objects}) or a file name.
+
 @c The following documentation was initially generated by
 @c (generate-documentation) in (gnu services messaging).  Manually maintained
 @c documentation is better, so we shouldn't hesitate to edit below as
@@ -14230,7 +14233,7 @@ Location of the Prosody data storage directory.  See
 Defaults to @samp{"/var/lib/prosody"}.
 @end deftypevr
 
-@deftypevr {@code{prosody-configuration} parameter} file-name-list plugin-paths
+@deftypevr {@code{prosody-configuration} parameter} file-object-list plugin-paths
 Additional plugin directories.  They are searched in all the specified
 paths in order.  See @url{https://prosody.im/doc/plugins_directory}.
 Defaults to @samp{()}.
@@ -14271,7 +14274,7 @@ should you want to disable them then add them to this list.
 Defaults to @samp{()}.
 @end deftypevr
 
-@deftypevr {@code{prosody-configuration} parameter} file-name groups-file
+@deftypevr {@code{prosody-configuration} parameter} file-object groups-file
 Path to a text file where the shared groups are defined.  If this path is
 empty then @samp{mod_groups} does nothing.  See
 @url{https://prosody.im/doc/modules/mod_groups}.
@@ -14304,13 +14307,13 @@ Path to your private key file.
 Path to your certificate file.
 @end deftypevr
 
-@deftypevr {@code{ssl-configuration} parameter} file-name capath
+@deftypevr {@code{ssl-configuration} parameter} file-object capath
 Path to directory containing root certificates that you wish Prosody to
 trust when verifying the certificates of remote servers.
 Defaults to @samp{"/etc/ssl/certs"}.
 @end deftypevr
 
-@deftypevr {@code{ssl-configuration} parameter} maybe-file-name cafile
+@deftypevr {@code{ssl-configuration} parameter} maybe-file-object cafile
 Path to a file containing root certificates that you wish Prosody to trust.
 Similar to @code{capath} but with all certificates concatenated together.
 @end deftypevr
@@ -14570,6 +14573,8 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@c end of Prosody auto-generated documentation
+
 @subsubheading BitlBee Service
 
 @cindex IRC (Internet Relay Chat)
diff --git a/gnu/services/configuration.scm b/gnu/services/configuration.scm
index c45340f02..016ef4e0e 100644
--- a/gnu/services/configuration.scm
+++ b/gnu/services/configuration.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015 Andy Wingo <wingo@igalia.com>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
-;;; Copyright © 2017 Clément Lassieur <clement@lassieur.org>
+;;; Copyright © 2017, 2018 Clément Lassieur <clement@lassieur.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -42,7 +42,8 @@
             define-configuration
             validate-configuration
             generate-documentation
-            serialize-package))
+            serialize-package
+            file-like?))
 
 ;;; Commentary:
 ;;;
@@ -74,11 +75,12 @@
   (documentation configuration-field-documentation))
 
 (define (serialize-configuration config fields)
-  (for-each (lambda (field)
-              ((configuration-field-serializer field)
-               (configuration-field-name field)
-               ((configuration-field-getter field) config)))
-            fields))
+  #~(string-append
+     #$@(map (lambda (field)
+               ((configuration-field-serializer field)
+                (configuration-field-name field)
+                ((configuration-field-getter field) config)))
+             fields)))
 
 (define (validate-configuration config fields)
   (for-each (lambda (field)
@@ -105,7 +107,7 @@
              (define (maybe-stem? val)
                (or (eq? val 'disabled) (stem? val)))
              (define (serialize-maybe-stem field-name val)
-               (when (stem? val) (serialize-stem field-name val)))))))))
+               (if (stem? val) (serialize-stem field-name val) ""))))))))
 
 (define-syntax define-configuration
   (lambda (stx)
@@ -147,7 +149,7 @@
                    conf))))))))
 
 (define (serialize-package field-name val)
-  #f)
+  "")
 
 ;; A little helper to make it easier to document all those fields.
 (define (generate-documentation documentation documentation-name)
@@ -182,3 +184,6 @@
                       (or (assq-ref sub-documentation field-name) '())))))
             fields)))))
   (stexi->texi `(*fragment* . ,(generate documentation-name))))
+
+(define (file-like? val)
+  (and (struct? val) ((@@ (guix gexp) lookup-compiler) val)))
diff --git a/gnu/services/messaging.scm b/gnu/services/messaging.scm
index 427e2121f..aa3d5f0a0 100644
--- a/gnu/services/messaging.scm
+++ b/gnu/services/messaging.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017 Clément Lassieur <clement@lassieur.org>
+;;; Copyright © 2017, 2018 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2015, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
 ;;;
@@ -115,16 +115,16 @@
                  "_")))
 
 (define (serialize-field field-name val)
-  (format #t "~a = ~a;\n" (uglify-field-name field-name) val))
+  #~(string-append
+     #$(format #f "~a = " (uglify-field-name field-name)) #$val ";\n"))
 (define (serialize-field-list field-name val)
   (serialize-field field-name
-                   (with-output-to-string
-                     (lambda ()
-                       (format #t "{\n")
-                       (for-each (lambda (x)
-                                   (format #t "~a;\n" x))
-                                 val)
-                       (format #t "}")))))
+                   #~(string-append
+                      "{\n"
+                      #$@(map (lambda (x)
+                                #~(string-append #$x ";\n"))
+                              val)
+                      "}")))
 
 (define (serialize-boolean field-name val)
   (serialize-field field-name (if val "true" "false")))
@@ -140,17 +140,17 @@
 (define (non-negative-integer? val)
   (and (exact-integer? val) (not (negative? val))))
 (define (serialize-non-negative-integer field-name val)
-  (serialize-field field-name val))
+  (serialize-field field-name (number->string val)))
 (define-maybe non-negative-integer)
 
 (define (non-negative-integer-list? val)
   (and (list? val) (and-map non-negative-integer? val)))
 (define (serialize-non-negative-integer-list field-name val)
-  (serialize-field-list field-name val))
+  (serialize-field-list field-name (map number->string val)))
 (define-maybe non-negative-integer-list)
 
 (define (enclose-quotes s)
-  (format #f "\"~a\"" s))
+  #~(string-append "\"" #$s "\""))
 (define (serialize-string field-name val)
   (serialize-field field-name (enclose-quotes val)))
 (define-maybe string)
@@ -183,10 +183,22 @@
   (serialize-string-list field-name val))
 (define-maybe file-name)
 
+(define (file-object? val)
+  (or (file-like? val) (file-name? val)))
+(define (serialize-file-object field-name val)
+  (serialize-string field-name val))
+(define-maybe file-object)
+
+(define (file-object-list? val)
+  (and (list? val) (and-map file-object? val)))
+(define (serialize-file-object-list field-name val)
+  (serialize-string-list field-name val))
+(define-maybe file-object)
+
 (define (raw-content? val)
   (not (eq? val 'disabled)))
 (define (serialize-raw-content field-name val)
-  (format #t "~a" val))
+  val)
 (define-maybe raw-content)
 
 (define-configuration mod-muc-configuration
@@ -224,12 +236,12 @@ just joined the room."))
    "Path to your certificate file.")
 
   (capath
-   (file-name "/etc/ssl/certs")
+   (file-object "/etc/ssl/certs")
    "Path to directory containing root certificates that you wish Prosody to
 trust when verifying the certificates of remote servers.")
 
   (cafile
-   (maybe-file-name 'disabled)
+   (maybe-file-object 'disabled)
    "Path to a file containing root certificates that you wish Prosody to trust.
 Similar to @code{capath} but with all certificates concatenated together.")
 
@@ -273,9 +285,10 @@ can create such a file with:
    (maybe-string 'disabled)
    "Password for encrypted private keys."))
 (define (serialize-ssl-configuration field-name val)
-  (format #t "ssl = {\n")
-  (serialize-configuration val ssl-configuration-fields)
-  (format #t "};\n"))
+  #~(string-append
+     "ssl = {\n"
+     #$(serialize-configuration val ssl-configuration-fields)
+     "};\n"))
 (define-maybe ssl-configuration)
 
 (define %default-modules-enabled
@@ -303,20 +316,23 @@ can create such a file with:
   (define (virtualhost-configuration-list? val)
     (and (list? val) (and-map virtualhost-configuration? val)))
   (define (serialize-virtualhost-configuration-list l)
-    (for-each
-     (lambda (val) (serialize-virtualhost-configuration val)) l))
+    #~(string-append
+       #$@(map (lambda (val)
+                 (serialize-virtualhost-configuration val)) l)))
 
   (define (int-component-configuration-list? val)
     (and (list? val) (and-map int-component-configuration? val)))
   (define (serialize-int-component-configuration-list l)
-    (for-each
-     (lambda (val) (serialize-int-component-configuration val)) l))
+    #~(string-append
+       #$@(map (lambda (val)
+                 (serialize-int-component-configuration val)) l)))
 
   (define (ext-component-configuration-list? val)
     (and (list? val) (and-map ext-component-configuration? val)))
   (define (serialize-ext-component-configuration-list l)
-    (for-each
-     (lambda (val) (serialize-ext-component-configuration val)) l))
+    #~(string-append
+       #$@(map (lambda (val)
+                 (serialize-ext-component-configuration val)) l)))
 
   (define-all-configurations prosody-configuration
     (prosody
@@ -331,7 +347,7 @@ can create such a file with:
      global)
 
     (plugin-paths
-     (file-name-list '())
+     (file-object-list '())
      "Additional plugin directories.  They are searched in all the specified
 paths in order.  See @url{https://prosody.im/doc/plugins_directory}."
      global)
@@ -372,7 +388,7 @@ should you want to disable them then add them to this list."
      common)
 
     (groups-file
-     (file-name "/var/lib/prosody/sharedgroups.txt")
+     (file-object "/var/lib/prosody/sharedgroups.txt")
      "Path to a text file where the shared groups are defined.  If this path is
 empty then @samp{mod_groups} does nothing.  See
 @url{https://prosody.im/doc/modules/mod_groups}."
@@ -566,8 +582,9 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
                '(domain))))
   (let ((domain (virtualhost-configuration-domain config))
         (rest (filter rest? virtualhost-configuration-fields)))
-    (format #t "VirtualHost \"~a\"\n" domain)
-    (serialize-configuration config rest)))
+    #~(string-append
+       #$(format #f "VirtualHost \"~a\"\n" domain)
+       #$(serialize-configuration config rest))))
 
 ;; Serialize Component line first.
 (define (serialize-int-component-configuration config)
@@ -577,8 +594,9 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
   (let ((hostname (int-component-configuration-hostname config))
         (plugin (int-component-configuration-plugin config))
         (rest (filter rest? int-component-configuration-fields)))
-    (format #t "Component \"~a\" \"~a\"\n" hostname plugin)
-    (serialize-configuration config rest)))
+    #~(string-append
+       #$(format #f "Component \"~a\" \"~a\"\n" hostname plugin)
+       #$(serialize-configuration config rest))))
 
 ;; Serialize Component line first.
 (define (serialize-ext-component-configuration config)
@@ -587,22 +605,24 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
                '(hostname))))
   (let ((hostname (ext-component-configuration-hostname config))
         (rest (filter rest? ext-component-configuration-fields)))
-    (format #t "Component \"~a\"\n" hostname)
-    (serialize-configuration config rest)))
+    #~(string-append
+       #$(format #f "Component \"~a\"\n" hostname)
+       #$(serialize-configuration #$config #$rest))))
 
 ;; Serialize virtualhosts and components last.
 (define (serialize-prosody-configuration config)
   (define (rest? field)
     (not (memq (configuration-field-name field)
                '(virtualhosts int-components ext-components))))
-  (let ((rest (filter rest? prosody-configuration-fields)))
-    (serialize-configuration config rest))
-  (serialize-virtualhost-configuration-list
-   (prosody-configuration-virtualhosts config))
-  (serialize-int-component-configuration-list
-   (prosody-configuration-int-components config))
-  (serialize-ext-component-configuration-list
-   (prosody-configuration-ext-components config)))
+  #~(string-append
+     #$(let ((rest (filter rest? prosody-configuration-fields)))
+         (serialize-configuration config rest))
+     #$(serialize-virtualhost-configuration-list
+        (prosody-configuration-virtualhosts config))
+     #$(serialize-int-component-configuration-list
+        (prosody-configuration-int-components config))
+     #$(serialize-ext-component-configuration-list
+        (prosody-configuration-ext-components config))))
 
 (define-configuration opaque-prosody-configuration
   (prosody
@@ -646,13 +666,10 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
          (default-certs-dir "/etc/prosody/certs")
          (data-path (prosody-configuration-data-path config))
          (pidfile-dir (dirname (prosody-configuration-pidfile config)))
-         (config-str
-          (if (opaque-prosody-configuration? config)
-              (opaque-prosody-configuration-prosody.cfg.lua config)
-              (with-output-to-string
-                (lambda ()
-                  (serialize-prosody-configuration config)))))
-         (config-file (plain-file "prosody.cfg.lua" config-str)))
+         (config-str (if (opaque-prosody-configuration? config)
+                         (opaque-prosody-configuration-prosody.cfg.lua config)
+                         (serialize-prosody-configuration config)))
+         (config-file (mixed-text-file "prosody.cfg.lua" config-str)))
     #~(begin
         (use-modules (guix build utils))
         (define %user (getpw "prosody"))
-- 
2.16.2

[bug#30657] Add support for file-like objects to the Prosody configuration

[Clément Lassieur]

Hello Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> I’m wondering if we could somehow avoid building the token list in this
> imperative manner.

Indeed it's much better if it's functional!  I just sent a new patch.
WDYT?

Thanks,
Clément

[bug#30657] [PATCH 2/4] gexp: Add 'file-like?'.

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

>> +(define (file-like? object)
>> +  (or (local-file? object)
>> +      (plain-file? object)
>> +      (computed-file? object)
>> +      (program-file? object)
>> +      (scheme-file? object)
>> +      (file-append? object)))
>
> This procedure would miss new types added with ‘define-gexp-compiler’.
>
> In fact I think you can simply write:
>
>   (define file-like? (@@ (guix gexp) lookup-compiler))
>
> Does that make sense?

Yes it's great!  One also needs to check that it's a struct though.  I
put it in (gnu services configuration).  Would it make sense to add the
'struct?' check in (guix gexp), or to add 'file-like?' there?

Clément

[bug#30657] [PATCH] services: messaging: Prosody config supports file-like objects.

[Clément Lassieur]

With a better use of 'format'...

Clément Lassieur <clement@lassieur.org> writes:

>  (define (serialize-field field-name val)
> -  (format #t "~a = ~a;\n" (uglify-field-name field-name) val))
> +  #~(string-append
> +     #$(format #f "~a = " (uglify-field-name field-name)) #$val ";\n"))

#~(format #f "~a = ~a;\n" #$(uglify-field-name field-name) #$val))

>  (define (serialize-field-list field-name val)
>    (serialize-field field-name
> -                   (with-output-to-string
> -                     (lambda ()
> -                       (format #t "{\n")
> -                       (for-each (lambda (x)
> -                                   (format #t "~a;\n" x))
> -                                 val)
> -                       (format #t "}")))))
> +                   #~(string-append
> +                      "{\n"
> +                      #$@(map (lambda (x)
> +                                #~(string-append #$x ";\n"))
> +                              val)
> +                      "}")))

(ice-9 format) can do miracles ;-)

(serialize-field field-name #~(format #f "{\n~@{~a;\n~}}" #$@val)))

>  (define (enclose-quotes s)
> -  (format #f "\"~a\"" s))
> +  #~(string-append "\"" #$s "\""))

#~(format #f "\"~a\"" #$s))

> @@ -273,9 +285,10 @@ can create such a file with:
>     (maybe-string 'disabled)
>     "Password for encrypted private keys."))
>  (define (serialize-ssl-configuration field-name val)
> -  (format #t "ssl = {\n")
> -  (serialize-configuration val ssl-configuration-fields)
> -  (format #t "};\n"))
> +  #~(string-append
> +     "ssl = {\n"
> +     #$(serialize-configuration val ssl-configuration-fields)
> +     "};\n"))

#~(format #f "ssl = {\n~a};\n"
          #$(serialize-configuration val ssl-configuration-fields)))

>  (define-configuration opaque-prosody-configuration
>    (prosody
> @@ -646,13 +666,10 @@ See also @url{https://prosody.im/doc/modules/mod_muc}."
>           (default-certs-dir "/etc/prosody/certs")
>           (data-path (prosody-configuration-data-path config))
>           (pidfile-dir (dirname (prosody-configuration-pidfile config)))
> -         (config-str
> -          (if (opaque-prosody-configuration? config)
> -              (opaque-prosody-configuration-prosody.cfg.lua config)
> -              (with-output-to-string
> -                (lambda ()
> -                  (serialize-prosody-configuration config)))))
> -         (config-file (plain-file "prosody.cfg.lua" config-str)))
> +         (config-str (if (opaque-prosody-configuration? config)
> +                         (opaque-prosody-configuration-prosody.cfg.lua config)
> +                         (serialize-prosody-configuration config)))
> +         (config-file (mixed-text-file "prosody.cfg.lua" config-str)))
>      #~(begin
>          (use-modules (guix build utils))
>          (define %user (getpw "prosody"))

                   (with-imported-modules '((ice-9 format))
                     #~(begin
                         (use-modules (ice-9 format))
                         #$(serialize-prosody-configuration config)))))

[bug#30657] [PATCH] services: messaging: Prosody config supports file-like objects.

[Ludovic Courtès]

Hi!

Clément Lassieur <clement@lassieur.org> skribis:

> Clément Lassieur <clement@lassieur.org> writes:
>
>>  (define (serialize-field field-name val)
>> -  (format #t "~a = ~a;\n" (uglify-field-name field-name) val))
>> +  #~(string-append
>> +     #$(format #f "~a = " (uglify-field-name field-name)) #$val ";\n"))
>
> #~(format #f "~a = ~a;\n" #$(uglify-field-name field-name) #$val))
>
>>  (define (serialize-field-list field-name val)
>>    (serialize-field field-name
>> -                   (with-output-to-string
>> -                     (lambda ()
>> -                       (format #t "{\n")
>> -                       (for-each (lambda (x)
>> -                                   (format #t "~a;\n" x))
>> -                                 val)
>> -                       (format #t "}")))))
>> +                   #~(string-append
>> +                      "{\n"
>> +                      #$@(map (lambda (x)
>> +                                #~(string-append #$x ";\n"))
>> +                              val)
>> +                      "}")))
>
> (ice-9 format) can do miracles ;-)
>
> (serialize-field field-name #~(format #f "{\n~@{~a;\n~}}" #$@val)))

Indeed, though you need to make sure (ice-9 format) is in scope on the
build side (the default ‘format’, aka. ‘simple-format’, doesn’t support
anything beyond ~a, ~s, and ~%).

>>  (define (enclose-quotes s)
>> -  (format #f "\"~a\"" s))
>> +  #~(string-append "\"" #$s "\""))
>
> #~(format #f "\"~a\"" #$s))

It’s a case where I prefer ‘string-append’ because is ensures that
everything is a string and reports a type error if not.  Conversely,
(format #f "~a" …) will silently convert anything to a string, which may
not be what you want.

>                    (with-imported-modules '((ice-9 format))

This would import (ice-9 format) from the host Guile into the build
environment.  Thus, if you build your system with Guix on Guile 2.2.2
and I build mine on Guile 2.0.14, we end up with different derivations,
which is not desirable.

Instead, what you need is this:

>                      #~(begin
>                          (use-modules (ice-9 format))

That puts (ice-9 format) in scope, which is all you need.  It’s the
(ice-9 format) of the build-side Guile that’s used.

Thanks,
Ludo’.

[bug#30657] [PATCH] services: messaging: Prosody config supports file-like objects.

[Ludovic Courtès]

Hello,

Clément Lassieur <clement@lassieur.org> skribis:

> * doc/guix.texi (Messaging Services): Update accordingly.
> * gnu/services/configuration.scm (serialize-configuration,
> serialize-maybe-stem, serialize-package): Return strings or string-valued
> gexps (these procedures were only used for their side-effects).
> (file-like?): New exported procedure.
> * gnu/services/messaging.scm (serialize-field, serialize-field-list,
> enclose-quotes, serialize-raw-content, serialize-ssl-configuration,
> serialize-virtualhost-configuration-list,
> serialize-int-component-configuration-list,
> serialize-ext-component-configuration-list,
> serialize-virtualhost-configuration, serialize-int-component-configuration,
> serialize-ext-component-configuration, serialize-prosody-configuration):
> Return strings or string-valued gexps and stop printing.
> (prosody-activation): Use SERIALIZE-PROSODY-CONFIGURATION's return value with
> MIXED-TEXT-FILE instead of using its output with PLAIN-FILE.
> (serialize-non-negative-integer, serialize-non-negative-integer-list): Convert
> numbers to strings.
> (file-object?, serialize-file-object, file-object-list?,
> serialize-file-object-list): New procedures.
> (ssl-configuration)[capath, cafile], (prosody-configuration)[plugin-paths,
> groups-file]: Replace FILE-NAME with FILE-OBJECT.

[...]

> +  #~(string-append
> +     #$@(map (lambda (field)
> +               ((configuration-field-serializer field)
> +                (configuration-field-name field)
> +                ((configuration-field-getter field) config)))
> +             fields)))

Awesome, I find it clearer.  :-)

> +(define (file-like? val)
> +  (and (struct? val) ((@@ (guix gexp) lookup-compiler) val)))

I’m not fond of the idea of using @@ in real code.  :-)

Could you move ‘file-like?’ to (guix gexp), with a docstring, and with
‘->bool’ to avoid exposing internal details:

  (define (file-like? value)
    (and (struct? value) (->bool (lookup-compiler value))))

?

> +(define (file-object? val)
> +  (or (file-like? val) (file-name? val)))

Do we need this predicate?  After all, all we can say is that a file
name is necessarily a string (or a string-valued gexp), but a string is
not necessarily a file name (IOW there’s no disjoint type for file
names.)

I suppose the configuration mechanism needs it though, right?  In that
context it’s probably OK.

Apart from this the patch LGTM, thank you!

Ludo’.

[bug#30657] [PATCH 2/4] gexp: Add 'file-like?'.

[Ludovic Courtès]

Clément Lassieur <clement@lassieur.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> +(define (file-like? object)
>>> +  (or (local-file? object)
>>> +      (plain-file? object)
>>> +      (computed-file? object)
>>> +      (program-file? object)
>>> +      (scheme-file? object)
>>> +      (file-append? object)))
>>
>> This procedure would miss new types added with ‘define-gexp-compiler’.
>>
>> In fact I think you can simply write:
>>
>>   (define file-like? (@@ (guix gexp) lookup-compiler))
>>
>> Does that make sense?
>
> Yes it's great!  One also needs to check that it's a struct though.  I
> put it in (gnu services configuration).  Would it make sense to add the
> 'struct?' check in (guix gexp), or to add 'file-like?' there?

Yes, it makes sense to add ‘file-like?’ to (guix gexp) proper, as I
wrote in my other reply.

I was hoping we could avoid that, but I understand there’s a need for it.

Thanks,
Ludo’.

bug#30657: [PATCH] services: messaging: Prosody config supports file-like objects.

[Clément Lassieur]

Ludovic Courtès <ludo@gnu.org> writes:

>> +(define (file-like? val)
>> +  (and (struct? val) ((@@ (guix gexp) lookup-compiler) val)))
>
> I’m not fond of the idea of using @@ in real code.  :-)
>
> Could you move ‘file-like?’ to (guix gexp), with a docstring, and with
> ‘->bool’ to avoid exposing internal details:
>
>   (define (file-like? value)
>     (and (struct? value) (->bool (lookup-compiler value))))
>
> ?

I did it.

>> +(define (file-object? val)
>> +  (or (file-like? val) (file-name? val)))
>
> Do we need this predicate?  After all, all we can say is that a file
> name is necessarily a string (or a string-valued gexp), but a string is
> not necessarily a file name (IOW there’s no disjoint type for file
> names.)
>
> I suppose the configuration mechanism needs it though, right?  In that
> context it’s probably OK.

The problem is that the check happens when the user evaluates
(prosody-configuration), which allow them to know right away about type
errors, without running "guix system reconfigure".  So the user doesn't
even need a store.

Plus, not everything should go to the store.  For example, certificate
keys shouldn't, as they are private.  Thus, having both 'file-object?'
and 'file-name?' makes sense to me.

> Apart from this the patch LGTM, thank you!

Pushed as bdcf0e6fd484a54240a98ddf8b6fa433c1b9bd6c, with the
modifications you suggested.

Thank you for the review!
Clément