[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

* gnu/services/admin.scm
  (<tailon-configuration>, <tailon-configuration-file>): New record types.
  (tailon-configuration-files-string, tailon-shepherd-service): New
  procedures.
  (%tailon-accounts, tailon-service-type: New variables.
* doc/guix.text (Monitoring Services: Document the Tailon service.
---
 doc/guix.texi          |  88 +++++++++++++++++++++++++++++
 gnu/services/admin.scm | 150 ++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 237 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 729ec081b..164dccba1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -218,6 +218,7 @@ Services
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -8901,6 +8902,7 @@ declaration.
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -13489,6 +13491,92 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@node Monitoring Services
+@subsubsection Monitoring Services
+
+@subsubheading Tailon Service
+
+Tailon is a web application for viewing and searching log files.
+
+The following example will configure the service with default values.
+By default, Tailon can be accessed on port 8080 (http://localhost:8080).
+
+@example
+(service tailon-service-type)
+@end example
+
+The following example customises more of the Tailon configuration,
+adding ``sed'' to the list of allowed commands.
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file
+             (tailon-configuration-file
+               (allowed-commands '("tail" "grep" "awk" "sed"))))))
+@end example
+
+
+@deftp {Data Type} tailon-configuration
+Data type representing the configuration of tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{config-file} (default: @code{(tailon-configuration-file)})
+The configuration file to use for Tailon. This can be set to a
+@dfn{tailon-configuration-file} record value, or any gexp
+(@pxref{G-Expressions}).
+
+For example, to instead use a local file, the @code{local-file} function
+can be used:
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file (local-file "./my-tailon.conf"))))
+@end example
+
+@item @code{package} (default: @code{tailon})
+The tailon package to use.
+
+@end table
+@end deftp
+
+@deftp {Data Type} tailon-configuration-file
+Data type representing the configuration options for Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{paths} (default: @code{(list "/var/log")})
+List of paths to display. Can include strings for a single path, or a
+list, where the first item is the name of a subsection, and the
+remaining items are in that subsection.
+
+@item @code{bind} (default: @code{"localhost:8080"})
+Address and port to which Tailon should bind on.
+
+@item @code{relative-root-path} (default: @code{#f})
+Prefix to use for all paths, set to @code{#f} to disable using a prefix.
+
+@item @code{allow-transfers?} (default: @code{#t})
+Allow downloading the log files in the web interface.
+
+@item @code{follow-names?} (default: @code{#t})
+Allow tailing of not-yet existent files.
+
+@item @code{tail-lines} (default: @code{200})
+Number of lines to read initially from each file.
+
+@item @code{allowed-commands} (default: @code{(list "tail" "grep" "awk")})
+Commands to allow running. By default, @code{sed} is disabled.
+
+@item @code{debug?} (default: @code{#f})
+Set @code{debug?} to @code{#t} to show debug messages.
+
+@end table
+@end deftp
+
+
 @node Kerberos Services
 @subsubsection Kerberos Services
 @cindex Kerberos
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index b9e3fa70a..eb1f70ec6 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -20,14 +20,19 @@
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages logging)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
   #:use-module (guix gexp)
+  #:use-module (guix store)
   #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 vlist)
+  #:use-module (ice-9 match)
   #:export (%default-rotations
             %rotated-files
 
@@ -41,7 +46,27 @@
             rottlog-configuration
             rottlog-configuration?
             rottlog-service
-            rottlog-service-type))
+            rottlog-service-type
+
+            <tailon-configuration-file>
+            tailon-configuration-file
+            tailon-configuration-file?
+            tailon-configuration-file-paths
+            tailon-configuration-file-bind
+            tailon-configuration-file-relative-root-path
+            tailon-configuration-file-allow-transfers?
+            tailon-configuration-file-follow-names?
+            tailon-configuration-file-tail-lines
+            tailon-configuration-file-allowed-commands
+            tailon-configuration-file-debug?
+
+            <tailon-configuration>
+            tailon-configuration
+            tailon-configuration?
+            tailon-configuration-config-file
+            tailon-configuration-package
+
+            tailon-service-type))
 
 ;;; Commentary:
 ;;;
@@ -172,4 +197,127 @@ for ROTATION."
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+\f
+;;;
+;;; Tailon
+;;;
+
+(define-record-type* <tailon-configuration-file>
+  tailon-configuration-file make-tailon-configuration-file
+  tailon-configuration-file?
+  (paths              tailon-configuration-paths
+                      (default '("/var/log")))
+  (bind               tailon-configuration-file-bind
+                      (default "localhost:8080"))
+  (relative-root-path tailon-configuration-file-relative-root-path
+                      (default #f))
+  (allow-transfers?   tailon-configuration-file-allow-transfers?
+                      (default #t))
+  (follow-names?      tailon-configuration-file-follow-names?
+                      (default #t))
+  (tail-lines         tailon-configuration-file-tail-lines
+                      (default 200))
+  (allowed-commands   tailon-configuration-file-allowed-commands
+                      (default '("tail" "grep" "awk")))
+  (debug?             tailon-configuration-file-debug?
+                      (default #f)))
+
+(define (tailon-configuration-files-string paths)
+  (string-append
+   "\n"
+   (string-join
+    (map
+     (lambda (x)
+       (string-append
+        "  - "
+        (cond
+         ((string? x)
+          (simple-format #f "'~A'" x))
+         ((list? x)
+          (string-join
+           (cons (simple-format #f "'~A':" (car x))
+                 (map
+                  (lambda (x) (simple-format #f "      - '~A'" x))
+                  (cdr x)))
+           "\n"))
+         (else (error x)))))
+     paths)
+    "\n")))
+
+(define-gexp-compiler (tailon-configuration-file-compiler
+                       (file <tailon-configuration-file>) system target)
+  (match file
+    (($ <tailon-configuration-file> paths bind relative-root-path
+                                    allow-transfers? follow-names?
+                                    tail-lines allowed-commands debug?)
+     (text-file
+      "tailon-config.yaml"
+      (string-concatenate
+       (filter-map
+        (match-lambda
+         ((key . #f) #f)
+         ((key . value) (string-append key ": " value "\n")))
+
+        `(("files" . ,(tailon-configuration-files-string paths))
+          ("bind" . ,bind)
+          ("relative-root" . ,relative-root-path)
+          ("allow-transfers" . ,(if allow-transfers? "true" "false"))
+          ("follow-names" . ,(if follow-names? "true" "false"))
+          ("tail-lines" . ,(number->string tail-lines))
+          ("commands" . ,(string-append "["
+                                        (string-join allowed-commands ", ")
+                                        "]"))
+          ,@(if debug? '(("debug" . "true")) '()))))))))
+
+(define-record-type* <tailon-configuration>
+  tailon-configuration make-tailon-configuration
+  tailon-configuration?
+  (config-file tailon-configuration-config-file
+               (default (tailon-configuration-file)))
+  (package tailon-configuration-package
+           (default tailon)))
+
+(define tailon-shepherd-service
+  (match-lambda
+    (($ <tailon-configuration> config-file package)
+     (list (shepherd-service
+            (provision '(tailon))
+            (documentation "Run the tailon daemon.")
+            (start #~(make-forkexec-constructor
+                      `(,(string-append #$package "/bin/tailon")
+                        "-c" ,#$config-file)
+                      #:user "tailon"
+                      #:group "tailon"))
+            (stop #~(make-kill-destructor)))))))
+
+(define %tailon-accounts
+  (list (user-group (name "tailon") (system? #t))
+        (user-account
+         (name "tailon")
+         (group "tailon")
+         (system? #t)
+         (comment "tailon")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define tailon-service-type
+  (service-type
+   (name 'tailon)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             tailon-shepherd-service)
+          (service-extension account-service-type
+                             (const %tailon-accounts))))
+   (compose concatenate)
+   (extend (lambda (parameter paths)
+             (tailon-configuration
+              (inherit parameter)
+              (config-file
+               (let ((old-config-file (tailon-configuration-config-file parameter)))
+                 (tailon-configuration-file
+                  (inherit old-config-file)
+                  (paths (append (tailon-configuration-paths old-config-file)
+                                 paths))))))))
+   (default-value (tailon-configuration))))
+
 ;;; admin.scm ends here
-- 
2.13.1

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Hi Christopher,

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/admin.scm
>   (<tailon-configuration>, <tailon-configuration-file>): New record types.
>   (tailon-configuration-files-string, tailon-shepherd-service): New
>   procedures.
>   (%tailon-accounts, tailon-service-type: New variables.
> * doc/guix.text (Monitoring Services: Document the Tailon service.

Neat.  Maybe a service we could take advantage of on our build farm
machines?

s/text/texi/  :-)

> +@node Monitoring Services
> +@subsubsection Monitoring Services
> +
> +@subsubheading Tailon Service
> +
> +Tailon is a web application for viewing and searching log files.

Please write @uref{https://…, Tailon} so readers can read more about it
if needed.

> +The following example will configure the service with default values.
> +By default, Tailon can be accessed on port 8080 (http://localhost:8080).
                                                    ^
@code

> +The following example customises more of the Tailon configuration,
> +adding ``sed'' to the list of allowed commands.
          ^^
@command{sed}

> +@deftp {Data Type} tailon-configuration
> +Data type representing the configuration of tailon.

Tailon (capital).

> +@table @asis
> +@item @code{paths} (default: @code{(list "/var/log")})
> +List of paths to display. Can include strings for a single path, or a
> +list, where the first item is the name of a subsection, and the
> +remaining items are in that subsection.

Please s/paths/directories/ (in GNU the convention is to use “file name”
or “directory name”, whereas “path” is reserved for search paths.)

> +  (paths              tailon-configuration-paths
> +                      (default '("/var/log")))
> +  (bind               tailon-configuration-file-bind
> +                      (default "localhost:8080"))
> +  (relative-root-path tailon-configuration-file-relative-root-path
> +                      (default #f))

Same here.

> +(define-gexp-compiler (tailon-configuration-file-compiler
> +                       (file <tailon-configuration-file>) system target)

That’s a good idea.  :-)

Otherwise LGTM!

Bonus points if you write a system test that checks that at least the
Web server shows up on port 8080.

Thank you!

Ludo’.

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

On Wed, 12 Jul 2017 14:13:25 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Hi Christopher,
> 
> Christopher Baines <mail@cbaines.net> skribis:
> 
> > * gnu/services/admin.scm
> >   (<tailon-configuration>, <tailon-configuration-file>): New record
> > types. (tailon-configuration-files-string,
> > tailon-shepherd-service): New procedures.
> >   (%tailon-accounts, tailon-service-type: New variables.
> > * doc/guix.text (Monitoring Services: Document the Tailon service.  
> 
> Neat.  Maybe a service we could take advantage of on our build farm
> machines?

Thanks for your review, I'll send an updated patch with the texinfo
improvements, and the path/file/directory naming changes.

> > +(define-gexp-compiler (tailon-configuration-file-compiler
> > +                       (file <tailon-configuration-file>) system
> > target)  
> 
> That’s a good idea.  :-)

I'm glad you think so. This pattern should make either using the
provided record type, or something else very easy, as there are no
special cases in the service code.

> Bonus points if you write a system test that checks that at least the
> Web server shows up on port 8080.

I've had a go at this, I'll send a separate patch. It doesn't pass for
me yet, and I'm not sure how to work out what is going on inside the
test system, does anyone have any tips for writing and debugging system
tests in general?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> On Wed, 12 Jul 2017 14:13:25 +0200

[...]

>> Bonus points if you write a system test that checks that at least the
>> Web server shows up on port 8080.
>
> I've had a go at this, I'll send a separate patch. It doesn't pass for
> me yet, and I'm not sure how to work out what is going on inside the
> test system, does anyone have any tips for writing and debugging system
> tests in general?

No specific tips.  Beware of QEMU port forwarding: it doesn’t allow you
to really diagnose what’s going on if you can’t connect from the host to
the guest (it’s hard to tell if that’s because the service in the guest
is not running.)  So on a few occasions I found that I’d first try
connecting from within the guest as my first attempt (using
‘marionette-eval’), to see if this would work at all.

Sometimes you’ll also want to just run a VM of the system under test
interactively so experiment a bit.

HTH,
Ludo’.

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

* gnu/services/admin.scm
  (<tailon-configuration>, <tailon-configuration-file>): New record types.
  (tailon-configuration-files-string, tailon-shepherd-service): New
  procedures.
  (%tailon-accounts, tailon-service-type: New variables.
* doc/guix.texi (Monitoring Services: Document the Tailon service.
---
 doc/guix.texi          |  89 +++++++++++++++++++++++++++++
 gnu/services/admin.scm | 151 ++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 239 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ecf961772..1226e1721 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -218,6 +218,7 @@ Services
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -8921,6 +8922,7 @@ declaration.
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -13509,6 +13511,93 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@node Monitoring Services
+@subsubsection Monitoring Services
+
+@subsubheading Tailon Service
+
+@uref{https://tailon.readthedocs.io/, Tailon} is a web application for
+viewing and searching log files.
+
+The following example will configure the service with default values.
+By default, Tailon can be accessed on port 8080 (@code{http://localhost:8080}).
+
+@example
+(service tailon-service-type)
+@end example
+
+The following example customises more of the Tailon configuration,
+adding @command{sed} to the list of allowed commands.
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file
+             (tailon-configuration-file
+               (allowed-commands '("tail" "grep" "awk" "sed"))))))
+@end example
+
+
+@deftp {Data Type} tailon-configuration
+Data type representing the configuration of Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{config-file} (default: @code{(tailon-configuration-file)})
+The configuration file to use for Tailon. This can be set to a
+@dfn{tailon-configuration-file} record value, or any gexp
+(@pxref{G-Expressions}).
+
+For example, to instead use a local file, the @code{local-file} function
+can be used:
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file (local-file "./my-tailon.conf"))))
+@end example
+
+@item @code{package} (default: @code{tailon})
+The tailon package to use.
+
+@end table
+@end deftp
+
+@deftp {Data Type} tailon-configuration-file
+Data type representing the configuration options for Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{files} (default: @code{(list "/var/log")})
+List of files to display. The list can include strings for a single
+file, or a list, where the first item is the name of a subsection, and
+the remaining items are the files in that subsection.
+
+@item @code{bind} (default: @code{"localhost:8080"})
+Address and port to which Tailon should bind on.
+
+@item @code{relative-root-path} (default: @code{#f})
+Prefix to use for all paths, set to @code{#f} to disable using a prefix.
+
+@item @code{allow-transfers?} (default: @code{#t})
+Allow downloading the log files in the web interface.
+
+@item @code{follow-names?} (default: @code{#t})
+Allow tailing of not-yet existent files.
+
+@item @code{tail-lines} (default: @code{200})
+Number of lines to read initially from each file.
+
+@item @code{allowed-commands} (default: @code{(list "tail" "grep" "awk")})
+Commands to allow running. By default, @code{sed} is disabled.
+
+@item @code{debug?} (default: @code{#f})
+Set @code{debug?} to @code{#t} to show debug messages.
+
+@end table
+@end deftp
+
+
 @node Kerberos Services
 @subsubsection Kerberos Services
 @cindex Kerberos
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index b9e3fa70a..26b5c6edb 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -20,14 +20,19 @@
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages logging)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
   #:use-module (guix gexp)
+  #:use-module (guix store)
   #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 vlist)
+  #:use-module (ice-9 match)
   #:export (%default-rotations
             %rotated-files
 
@@ -41,7 +46,27 @@
             rottlog-configuration
             rottlog-configuration?
             rottlog-service
-            rottlog-service-type))
+            rottlog-service-type
+
+            <tailon-configuration-file>
+            tailon-configuration-file
+            tailon-configuration-file?
+            tailon-configuration-file-files
+            tailon-configuration-file-bind
+            tailon-configuration-file-relative-root-directory
+            tailon-configuration-file-allow-transfers?
+            tailon-configuration-file-follow-names?
+            tailon-configuration-file-tail-lines
+            tailon-configuration-file-allowed-commands
+            tailon-configuration-file-debug?
+
+            <tailon-configuration>
+            tailon-configuration
+            tailon-configuration?
+            tailon-configuration-config-file
+            tailon-configuration-package
+
+            tailon-service-type))
 
 ;;; Commentary:
 ;;;
@@ -172,4 +197,128 @@ for ROTATION."
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+\f
+;;;
+;;; Tailon
+;;;
+
+(define-record-type* <tailon-configuration-file>
+  tailon-configuration-file make-tailon-configuration-file
+  tailon-configuration-file?
+  (files                   tailon-configuration-file-files
+                           (default '("/var/log")))
+  (bind                    tailon-configuration-file-bind
+                           (default "localhost:8080"))
+  (relative-root-directory tailon-configuration-file-relative-root-directory
+                           (default #f))
+  (allow-transfers?        tailon-configuration-file-allow-transfers?
+                           (default #t))
+  (follow-names?           tailon-configuration-file-follow-names?
+                           (default #t))
+  (tail-lines              tailon-configuration-file-tail-lines
+                           (default 200))
+  (allowed-commands        tailon-configuration-file-allowed-commands
+                           (default '("tail" "grep" "awk")))
+  (debug?                  tailon-configuration-file-debug?
+                           (default #f)))
+
+(define (tailon-configuration-files-string files)
+  (string-append
+   "\n"
+   (string-join
+    (map
+     (lambda (x)
+       (string-append
+        "  - "
+        (cond
+         ((string? x)
+          (simple-format #f "'~A'" x))
+         ((list? x)
+          (string-join
+           (cons (simple-format #f "'~A':" (car x))
+                 (map
+                  (lambda (x) (simple-format #f "      - '~A'" x))
+                  (cdr x)))
+           "\n"))
+         (else (error x)))))
+     files)
+    "\n")))
+
+(define-gexp-compiler (tailon-configuration-file-compiler
+                       (file <tailon-configuration-file>) system target)
+  (match file
+    (($ <tailon-configuration-file> files bind relative-root-directory
+                                    allow-transfers? follow-names?
+                                    tail-lines allowed-commands debug?)
+     (text-file
+      "tailon-config.yaml"
+      (string-concatenate
+       (filter-map
+        (match-lambda
+         ((key . #f) #f)
+         ((key . value) (string-append key ": " value "\n")))
+
+        `(("files" . ,(tailon-configuration-files-string files))
+          ("bind" . ,bind)
+          ("relative-root" . ,relative-root-directory)
+          ("allow-transfers" . ,(if allow-transfers? "true" "false"))
+          ("follow-names" . ,(if follow-names? "true" "false"))
+          ("tail-lines" . ,(number->string tail-lines))
+          ("commands" . ,(string-append "["
+                                        (string-join allowed-commands ", ")
+                                        "]"))
+          ,@(if debug? '(("debug" . "true")) '()))))))))
+
+(define-record-type* <tailon-configuration>
+  tailon-configuration make-tailon-configuration
+  tailon-configuration?
+  (config-file tailon-configuration-config-file
+               (default (tailon-configuration-file)))
+  (package tailon-configuration-package
+           (default tailon)))
+
+(define tailon-shepherd-service
+  (match-lambda
+    (($ <tailon-configuration> config-file package)
+     (list (shepherd-service
+            (provision '(tailon))
+            (documentation "Run the tailon daemon.")
+            (start #~(make-forkexec-constructor
+                      `(,(string-append #$package "/bin/tailon")
+                        "-c" ,#$config-file)
+                      #:user "tailon"
+                      #:group "tailon"))
+            (stop #~(make-kill-destructor)))))))
+
+(define %tailon-accounts
+  (list (user-group (name "tailon") (system? #t))
+        (user-account
+         (name "tailon")
+         (group "tailon")
+         (system? #t)
+         (comment "tailon")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define tailon-service-type
+  (service-type
+   (name 'tailon)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             tailon-shepherd-service)
+          (service-extension account-service-type
+                             (const %tailon-accounts))))
+   (compose concatenate)
+   (extend (lambda (parameter files)
+             (tailon-configuration
+              (inherit parameter)
+              (config-file
+               (let ((old-config-file
+                      (tailon-configuration-config-file parameter)))
+                 (tailon-configuration-file
+                  (inherit old-config-file)
+                  (files (append (tailon-configuration-file-files old-config-file)
+                                 files))))))))
+   (default-value (tailon-configuration))))
+
 ;;; admin.scm ends here
-- 
2.13.1

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/admin.scm
>   (<tailon-configuration>, <tailon-configuration-file>): New record types.
>   (tailon-configuration-files-string, tailon-shepherd-service): New
>   procedures.
>   (%tailon-accounts, tailon-service-type: New variables.
> * doc/guix.texi (Monitoring Services: Document the Tailon service.

[...]

> +@deftp {Data Type} tailon-configuration-file
> +Data type representing the configuration options for Tailon.
> +This type has the following parameters:
> +
> +@table @asis
> +@item @code{files} (default: @code{(list "/var/log")})
> +List of files to display. The list can include strings for a single
> +file, or a list, where the first item is the name of a subsection, and

s/file/file or directory/ (I assume it recurses into directories?)

> +@item @code{relative-root-path} (default: @code{#f})
> +Prefix to use for all paths, set to @code{#f} to disable using a prefix.

s/path/directory/ :-)

OK to push with these changes, thank you!

Ludo’.

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

On Mon, 17 Jul 2017 11:52:57 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail@cbaines.net> skribis:
> 
> > * gnu/services/admin.scm
> >   (<tailon-configuration>, <tailon-configuration-file>): New record
> > types. (tailon-configuration-files-string,
> > tailon-shepherd-service): New procedures.
> >   (%tailon-accounts, tailon-service-type: New variables.
> > * doc/guix.texi (Monitoring Services: Document the Tailon service.  
> 
> [...]
> 
> > +@deftp {Data Type} tailon-configuration-file
> > +Data type representing the configuration options for Tailon.
> > +This type has the following parameters:
> > +
> > +@table @asis
> > +@item @code{files} (default: @code{(list "/var/log")})
> > +List of files to display. The list can include strings for a single
> > +file, or a list, where the first item is the name of a subsection,
> > and  
> 
> s/file/file or directory/ (I assume it recurses into directories?)

Yep, it shows the files in the directory. I've sent an updated patch
with this change.

> > +@item @code{relative-root-path} (default: @code{#f})
> > +Prefix to use for all paths, set to @code{#f} to disable using a
> > prefix.  
> 
> s/path/directory/ :-)

So, it turns out I'd completely misinterpreted this option, I was
thinking it was about the log files, as in you would set it to /var/log
if you wanted to specify filenames relative to there. That is wrong,
its actually about the URL path, so if you want to have Tailon at
http://localhost/tailon/ you set the relative-root to /tailon/. I've
now updated the code and docs for this.

> OK to push with these changes, thank you!

I've sent the two patches again. I also was able to debug the issue I
was having with the system test, I needed to get Tailon to bind to
0.0.0.0, rather than localhost for it to work, and with that change,
the system test works!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27650] [PATCH] tests: Add 'tailon-service-type' test.

[Christopher Baines]

* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* gnu/tests/web.scm: New file.
---
 gnu/local.mk        |  1 +
 gnu/tests/admin.scm | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
 create mode 100644 gnu/tests/admin.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 50fac3040..4bb7ce901 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -476,6 +476,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/build/vm.scm				\
 						\
   %D%/tests.scm					\
+  %D%/tests/admin.scm				\
   %D%/tests/base.scm				\
   %D%/tests/dict.scm				\
   %D%/tests/nfs.scm				\
diff --git a/gnu/tests/admin.scm b/gnu/tests/admin.scm
new file mode 100644
index 000000000..5e2424bdb
--- /dev/null
+++ b/gnu/tests/admin.scm
@@ -0,0 +1,96 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Christopher Baines <mail@cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests admin)
+  #:use-module (gnu tests)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system vm)
+  #:use-module (gnu services)
+  #:use-module (gnu services admin)
+  #:use-module (gnu services networking)
+  #:use-module (guix gexp)
+  #:use-module (guix store)
+  #:use-module (guix monads)
+  #:export (%test-tailon))
+
+(define %tailon-os
+  ;; Operating system under test.
+  (simple-operating-system
+   (dhcp-client-service)
+   (service tailon-service-type)))
+
+(define* (run-tailon-test #:optional (http-port 8042))
+  "Run tests in %TAILON-OS, which has tailon running and listening on
+HTTP-PORT."
+  (mlet* %store-monad ((os ->   (marionette-operating-system
+                                 %tailon-os
+                                 #:imported-modules '((gnu services herd)
+                                                      (guix combinators))))
+                       (command (system-qemu-image/shared-store-script
+                                 os #:graphic? #f)))
+    (define test
+      (with-imported-modules '((gnu build marionette))
+        #~(begin
+            (use-modules (srfi srfi-11) (srfi srfi-64)
+                         (gnu build marionette)
+                         (web uri)
+                         (web client)
+                         (web response))
+
+            (define marionette
+              ;; Forward the guest's HTTP-PORT, where tailon is listening, to
+              ;; port 8080 in the host.
+              (make-marionette (list #$command "-net"
+                                     (string-append
+                                      "user,hostfwd=tcp::8080-:"
+                                      #$(number->string http-port)))))
+
+            (mkdir #$output)
+            (chdir #$output)
+
+            (test-begin "tailon")
+
+            ;; Wait for tailon to be up and running.
+            (test-eq "service running"
+              'running!
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd))
+                  (start-service 'tailon)
+                  'running!)
+               marionette))
+
+            (test-equal "http-get"
+              200
+              (let-values (((response text)
+                            (http-get "http://localhost:8080/"
+                                      #:decode-body? #t)))
+                (response-code response)))
+
+            (test-end)
+            (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+    (gexp->derivation "tailon-test" test)))
+
+(define %test-tailon
+  (system-test
+   (name "tailon")
+   (description "Connect to a running Tailon server.")
+   (value (run-tailon-test))))
-- 
2.13.1

[bug#27650] [PATCH] tests: Add 'tailon-service-type' test.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> * gnu/tests/web.scm: New file.

Nice!  LGTM, thank you!

Ludo’.

[bug#27650] [PATCH 1/2] gnu: services: admin: Add tailon.

[Christopher Baines]

* gnu/services/admin.scm
  (<tailon-configuration>, <tailon-configuration-file>): New record types.
  (tailon-configuration-files-string, tailon-shepherd-service): New
  procedures.
  (%tailon-accounts, tailon-service-type: New variables.
* doc/guix.texi (Monitoring Services: Document the Tailon service.
---
 doc/guix.texi          |  90 +++++++++++++++++++++++++++++
 gnu/services/admin.scm | 151 ++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 240 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e8c4e0eaf..345285031 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -219,6 +219,7 @@ Services
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -9011,6 +9012,7 @@ declaration.
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -13599,6 +13601,94 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@node Monitoring Services
+@subsubsection Monitoring Services
+
+@subsubheading Tailon Service
+
+@uref{https://tailon.readthedocs.io/, Tailon} is a web application for
+viewing and searching log files.
+
+The following example will configure the service with default values.
+By default, Tailon can be accessed on port 8080 (@code{http://localhost:8080}).
+
+@example
+(service tailon-service-type)
+@end example
+
+The following example customises more of the Tailon configuration,
+adding @command{sed} to the list of allowed commands.
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file
+             (tailon-configuration-file
+               (allowed-commands '("tail" "grep" "awk" "sed"))))))
+@end example
+
+
+@deftp {Data Type} tailon-configuration
+Data type representing the configuration of Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{config-file} (default: @code{(tailon-configuration-file)})
+The configuration file to use for Tailon. This can be set to a
+@dfn{tailon-configuration-file} record value, or any gexp
+(@pxref{G-Expressions}).
+
+For example, to instead use a local file, the @code{local-file} function
+can be used:
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file (local-file "./my-tailon.conf"))))
+@end example
+
+@item @code{package} (default: @code{tailon})
+The tailon package to use.
+
+@end table
+@end deftp
+
+@deftp {Data Type} tailon-configuration-file
+Data type representing the configuration options for Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{files} (default: @code{(list "/var/log")})
+List of files to display. The list can include strings for a single file
+or directory, or a list, where the first item is the name of a
+subsection, and the remaining items are the files or directories in that
+subsection.
+
+@item @code{bind} (default: @code{"localhost:8080"})
+Address and port to which Tailon should bind on.
+
+@item @code{relative-root} (default: @code{#f})
+URL path to use for Tailon, set to @code{#f} to not use a path.
+
+@item @code{allow-transfers?} (default: @code{#t})
+Allow downloading the log files in the web interface.
+
+@item @code{follow-names?} (default: @code{#t})
+Allow tailing of not-yet existent files.
+
+@item @code{tail-lines} (default: @code{200})
+Number of lines to read initially from each file.
+
+@item @code{allowed-commands} (default: @code{(list "tail" "grep" "awk")})
+Commands to allow running. By default, @code{sed} is disabled.
+
+@item @code{debug?} (default: @code{#f})
+Set @code{debug?} to @code{#t} to show debug messages.
+
+@end table
+@end deftp
+
+
 @node Kerberos Services
 @subsubsection Kerberos Services
 @cindex Kerberos
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index b9e3fa70a..1044833fe 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -20,14 +20,19 @@
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages logging)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
   #:use-module (guix gexp)
+  #:use-module (guix store)
   #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 vlist)
+  #:use-module (ice-9 match)
   #:export (%default-rotations
             %rotated-files
 
@@ -41,7 +46,27 @@
             rottlog-configuration
             rottlog-configuration?
             rottlog-service
-            rottlog-service-type))
+            rottlog-service-type
+
+            <tailon-configuration-file>
+            tailon-configuration-file
+            tailon-configuration-file?
+            tailon-configuration-file-files
+            tailon-configuration-file-bind
+            tailon-configuration-file-relative-root
+            tailon-configuration-file-allow-transfers?
+            tailon-configuration-file-follow-names?
+            tailon-configuration-file-tail-lines
+            tailon-configuration-file-allowed-commands
+            tailon-configuration-file-debug?
+
+            <tailon-configuration>
+            tailon-configuration
+            tailon-configuration?
+            tailon-configuration-config-file
+            tailon-configuration-package
+
+            tailon-service-type))
 
 ;;; Commentary:
 ;;;
@@ -172,4 +197,128 @@ for ROTATION."
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+\f
+;;;
+;;; Tailon
+;;;
+
+(define-record-type* <tailon-configuration-file>
+  tailon-configuration-file make-tailon-configuration-file
+  tailon-configuration-file?
+  (files                   tailon-configuration-file-files
+                           (default '("/var/log")))
+  (bind                    tailon-configuration-file-bind
+                           (default "localhost:8080"))
+  (relative-root           tailon-configuration-file-relative-root
+                           (default #f))
+  (allow-transfers?        tailon-configuration-file-allow-transfers?
+                           (default #t))
+  (follow-names?           tailon-configuration-file-follow-names?
+                           (default #t))
+  (tail-lines              tailon-configuration-file-tail-lines
+                           (default 200))
+  (allowed-commands        tailon-configuration-file-allowed-commands
+                           (default '("tail" "grep" "awk")))
+  (debug?                  tailon-configuration-file-debug?
+                           (default #f)))
+
+(define (tailon-configuration-files-string files)
+  (string-append
+   "\n"
+   (string-join
+    (map
+     (lambda (x)
+       (string-append
+        "  - "
+        (cond
+         ((string? x)
+          (simple-format #f "'~A'" x))
+         ((list? x)
+          (string-join
+           (cons (simple-format #f "'~A':" (car x))
+                 (map
+                  (lambda (x) (simple-format #f "      - '~A'" x))
+                  (cdr x)))
+           "\n"))
+         (else (error x)))))
+     files)
+    "\n")))
+
+(define-gexp-compiler (tailon-configuration-file-compiler
+                       (file <tailon-configuration-file>) system target)
+  (match file
+    (($ <tailon-configuration-file> files bind relative-root
+                                    allow-transfers? follow-names?
+                                    tail-lines allowed-commands debug?)
+     (text-file
+      "tailon-config.yaml"
+      (string-concatenate
+       (filter-map
+        (match-lambda
+         ((key . #f) #f)
+         ((key . value) (string-append key ": " value "\n")))
+
+        `(("files" . ,(tailon-configuration-files-string files))
+          ("bind" . ,bind)
+          ("relative-root" . ,relative-root)
+          ("allow-transfers" . ,(if allow-transfers? "true" "false"))
+          ("follow-names" . ,(if follow-names? "true" "false"))
+          ("tail-lines" . ,(number->string tail-lines))
+          ("commands" . ,(string-append "["
+                                        (string-join allowed-commands ", ")
+                                        "]"))
+          ,@(if debug? '(("debug" . "true")) '()))))))))
+
+(define-record-type* <tailon-configuration>
+  tailon-configuration make-tailon-configuration
+  tailon-configuration?
+  (config-file tailon-configuration-config-file
+               (default (tailon-configuration-file)))
+  (package tailon-configuration-package
+           (default tailon)))
+
+(define tailon-shepherd-service
+  (match-lambda
+    (($ <tailon-configuration> config-file package)
+     (list (shepherd-service
+            (provision '(tailon))
+            (documentation "Run the tailon daemon.")
+            (start #~(make-forkexec-constructor
+                      `(,(string-append #$package "/bin/tailon")
+                        "-c" ,#$config-file)
+                      #:user "tailon"
+                      #:group "tailon"))
+            (stop #~(make-kill-destructor)))))))
+
+(define %tailon-accounts
+  (list (user-group (name "tailon") (system? #t))
+        (user-account
+         (name "tailon")
+         (group "tailon")
+         (system? #t)
+         (comment "tailon")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define tailon-service-type
+  (service-type
+   (name 'tailon)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             tailon-shepherd-service)
+          (service-extension account-service-type
+                             (const %tailon-accounts))))
+   (compose concatenate)
+   (extend (lambda (parameter files)
+             (tailon-configuration
+              (inherit parameter)
+              (config-file
+               (let ((old-config-file
+                      (tailon-configuration-config-file parameter)))
+                 (tailon-configuration-file
+                  (inherit old-config-file)
+                  (files (append (tailon-configuration-file-files old-config-file)
+                                 files))))))))
+   (default-value (tailon-configuration))))
+
 ;;; admin.scm ends here
-- 
2.13.3

[bug#27650] [PATCH 2/2] tests: Add 'tailon-service-type' test.

[Christopher Baines]

* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* gnu/tests/web.scm: New file.
---
 gnu/local.mk        |   1 +
 gnu/tests/admin.scm | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 107 insertions(+)
 create mode 100644 gnu/tests/admin.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 9dfca9dbb..90dfcfeb5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -479,6 +479,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/build/vm.scm				\
 						\
   %D%/tests.scm					\
+  %D%/tests/admin.scm				\
   %D%/tests/base.scm				\
   %D%/tests/dict.scm				\
   %D%/tests/nfs.scm				\
diff --git a/gnu/tests/admin.scm b/gnu/tests/admin.scm
new file mode 100644
index 000000000..06373be14
--- /dev/null
+++ b/gnu/tests/admin.scm
@@ -0,0 +1,106 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Christopher Baines <mail@cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests admin)
+  #:use-module (gnu tests)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system vm)
+  #:use-module (gnu services)
+  #:use-module (gnu services admin)
+  #:use-module (gnu services networking)
+  #:use-module (guix gexp)
+  #:use-module (guix store)
+  #:use-module (guix monads)
+  #:export (%test-tailon))
+
+(define %tailon-os
+  ;; Operating system under test.
+  (simple-operating-system
+   (dhcp-client-service)
+   (service tailon-service-type
+            (tailon-configuration
+             (config-file
+              (tailon-configuration-file
+               (bind "0.0.0.0:8080")))))))
+
+(define* (run-tailon-test #:optional (http-port 8081))
+  "Run tests in %TAILON-OS, which has tailon running and listening on
+HTTP-PORT."
+  (mlet* %store-monad ((os ->   (marionette-operating-system
+                                 %tailon-os
+                                 #:imported-modules '((gnu services herd)
+                                                      (guix combinators))))
+                       (command (system-qemu-image/shared-store-script
+                                 os #:graphic? #f)))
+    (define test
+      (with-imported-modules '((gnu build marionette))
+        #~(begin
+            (use-modules (srfi srfi-11) (srfi srfi-64)
+                         (gnu build marionette)
+                         (web uri)
+                         (web client)
+                         (web response))
+
+            (define marionette
+              ;; Forward the guest's HTTP-PORT, where tailon is listening, to
+              ;; port 8080 in the host.
+              (make-marionette (list #$command "-net"
+                                     #$(simple-format
+                                        #f
+                                        "user,hostfwd=tcp::~A-:8080"
+                                        http-port))))
+
+            (mkdir #$output)
+            (chdir #$output)
+
+            (test-begin "tailon")
+
+            (test-eq "service running"
+              'running!
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd))
+                  (start-service 'tailon)
+                  'running!)
+               marionette))
+
+            ;; Wait for tailon to be up and running.
+            (sleep 5)
+
+            (test-equal "http-get"
+              200
+              (let-values (((response text)
+                            (http-get #$(simple-format
+                                         #f
+                                         "http://localhost:~A/"
+                                         http-port)
+                                      #:decode-body? #t)))
+                (response-code response)))
+
+            (test-end)
+            (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+    (gexp->derivation "tailon-test" test)))
+
+(define %test-tailon
+  (system-test
+   (name "tailon")
+   (description "Connect to a running Tailon server.")
+   (value (run-tailon-test))))
-- 
2.13.3

[bug#27650] [PATCH 2/2] tests: Add 'tailon-service-type' test.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> * gnu/tests/web.scm: New file.

[...]

> +(define* (run-tailon-test #:optional (http-port 8081))
> +  "Run tests in %TAILON-OS, which has tailon running and listening on
> +HTTP-PORT."
> +  (mlet* %store-monad ((os ->   (marionette-operating-system
> +                                 %tailon-os
> +                                 #:imported-modules '((gnu services herd)
> +                                                      (guix combinators))))
> +                       (command (system-qemu-image/shared-store-script
> +                                 os #:graphic? #f)))


[...]

> +            (define marionette
> +              ;; Forward the guest's HTTP-PORT, where tailon is listening, to
> +              ;; port 8080 in the host.
> +              (make-marionette (list #$command "-net"
> +                                     #$(simple-format
> +                                        #f
> +                                        "user,hostfwd=tcp::~A-:8080"
> +                                        http-port))))

This can be simplified using the new ‘virtual-machine’ objects:

   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27751

Could you give it a try?

> +            (test-equal "http-get"
> +              200
> +              (let-values (((response text)
> +                            (http-get #$(simple-format
> +                                         #f
> +                                         "http://localhost:~A/"
> +                                         http-port)
> +                                      #:decode-body? #t)))
> +                (response-code response)))

If there was a more specific API, like a rest API to check whether a
specific log file exists, perhaps that’d be a good extra test.
Otherwise that’s fine.

BTW, it’s enough to use ‘format’ rather than ‘simple-format’ here: it’s
(roughly) equivalent and incurs less typing.  :-)

Feel free to squeeze this patch with the previous one (thus making a
self-contained commit), or leave it separately if it’s more convenient
for you.

Thanks for taking the time to write this test!

Ludo’.

[bug#27650] [PATCH 2/2] tests: Add 'tailon-service-type' test.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3083 bytes --]

On Wed, 26 Jul 2017 22:46:37 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail@cbaines.net> skribis:
> 
> > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> > * gnu/tests/web.scm: New file.  
> 
> [...]
> 
> > +(define* (run-tailon-test #:optional (http-port 8081))
> > +  "Run tests in %TAILON-OS, which has tailon running and listening
> > on +HTTP-PORT."
> > +  (mlet* %store-monad ((os ->   (marionette-operating-system
> > +                                 %tailon-os
> > +                                 #:imported-modules '((gnu
> > services herd)
> > +                                                      (guix
> > combinators))))
> > +                       (command
> > (system-qemu-image/shared-store-script
> > +                                 os #:graphic? #f)))  
> 
> 
> [...]
> 
> > +            (define marionette
> > +              ;; Forward the guest's HTTP-PORT, where tailon is
> > listening, to
> > +              ;; port 8080 in the host.
> > +              (make-marionette (list #$command "-net"
> > +                                     #$(simple-format
> > +                                        #f
> > +
> > "user,hostfwd=tcp::~A-:8080"
> > +                                        http-port))))  
> 
> This can be simplified using the new ‘virtual-machine’ objects:
> 
>    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27751
> 
> Could you give it a try?

Yep, I'll send an updated patch that switches to use this approach.

> > +            (test-equal "http-get"
> > +              200
> > +              (let-values (((response text)
> > +                            (http-get #$(simple-format
> > +                                         #f
> > +                                         "http://localhost:~A/"
> > +                                         http-port)
> > +                                      #:decode-body? #t)))
> > +                (response-code response)))  
> 
> If there was a more specific API, like a rest API to check whether a
> specific log file exists, perhaps that’d be a good extra test.
> Otherwise that’s fine.

I'm not sure there is an API, but I'll have a think about more useful
testing approaches. Maybe checking for things in the response, or
trying to run Tailon's test suite...

> BTW, it’s enough to use ‘format’ rather than ‘simple-format’ here:
> it’s (roughly) equivalent and incurs less typing.  :-)

Ok, I'll change to use format.

> Feel free to squeeze this patch with the previous one (thus making a
> self-contained commit), or leave it separately if it’s more convenient
> for you.

Squashing them is fine, I'll do this.

> Thanks for taking the time to write this test!

No problem, thanks for your and everyone elses advice on QEMU on IRC,
it was very useful.

Also, I've removed the sleep, and replaced it with a function to retry
the connection on error, which should hopefully avoid any issues with
it failing just because Tailon hasn't finished starting yet.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

* gnu/services/admin.scm
  (<tailon-configuration>, <tailon-configuration-file>): New record types.
  (tailon-configuration-files-string, tailon-shepherd-service): New
  procedures.
  (%tailon-accounts, tailon-service-type: New variables.
* doc/guix.texi (Monitoring Services: Document the Tailon service.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* gnu/tests/admin.scm: New file.
---
 doc/guix.texi          |  90 +++++++++++++++++++++++++++++
 gnu/local.mk           |   1 +
 gnu/services/admin.scm | 151 ++++++++++++++++++++++++++++++++++++++++++++++++-
 gnu/tests/admin.scm    | 128 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 369 insertions(+), 1 deletion(-)
 create mode 100644 gnu/tests/admin.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index e8c4e0eaf..345285031 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -219,6 +219,7 @@ Services
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -9011,6 +9012,7 @@ declaration.
 * Database Services::           SQL databases.
 * Mail Services::               IMAP, POP3, SMTP, and all that.
 * Messaging Services::          Messaging services.
+* Monitoring Services::         Monitoring services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
 * DNS Services::                DNS daemons.
@@ -13599,6 +13601,94 @@ string, you could instantiate a prosody service like this:
           (prosody.cfg.lua "")))
 @end example
 
+@node Monitoring Services
+@subsubsection Monitoring Services
+
+@subsubheading Tailon Service
+
+@uref{https://tailon.readthedocs.io/, Tailon} is a web application for
+viewing and searching log files.
+
+The following example will configure the service with default values.
+By default, Tailon can be accessed on port 8080 (@code{http://localhost:8080}).
+
+@example
+(service tailon-service-type)
+@end example
+
+The following example customises more of the Tailon configuration,
+adding @command{sed} to the list of allowed commands.
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file
+             (tailon-configuration-file
+               (allowed-commands '("tail" "grep" "awk" "sed"))))))
+@end example
+
+
+@deftp {Data Type} tailon-configuration
+Data type representing the configuration of Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{config-file} (default: @code{(tailon-configuration-file)})
+The configuration file to use for Tailon. This can be set to a
+@dfn{tailon-configuration-file} record value, or any gexp
+(@pxref{G-Expressions}).
+
+For example, to instead use a local file, the @code{local-file} function
+can be used:
+
+@example
+(service tailon-service-type
+         (tailon-configuration
+           (config-file (local-file "./my-tailon.conf"))))
+@end example
+
+@item @code{package} (default: @code{tailon})
+The tailon package to use.
+
+@end table
+@end deftp
+
+@deftp {Data Type} tailon-configuration-file
+Data type representing the configuration options for Tailon.
+This type has the following parameters:
+
+@table @asis
+@item @code{files} (default: @code{(list "/var/log")})
+List of files to display. The list can include strings for a single file
+or directory, or a list, where the first item is the name of a
+subsection, and the remaining items are the files or directories in that
+subsection.
+
+@item @code{bind} (default: @code{"localhost:8080"})
+Address and port to which Tailon should bind on.
+
+@item @code{relative-root} (default: @code{#f})
+URL path to use for Tailon, set to @code{#f} to not use a path.
+
+@item @code{allow-transfers?} (default: @code{#t})
+Allow downloading the log files in the web interface.
+
+@item @code{follow-names?} (default: @code{#t})
+Allow tailing of not-yet existent files.
+
+@item @code{tail-lines} (default: @code{200})
+Number of lines to read initially from each file.
+
+@item @code{allowed-commands} (default: @code{(list "tail" "grep" "awk")})
+Commands to allow running. By default, @code{sed} is disabled.
+
+@item @code{debug?} (default: @code{#f})
+Set @code{debug?} to @code{#t} to show debug messages.
+
+@end table
+@end deftp
+
+
 @node Kerberos Services
 @subsubsection Kerberos Services
 @cindex Kerberos
diff --git a/gnu/local.mk b/gnu/local.mk
index 9dfca9dbb..90dfcfeb5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -479,6 +479,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/build/vm.scm				\
 						\
   %D%/tests.scm					\
+  %D%/tests/admin.scm				\
   %D%/tests/base.scm				\
   %D%/tests/dict.scm				\
   %D%/tests/nfs.scm				\
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index b9e3fa70a..1044833fe 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -20,14 +20,19 @@
 (define-module (gnu services admin)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages logging)
   #:use-module (gnu services)
   #:use-module (gnu services mcron)
   #:use-module (gnu services shepherd)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
   #:use-module (guix gexp)
+  #:use-module (guix store)
   #:use-module (guix packages)
   #:use-module (guix records)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 vlist)
+  #:use-module (ice-9 match)
   #:export (%default-rotations
             %rotated-files
 
@@ -41,7 +46,27 @@
             rottlog-configuration
             rottlog-configuration?
             rottlog-service
-            rottlog-service-type))
+            rottlog-service-type
+
+            <tailon-configuration-file>
+            tailon-configuration-file
+            tailon-configuration-file?
+            tailon-configuration-file-files
+            tailon-configuration-file-bind
+            tailon-configuration-file-relative-root
+            tailon-configuration-file-allow-transfers?
+            tailon-configuration-file-follow-names?
+            tailon-configuration-file-tail-lines
+            tailon-configuration-file-allowed-commands
+            tailon-configuration-file-debug?
+
+            <tailon-configuration>
+            tailon-configuration
+            tailon-configuration?
+            tailon-configuration-config-file
+            tailon-configuration-package
+
+            tailon-service-type))
 
 ;;; Commentary:
 ;;;
@@ -172,4 +197,128 @@ for ROTATION."
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+\f
+;;;
+;;; Tailon
+;;;
+
+(define-record-type* <tailon-configuration-file>
+  tailon-configuration-file make-tailon-configuration-file
+  tailon-configuration-file?
+  (files                   tailon-configuration-file-files
+                           (default '("/var/log")))
+  (bind                    tailon-configuration-file-bind
+                           (default "localhost:8080"))
+  (relative-root           tailon-configuration-file-relative-root
+                           (default #f))
+  (allow-transfers?        tailon-configuration-file-allow-transfers?
+                           (default #t))
+  (follow-names?           tailon-configuration-file-follow-names?
+                           (default #t))
+  (tail-lines              tailon-configuration-file-tail-lines
+                           (default 200))
+  (allowed-commands        tailon-configuration-file-allowed-commands
+                           (default '("tail" "grep" "awk")))
+  (debug?                  tailon-configuration-file-debug?
+                           (default #f)))
+
+(define (tailon-configuration-files-string files)
+  (string-append
+   "\n"
+   (string-join
+    (map
+     (lambda (x)
+       (string-append
+        "  - "
+        (cond
+         ((string? x)
+          (simple-format #f "'~A'" x))
+         ((list? x)
+          (string-join
+           (cons (simple-format #f "'~A':" (car x))
+                 (map
+                  (lambda (x) (simple-format #f "      - '~A'" x))
+                  (cdr x)))
+           "\n"))
+         (else (error x)))))
+     files)
+    "\n")))
+
+(define-gexp-compiler (tailon-configuration-file-compiler
+                       (file <tailon-configuration-file>) system target)
+  (match file
+    (($ <tailon-configuration-file> files bind relative-root
+                                    allow-transfers? follow-names?
+                                    tail-lines allowed-commands debug?)
+     (text-file
+      "tailon-config.yaml"
+      (string-concatenate
+       (filter-map
+        (match-lambda
+         ((key . #f) #f)
+         ((key . value) (string-append key ": " value "\n")))
+
+        `(("files" . ,(tailon-configuration-files-string files))
+          ("bind" . ,bind)
+          ("relative-root" . ,relative-root)
+          ("allow-transfers" . ,(if allow-transfers? "true" "false"))
+          ("follow-names" . ,(if follow-names? "true" "false"))
+          ("tail-lines" . ,(number->string tail-lines))
+          ("commands" . ,(string-append "["
+                                        (string-join allowed-commands ", ")
+                                        "]"))
+          ,@(if debug? '(("debug" . "true")) '()))))))))
+
+(define-record-type* <tailon-configuration>
+  tailon-configuration make-tailon-configuration
+  tailon-configuration?
+  (config-file tailon-configuration-config-file
+               (default (tailon-configuration-file)))
+  (package tailon-configuration-package
+           (default tailon)))
+
+(define tailon-shepherd-service
+  (match-lambda
+    (($ <tailon-configuration> config-file package)
+     (list (shepherd-service
+            (provision '(tailon))
+            (documentation "Run the tailon daemon.")
+            (start #~(make-forkexec-constructor
+                      `(,(string-append #$package "/bin/tailon")
+                        "-c" ,#$config-file)
+                      #:user "tailon"
+                      #:group "tailon"))
+            (stop #~(make-kill-destructor)))))))
+
+(define %tailon-accounts
+  (list (user-group (name "tailon") (system? #t))
+        (user-account
+         (name "tailon")
+         (group "tailon")
+         (system? #t)
+         (comment "tailon")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define tailon-service-type
+  (service-type
+   (name 'tailon)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             tailon-shepherd-service)
+          (service-extension account-service-type
+                             (const %tailon-accounts))))
+   (compose concatenate)
+   (extend (lambda (parameter files)
+             (tailon-configuration
+              (inherit parameter)
+              (config-file
+               (let ((old-config-file
+                      (tailon-configuration-config-file parameter)))
+                 (tailon-configuration-file
+                  (inherit old-config-file)
+                  (files (append (tailon-configuration-file-files old-config-file)
+                                 files))))))))
+   (default-value (tailon-configuration))))
+
 ;;; admin.scm ends here
diff --git a/gnu/tests/admin.scm b/gnu/tests/admin.scm
new file mode 100644
index 000000000..ef3ddf079
--- /dev/null
+++ b/gnu/tests/admin.scm
@@ -0,0 +1,128 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Christopher Baines <mail@cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests admin)
+  #:use-module (gnu tests)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system vm)
+  #:use-module (gnu services)
+  #:use-module (gnu services admin)
+  #:use-module (gnu services networking)
+  #:use-module (guix gexp)
+  #:use-module (guix store)
+  #:use-module (guix monads)
+  #:export (%test-tailon))
+
+(define %tailon-os
+  ;; Operating system under test.
+  (simple-operating-system
+   (dhcp-client-service)
+   (service tailon-service-type
+            (tailon-configuration
+             (config-file
+              (tailon-configuration-file
+               (bind "0.0.0.0:8080")))))))
+
+(define* (run-tailon-test #:optional (http-port 8081))
+  "Run tests in %TAILON-OS, which has tailon running and listening on
+HTTP-PORT."
+  (define os
+    (marionette-operating-system
+     %tailon-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (port-forwardings `((,http-port . 8080)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (ice-9 match)
+                       (gnu build marionette)
+                       (web uri)
+                       (web client)
+                       (web response))
+
+          (define marionette
+            ;; Forward the guest's HTTP-PORT, where tailon is listening, to
+            ;; port 8080 in the host.
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "tailon")
+
+          (test-eq "service running"
+            'running!
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'tailon)
+                'running!)
+             marionette))
+
+          (define* (retry-on-error f #:key times delay)
+            (let loop ((attempt 1))
+              (match (catch
+                      #t
+                      (lambda ()
+                        (cons #t
+                              (f)))
+                      (lambda args
+                        (cons #f
+                              args)))
+                ((#t . return-value)
+                 return-value)
+                ((#f . error-args)
+                 (if (>= attempt times)
+                     error-args
+                     (begin
+                       (sleep delay)
+                       (loop (+ 1 attempt))))))))
+
+          (test-equal "http-get"
+            200
+            (retry-on-error
+             (lambda ()
+               (let-values (((response text)
+                             (http-get #$(format
+                                          #f
+                                          "http://localhost:~A/"
+                                          http-port)
+                                       #:decode-body? #t)))
+                 (response-code response)))
+             #:times 5
+             #:delay 2))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "tailon-test" test))
+
+(define %test-tailon
+  (system-test
+   (name "tailon")
+   (description "Connect to a running Tailon server.")
+   (value (run-tailon-test))))
-- 
2.13.3

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/admin.scm
>   (<tailon-configuration>, <tailon-configuration-file>): New record types.
>   (tailon-configuration-files-string, tailon-shepherd-service): New
>   procedures.
>   (%tailon-accounts, tailon-service-type: New variables.
> * doc/guix.texi (Monitoring Services: Document the Tailon service.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> * gnu/tests/admin.scm: New file.

You also need to add gnu/tests/admin.scm to local.mk, but apart from
that it looks perfect to me.  Thank you!

Ludo’.

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On Fri, 28 Jul 2017 21:36:10 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail@cbaines.net> skribis:
> 
> > * gnu/services/admin.scm
> >   (<tailon-configuration>, <tailon-configuration-file>): New record
> > types. (tailon-configuration-files-string,
> > tailon-shepherd-service): New procedures.
> >   (%tailon-accounts, tailon-service-type: New variables.
> > * doc/guix.texi (Monitoring Services: Document the Tailon service.
> > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> > * gnu/tests/admin.scm: New file.  
> 
> You also need to add gnu/tests/admin.scm to local.mk, but apart from
> that it looks perfect to me.  Thank you!

The latest patch I send should have included updating local.mk, at
least the quoted changelog claims to change the file. I'll clarify what
exactly was added, and look to push this, but do let me know if there
is something that still needs doing.

Thanks for your help and review,

Chris

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

bug#27650: [PATCH] gnu: services: admin: Add tailon.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On Fri, 28 Jul 2017 22:00:22 +0100
Christopher Baines <mail@cbaines.net> wrote:

> On Fri, 28 Jul 2017 21:36:10 +0200
> ludo@gnu.org (Ludovic Courtès) wrote:
> 
> > Christopher Baines <mail@cbaines.net> skribis:
> >   
> > > * gnu/services/admin.scm
> > >   (<tailon-configuration>, <tailon-configuration-file>): New
> > > record types. (tailon-configuration-files-string,
> > > tailon-shepherd-service): New procedures.
> > >   (%tailon-accounts, tailon-service-type: New variables.
> > > * doc/guix.texi (Monitoring Services: Document the Tailon service.
> > > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> > > * gnu/tests/admin.scm: New file.    
> > 
> > You also need to add gnu/tests/admin.scm to local.mk, but apart from
> > that it looks perfect to me.  Thank you!  
> 
> The latest patch I send should have included updating local.mk, at
> least the quoted changelog claims to change the file. I'll clarify
> what exactly was added, and look to push this, but do let me know if
> there is something that still needs doing.

Now pushed!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

[bug#27650] [PATCH] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> On Fri, 28 Jul 2017 21:36:10 +0200
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Christopher Baines <mail@cbaines.net> skribis:
>> 
>> > * gnu/services/admin.scm
>> >   (<tailon-configuration>, <tailon-configuration-file>): New record
>> > types. (tailon-configuration-files-string,
>> > tailon-shepherd-service): New procedures.
>> >   (%tailon-accounts, tailon-service-type: New variables.
>> > * doc/guix.texi (Monitoring Services: Document the Tailon service.
>> > * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
>> > * gnu/tests/admin.scm: New file.  
>> 
>> You also need to add gnu/tests/admin.scm to local.mk, but apart from
>> that it looks perfect to me.  Thank you!
>
> The latest patch I send should have included updating local.mk, at
> least the quoted changelog claims to change the file. I'll clarify what
> exactly was added, and look to push this, but do let me know if there
> is something that still needs doing.

I guess we’re done now, thanks!

Ludo’.

[bug#27650] [PATCH 1/2] gnu: services: admin: Add tailon.

[Ludovic Courtès]

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/admin.scm
>   (<tailon-configuration>, <tailon-configuration-file>): New record types.
>   (tailon-configuration-files-string, tailon-shepherd-service): New
>   procedures.
>   (%tailon-accounts, tailon-service-type: New variables.
> * doc/guix.texi (Monitoring Services: Document the Tailon service.

LGTM!