[PATCH] services: nginx: Allow for server extensions.

[PATCH] services: nginx: Allow for server extensions.

[David Thompson]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]

Hey folks,

Looking for some feedback on my first stab at making the nginx service
extensible.  With this extension mechanism, future web applications
(such as GNU MediaGoblin) that use nginx as a front-end web server will
be able to extend nginx with the server configuration that they need in
order to work.

Here's a useless service that adds nginx configuration to serve the
contents of /tmp:

    (define server
      (plain-file "foo.conf"
                  "
    server {
      listen 80;
      root /tmp;
      index index.html;
      server_name dthompson.us;
    }
    "))

    (define foo-service-type
      (service-type (name 'foo)
                    (extensions
                     (list (service-extension nginx-service-type
                                              (const (list server)))))))

    (define foo-service
      (service foo-service-type #f))

One big question I have is whether I should enforce that configuration
be in file-like objects or if I should allow strings, too.  Thoughts?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-services-nginx-Allow-for-server-extensions.patch --]
[-- Type: text/x-patch, Size: 7190 bytes --]

From 108db2d183526c42b53060e55f7fb292b53663cb Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Mon, 30 Nov 2015 08:49:08 -0500
Subject: [PATCH] services: nginx: Allow for server extensions.

* gnu/services/web.scm (<nginx-configuration>)[servers]: New field.
  (nginx-configuration-servers): New accessor.
  (default-nginx-config): Delete.
  (nginx-configuration-file*): New procedure.
  (nginx-activation): Perform the syntax check on the full computed
  configuration file.
  (nginx-dmd-service): Use the full computed configuration file when
  starting the service.
  (extend-nginx): New procedure.
  (nginx-service-type): Specify extension procedures.
  (nginx-service): Add #:servers argument.
---
 gnu/services/web.scm | 97 ++++++++++++++++++++++++++++++++--------------------
 1 file changed, 60 insertions(+), 37 deletions(-)

diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 84bb30d..a5bc364 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -26,7 +26,16 @@
   #:use-module (guix records)
   #:use-module (guix gexp)
   #:use-module (ice-9 match)
-  #:export (nginx-service))
+  #:use-module (srfi srfi-1)
+  #:export (nginx-configuration
+            nginx-configuration?
+            nginx-configuration-log-directory
+            nginx-configuration-run-directory
+            nginx-configuration-file
+            nginx-configuration-servers
+
+            nginx-service-type
+            nginx-service))
 
 ;;; Commentary:
 ;;;
@@ -37,23 +46,26 @@
 (define-record-type* <nginx-configuration>
   nginx-configuration make-nginx-configuration
   nginx-configuration?
-  (nginx         nginx-configuration-nginx)         ;<package>
-  (log-directory nginx-configuration-log-directory) ;string
-  (run-directory nginx-configuration-run-directory) ;string
-  (file          nginx-configuration-file))         ;string | file-like
+  (nginx         nginx-configuration-nginx)         ; <package>
+  (log-directory nginx-configuration-log-directory) ; string
+  (run-directory nginx-configuration-run-directory) ; string
+  (file          nginx-configuration-file)          ; file-like
+  (servers       nginx-configuration-servers))      ; list of file-like
 
-(define (default-nginx-config log-directory run-directory)
-  (plain-file "nginx.conf"
-              (string-append
-               "user nginx nginx;\n"
-               "pid " run-directory "/pid;\n"
-               "error_log " log-directory "/error.log info;\n"
-               "http {\n"
-               "    access_log " log-directory "/access.log;\n"
-               "    root /var/www;\n"
-               "    server {}\n"
-               "}\n"
-               "events {}\n")))
+(define (nginx-configuration-file* config)
+  (match config
+    (($ <nginx-configuration> _ log run file servers)
+     (apply mixed-text-file "nginx.conf"
+            `("user nginx nginx;\n"
+              "pid " ,run "/pid;\n"
+              "error_log " ,log "/error.log info;\n"
+              "include " ,file ";\n"
+              "http {\n"
+              "  access_log " ,log "/access.log;\n"
+              ,@(append-map (lambda (server-config)
+                              (list "include " server-config ";\n"))
+                            servers)
+              "}\n")))))
 
 (define %nginx-accounts
   (list (user-group (name "nginx") (system? #t))
@@ -66,37 +78,43 @@
          (shell #~(string-append #$shadow "/sbin/nologin")))))
 
 (define nginx-activation
-  (match-lambda
-    (($ <nginx-configuration> nginx log-directory run-directory config-file)
-     #~(begin
-         (use-modules (guix build utils))
+  (lambda (config)
+    (match config
+      (($ <nginx-configuration> nginx log-directory run-directory _)
+       #~(begin
+           (use-modules (guix build utils))
 
-         (format #t "creating nginx log directory '~a'~%" #$log-directory)
-         (mkdir-p #$log-directory)
-         (format #t "creating nginx run directory '~a'~%" #$run-directory)
-         (mkdir-p #$run-directory)
-         ;; Check configuration file syntax.
-         (system* (string-append #$nginx "/bin/nginx")
-                  "-c" #$config-file "-t")))))
+           (format #t "creating nginx log directory '~a'~%" #$log-directory)
+           (mkdir-p #$log-directory)
+           (format #t "creating nginx run directory '~a'~%" #$run-directory)
+           (mkdir-p #$run-directory)
+           ;; Check configuration file syntax.
+           (system* (string-append #$nginx "/sbin/nginx")
+                    "-c" #$(nginx-configuration-file* config) "-t"))))))
 
-(define nginx-dmd-service
-  (match-lambda
-    (($ <nginx-configuration> nginx log-directory run-directory config-file)
+(define (nginx-dmd-service config)
+  (match config
+    (($ <nginx-configuration> nginx log run file servers)
      (let* ((nginx-binary #~(string-append #$nginx "/sbin/nginx"))
+            (config-file (nginx-configuration-file* config))
             (nginx-action
              (lambda args
                #~(lambda _
                    (zero?
                     (system* #$nginx-binary "-c" #$config-file #$@args))))))
 
-       ;; TODO: Add 'reload' action.
        (list (dmd-service
               (provision '(nginx))
               (documentation "Run the nginx daemon.")
               (requirement '(user-processes loopback))
-              (start (nginx-action "-p" run-directory))
+              (start (nginx-action "-p" run))
               (stop (nginx-action "-s" "stop"))))))))
 
+(define (extend-nginx config servers)
+  (nginx-configuration
+   (inherit config)
+   (servers (append (nginx-configuration-servers config) servers))))
+
 (define nginx-service-type
   (service-type (name 'nginx)
                 (extensions
@@ -105,13 +123,17 @@
                        (service-extension activation-service-type
                                           nginx-activation)
                        (service-extension account-service-type
-                                          (const %nginx-accounts))))))
+                                          (const %nginx-accounts))))
+                (compose concatenate)
+                (extend extend-nginx)))
 
 (define* (nginx-service #:key (nginx nginx)
                         (log-directory "/var/log/nginx")
                         (run-directory "/var/run/nginx")
-                        (config-file
-                         (default-nginx-config log-directory run-directory)))
+                        ;; Nginx requires an 'events' block.
+                        (config-file (plain-file "nginx-main.conf"
+                                                 "events {}"))
+                        (servers '()))
   "Return a service that runs NGINX, the nginx web server.
 
 The nginx daemon loads its runtime configuration from CONFIG-FIGLE, stores log
@@ -121,4 +143,5 @@ files in LOG-DIRECTORY, and stores temporary runtime files in RUN-DIRECTORY."
             (nginx nginx)
             (log-directory log-directory)
             (run-directory run-directory)
-            (file config-file))))
+            (file config-file)
+            (servers servers))))
-- 
2.5.0


[-- Attachment #3: Type: text/plain, Size: 37 bytes --]


--
David Thompson
GPG Key: 0FF1D807

Re: [PATCH] services: nginx: Allow for server extensions.

[Ludovic Courtès]

David Thompson <dthompson2@worcester.edu> skribis:

> Looking for some feedback on my first stab at making the nginx service
> extensible.  With this extension mechanism, future web applications
> (such as GNU MediaGoblin) that use nginx as a front-end web server will
> be able to extend nginx with the server configuration that they need in
> order to work.

Excellent!

> Here's a useless service that adds nginx configuration to serve the
> contents of /tmp:
>
>     (define server
>       (plain-file "foo.conf"
>                   "
>     server {
>       listen 80;
>       root /tmp;
>       index index.html;
>       server_name dthompson.us;
>     }
>     "))

Do you think it would make sense to provide Scheme bindings for those
‘server’ configuration snippets, or would we lose too much
expressiveness?

> From 108db2d183526c42b53060e55f7fb292b53663cb Mon Sep 17 00:00:00 2001
> From: David Thompson <dthompson2@worcester.edu>
> Date: Mon, 30 Nov 2015 08:49:08 -0500
> Subject: [PATCH] services: nginx: Allow for server extensions.
>
> * gnu/services/web.scm (<nginx-configuration>)[servers]: New field.
>   (nginx-configuration-servers): New accessor.
>   (default-nginx-config): Delete.
>   (nginx-configuration-file*): New procedure.
>   (nginx-activation): Perform the syntax check on the full computed
>   configuration file.
>   (nginx-dmd-service): Use the full computed configuration file when
>   starting the service.
>   (extend-nginx): New procedure.
>   (nginx-service-type): Specify extension procedures.
>   (nginx-service): Add #:servers argument.

[...]

> +(define (nginx-configuration-file* config)

‘nginx-configuration->file’ maybe?

Otherwise LGTM!

As an exercise, and while waiting for Chris to finish packaging
MediaGoblin ;-), it might make sense to try to use nginx in
‘guix-publish-service’ or a variant thereof.

Thank you!

Ludo’.

Re: [PATCH] services: nginx: Allow for server extensions.

[Leo Famulari]

On Wed, Dec 02, 2015 at 10:07:23AM +0200, Ludovic Courtès wrote:
> David Thompson <dthompson2@worcester.edu> skribis:
> 
> > Looking for some feedback on my first stab at making the nginx service
> > extensible.  With this extension mechanism, future web applications
> > (such as GNU MediaGoblin) that use nginx as a front-end web server will
> > be able to extend nginx with the server configuration that they need in
> > order to work.
> 
> Excellent!
> 
> > Here's a useless service that adds nginx configuration to serve the
> > contents of /tmp:
> >
> >     (define server
> >       (plain-file "foo.conf"
> >                   "
> >     server {
> >       listen 80;
> >       root /tmp;
> >       index index.html;
> >       server_name dthompson.us;
> >     }
> >     "))
> 
> Do you think it would make sense to provide Scheme bindings for those
> ‘server’ configuration snippets, or would we lose too much
> expressiveness?

I think you might lose too much, although I don't use GuixSD so I
don't know if there are other programs that are configured through
Scheme bindings. I know that I don't like the Nix bindings for systemd
in NixOS. They can't keep up with the pace of development in systemd and
provide a limited set of features.

> 
> > From 108db2d183526c42b53060e55f7fb292b53663cb Mon Sep 17 00:00:00 2001
> > From: David Thompson <dthompson2@worcester.edu>
> > Date: Mon, 30 Nov 2015 08:49:08 -0500
> > Subject: [PATCH] services: nginx: Allow for server extensions.
> >
> > * gnu/services/web.scm (<nginx-configuration>)[servers]: New field.
> >   (nginx-configuration-servers): New accessor.
> >   (default-nginx-config): Delete.
> >   (nginx-configuration-file*): New procedure.
> >   (nginx-activation): Perform the syntax check on the full computed
> >   configuration file.
> >   (nginx-dmd-service): Use the full computed configuration file when
> >   starting the service.
> >   (extend-nginx): New procedure.
> >   (nginx-service-type): Specify extension procedures.
> >   (nginx-service): Add #:servers argument.
> 
> [...]
> 
> > +(define (nginx-configuration-file* config)
> 
> ‘nginx-configuration->file’ maybe?
> 
> Otherwise LGTM!
> 
> As an exercise, and while waiting for Chris to finish packaging
> MediaGoblin ;-), it might make sense to try to use nginx in
> ‘guix-publish-service’ or a variant thereof.
> 
> Thank you!
> 
> Ludo’.
> 

Re: [PATCH] services: nginx: Allow for server extensions.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Wed, Dec 02, 2015 at 10:07:23AM +0200, Ludovic Courtès wrote:
>> David Thompson <dthompson2@worcester.edu> skribis:
>> 
>> > Looking for some feedback on my first stab at making the nginx service
>> > extensible.  With this extension mechanism, future web applications
>> > (such as GNU MediaGoblin) that use nginx as a front-end web server will
>> > be able to extend nginx with the server configuration that they need in
>> > order to work.
>> 
>> Excellent!
>> 
>> > Here's a useless service that adds nginx configuration to serve the
>> > contents of /tmp:
>> >
>> >     (define server
>> >       (plain-file "foo.conf"
>> >                   "
>> >     server {
>> >       listen 80;
>> >       root /tmp;
>> >       index index.html;
>> >       server_name dthompson.us;
>> >     }
>> >     "))
>> 
>> Do you think it would make sense to provide Scheme bindings for those
>> ‘server’ configuration snippets, or would we lose too much
>> expressiveness?
>
> I think you might lose too much, although I don't use GuixSD so I
> don't know if there are other programs that are configured through
> Scheme bindings. I know that I don't like the Nix bindings for systemd
> in NixOS. They can't keep up with the pace of development in systemd and
> provide a limited set of features.

Right, that’s exactly why I was asking: in some cases it’s OK to write
bindings (they’re more convenient and just as expressive), but in other
cases they’d be a loss, as you explain.

Thanks for your feedback!

Ludo’.

Re: [PATCH] services: nginx: Allow for server extensions.

[Ludovic Courtès]

Hi!

Ping!  :-)

Ludo’.

ludo@gnu.org (Ludovic Courtès) skribis:

> David Thompson <dthompson2@worcester.edu> skribis:
>
>> Looking for some feedback on my first stab at making the nginx service
>> extensible.  With this extension mechanism, future web applications
>> (such as GNU MediaGoblin) that use nginx as a front-end web server will
>> be able to extend nginx with the server configuration that they need in
>> order to work.
>
> Excellent!
>
>> Here's a useless service that adds nginx configuration to serve the
>> contents of /tmp:
>>
>>     (define server
>>       (plain-file "foo.conf"
>>                   "
>>     server {
>>       listen 80;
>>       root /tmp;
>>       index index.html;
>>       server_name dthompson.us;
>>     }
>>     "))
>
> Do you think it would make sense to provide Scheme bindings for those
> ‘server’ configuration snippets, or would we lose too much
> expressiveness?
>
>> From 108db2d183526c42b53060e55f7fb292b53663cb Mon Sep 17 00:00:00 2001
>> From: David Thompson <dthompson2@worcester.edu>
>> Date: Mon, 30 Nov 2015 08:49:08 -0500
>> Subject: [PATCH] services: nginx: Allow for server extensions.
>>
>> * gnu/services/web.scm (<nginx-configuration>)[servers]: New field.
>>   (nginx-configuration-servers): New accessor.
>>   (default-nginx-config): Delete.
>>   (nginx-configuration-file*): New procedure.
>>   (nginx-activation): Perform the syntax check on the full computed
>>   configuration file.
>>   (nginx-dmd-service): Use the full computed configuration file when
>>   starting the service.
>>   (extend-nginx): New procedure.
>>   (nginx-service-type): Specify extension procedures.
>>   (nginx-service): Add #:servers argument.
>
> [...]
>
>> +(define (nginx-configuration-file* config)
>
> ‘nginx-configuration->file’ maybe?
>
> Otherwise LGTM!
>
> As an exercise, and while waiting for Chris to finish packaging
> MediaGoblin ;-), it might make sense to try to use nginx in
> ‘guix-publish-service’ or a variant thereof.
>
> Thank you!
>
> Ludo’.