[bug#61740] [PATCH] services: Add rspamd-service-type.

[Saku Laesvuori via Guix-patches via <guix-patches@gnu.org>]

[-- Attachment #1.1: Type: text/plain, Size: 637 bytes --]

On Fri, Sep 15, 2023 at 01:47:14PM -0700, Felix Lechner wrote:
> Hi Saku,
> 
> > I happened to need rspamd myself
> 
> So do I but it does not seem to start locally. It created some folders
> and, per the log, the configuration file passed the syntax check, but
> then the boot stalls.
> 
> I used (service rspamd-service-type) and nothing else in my system
> configuration. Should it be sufficient? Thanks!

Oops, the version I tested (and am running right now) used
activation-service-type and I forgot to test that it was still working
after moving the code from there to the shepherd service. A fixed v3 is
attached.

[-- Attachment #1.2: v3-0001-services-Add-rspamd-service-type.patch --]
[-- Type: text/plain, Size: 15745 bytes --]

From 886c32091bdc032c4180d490125a131df79b705c Mon Sep 17 00:00:00 2001
Message-ID: <886c32091bdc032c4180d490125a131df79b705c.1694894798.git.saku@laesvuori.fi>
From: Thomas Ieong <th.ieong@free.fr>
Date: Thu, 23 Feb 2023 21:16:14 +0100
Subject: [PATCH v3] services: Add rspamd-service-type.

* gnu/services/mail.scm (rspamd-service-type): New variable.
* gnu/tests/mail.scm (%test-rspamd): New variable.
* doc/guix.texi: Document it.

Co-authored-by: Saku Laesvuori <saku@laesvuori.fi>
---
This version imports (gnu build activation) in the shepherd start action
and removes the syntax check, because I expect the service to fail
anyway if the configuration file syntax is invalid and the check seemed
to prevent booting (no idea why).

 doc/guix.texi         |  61 +++++++++++++
 gnu/services/mail.scm | 197 +++++++++++++++++++++++++++++++++++++++++-
 gnu/tests/mail.scm    |  87 ++++++++++++++++++-
 3 files changed, 343 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index f82bb99069..04e4a60f97 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -119,6 +119,8 @@
 Copyright @copyright{} 2023 Zheng Junjie@*
 Copyright @copyright{} 2023 Brian Cully@*
 Copyright @copyright{} 2023 Felix Lechner@*
+Copyright @copyright{} 2023 Thomas Ieong@*
+Copyright @copyright{} 2023 Saku Laesvuori@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -27393,6 +27395,65 @@ Mail Services
 @end table
 @end deftp
 
+@subsubheading Rspamd Service
+@cindex email
+@cindex spam
+
+@defvar rspamd-service-type
+This is the type of the @uref{https://rspamd.com/, Rspamd} filtering
+system whose value should be a @code{rspamd-configuration}.
+@end defvar
+
+@c %start of fragment
+
+@deftp {Data Type} rspamd-configuration
+Available @code{rspamd-configuration} fields are:
+
+@table @asis
+@item @code{package} (default: @code{rspamd}) (type: file-like)
+The package that provides rspamd.
+
+@item @code{config-file} (type: file-like)
+File-like object of the configuration file to use.  By default all
+workers are enabled except fuzzy and they are binded to their usual
+ports, e.g localhost:11334, localhost:11333 and so on
+
+@item @code{local.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in local.d, provided as a list of two element lists
+where the first element is the filename and the second one is a
+file-like object.  Settings in these files will be merged with the
+defaults.
+
+@item @code{override.d-files} (default: @code{()}) (type: directory-tree)
+Configuration files in override.d, provided as a list of two element
+lists where the first element is the filename and the second one is a
+file-like object.  Settings in these files will override the defaults.
+
+@item @code{user} (default: @code{"rspamd"}) (type: string)
+The user to run rspamd as.
+
+@item @code{group} (default: @code{"rspamd"}) (type: string)
+The group to run rspamd as.
+
+@item @code{pid-file} (default: @code{"/var/run/rspamd/rspamd.pid"}) (type: string)
+Where to store the PID file.
+
+@item @code{debug?} (default: @code{#f}) (type: boolean)
+Force debug output.
+
+@item @code{insecure?} (default: @code{#f}) (type: boolean)
+Ignore running workers as privileged users (insecure).
+
+@item @code{skip-template?} (default: @code{#f}) (type: boolean)
+Do not apply Jinja templates.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Messaging Services
 @subsection Messaging Services
 
diff --git a/gnu/services/mail.scm b/gnu/services/mail.scm
index 12dcc8e71d..d8720907c8 100644
--- a/gnu/services/mail.scm
+++ b/gnu/services/mail.scm
@@ -5,6 +5,8 @@
 ;;; Copyright © 2017, 2020 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Kristofer Buffington <kristoferbuffington@gmail.com>
 ;;; Copyright © 2020 Jonathan Brielmaier <jonathan.brielmaier@web.de>
+;;; Copyright © 2023 Thomas Ieong <th.ieong@free.fr>
+;;; Copyright © 2023 Saku Laesvuori <saku@laesvuori.fi>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -80,7 +82,11 @@ (define-module (gnu services mail)
             radicale-configuration
             radicale-configuration?
             radicale-service-type
-            %default-radicale-config-file))
+            %default-radicale-config-file
+
+            rspamd-configuration
+            rspamd-service-type
+            %default-rspamd-config-file))
 
 ;;; Commentary:
 ;;;
@@ -1987,3 +1993,192 @@ (define radicale-service-type
           (service-extension account-service-type (const %radicale-accounts))
           (service-extension activation-service-type radicale-activation)))
    (default-value (radicale-configuration))))
+
+;;;
+;;; Rspamd.
+;;;
+
+(define (directory-tree? xs)
+  (match xs
+    (((file-name file-like) ...)
+     (and (every string? file-name)
+          (every file-like? file-like)))
+    (_ #f)))
+
+(define-configuration/no-serialization rspamd-configuration
+  (package
+   (file-like rspamd)
+   "The package that provides rspamd.")
+  (config-file
+   (file-like %default-rspamd-config-file)
+   "File-like object of the configuration file to use.  By default
+all workers are enabled except fuzzy and they are binded
+to their usual ports, e.g localhost:11334, localhost:11333 and so on")
+  (local.d-files
+   (directory-tree '())
+   "Configuration files in local.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will be merged with the defaults.")
+  (override.d-files
+   (directory-tree '())
+   "Configuration files in override.d, provided as a list of two element lists where
+the first element is the filename and the second one is a file-like object.  Settings
+in these files will override the defaults.")
+  (user
+   (string "rspamd")
+   "The user to run rspamd as.")
+  (group
+   (string "rspamd")
+   "The group to run rspamd as.")
+  (pid-file
+   (string "/var/run/rspamd/rspamd.pid")
+   "Where to store the PID file.")
+  (debug?
+   (boolean #f)
+   "Force debug output.")
+  (insecure?
+   (boolean #f)
+   "Ignore running workers as privileged users (insecure).")
+  (skip-template?
+   (boolean #f)
+   "Do not apply Jinja templates."))
+
+(define %default-rspamd-config-file
+  (plain-file "rspamd.conf" "
+.include \"$CONFDIR/common.conf\"
+
+options {
+    pidfile = \"$RUNDIR/rspamd.pid\";
+    .include \"$CONFDIR/options.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/options.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/options.inc\"
+}
+
+logging {
+    type = \"file\";
+    filename = \"$LOGDIR/rspamd.log\";
+    .include \"$CONFDIR/logging.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/logging.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/logging.inc\"
+}
+
+worker \"normal\" {
+    bind_socket = \"localhost:11333\";
+    .include \"$CONFDIR/worker-normal.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-normal.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-normal.inc\"
+}
+
+worker \"controller\" {
+    bind_socket = \"localhost:11334\";
+    .include \"$CONFDIR/worker-controller.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-controller.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-controller.inc\"
+}
+
+worker \"rspamd_proxy\" {
+    bind_socket = \"localhost:11332\";
+    .include \"$CONFDIR/worker-proxy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-proxy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-proxy.inc\"
+}
+
+# Local fuzzy storage is disabled by default
+
+worker \"fuzzy\" {
+    bind_socket = \"localhost:11335\";
+    count = -1; # Disable by default
+    .include \"$CONFDIR/worker-fuzzy.inc\"
+    .include(try=true; priority=1,duplicate=merge) \"$LOCAL_CONFDIR/local.d/worker-fuzzy.inc\"
+    .include(try=true; priority=10) \"$LOCAL_CONFDIR/override.d/worker-fuzzy.inc\"
+}
+"))
+
+(define (rspamd-accounts config)
+  (match-record config <rspamd-configuration>
+    (user group)
+    (list
+     (user-group
+      (name group)
+      (system? #t))
+     (user-account
+      (name user)
+      (group group)
+      (system? #t)
+      (comment "Rspamd daemon")
+      (home-directory "/var/empty")
+      (shell (file-append shadow "/sbin/nologin"))))))
+
+(define (rspamd-shepherd-service config)
+  (match-record config <rspamd-configuration>
+    (package config-file user group pid-file debug? insecure? skip-template?
+     local.d-files override.d-files)
+    (list
+     (shepherd-service
+      (provision '(rspamd))
+      (documentation "Run the rspamd daemon.")
+      (requirement '(networking))
+      (start (let ((rspamd (file-append package "/bin/rspamd"))
+                   (local-confdir
+                     (file-union
+                      "rspamd-local-confdir"
+                      `(("local.d" ,(file-union "local.d" local.d-files))
+                        ("override.d" ,(file-union "override.d" override.d-files))))))
+               (with-imported-modules (source-module-closure '((gnu build activation)))
+                 #~(begin
+                     (use-modules (gnu build activation)) ; for mkdir-p/perms
+                     (let ((user (getpwnam #$user)))
+                       (mkdir-p/perms "/var/run/rspamd" user #o755)
+                       (mkdir-p/perms "/var/log/rspamd" user #o755)
+                       (mkdir-p/perms "/var/lib/rspamd" user #o755))
+                     (make-forkexec-constructor
+                      (list #$rspamd "-c" #$config-file
+                            "--var" (string-append "LOCAL_CONFDIR=" #$local-confdir)
+                            "--no-fork"
+                            #$@(if debug?
+                                 '("--debug")
+                                 '())
+                            #$@(if insecure?
+                                 '("--insecure")
+                                 '())
+                            #$@(if skip-template?
+                                 '("--skip-template")
+                                 '()))
+                      #:user #$user
+                      #:group #$group)))))
+      (stop #~(make-kill-destructor))
+      (actions
+       (list
+        (shepherd-configuration-action config-file)
+        (shepherd-action
+         (name 'reload)
+         (documentation "Reload rspamd.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGHUP)
+                  (display "Service rspamd has been reloaded"))
+                (format #t "Service rspamd is not running.")))))
+        (shepherd-action
+         (name 'reopenlog)
+         (documentation "Reopen log files.")
+         (procedure
+          #~(lambda (pid)
+              (if pid
+                (begin
+                  (kill pid SIGUSR1)
+                  (display "Reopening the logs for rspamd"))
+                (format #t "Service rspamd is not running.")))))))))))
+
+(define rspamd-service-type
+  (service-type
+   (name 'rspamd)
+   (description "Run the rapid spam filtering system.")
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type rspamd-shepherd-service)
+     (service-extension account-service-type rspamd-accounts)
+     (service-extension profile-service-type
+                        (compose list rspamd-configuration-package))))
+   (default-value (rspamd-configuration))))
diff --git a/gnu/tests/mail.scm b/gnu/tests/mail.scm
index dcb8f08ea8..4dae6886b2 100644
--- a/gnu/tests/mail.scm
+++ b/gnu/tests/mail.scm
@@ -6,6 +6,7 @@
 ;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
 ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2023 Thomas Ieong <th.ieong@free.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -40,7 +41,8 @@ (define-module (gnu tests mail)
   #:export (%test-opensmtpd
             %test-exim
             %test-dovecot
-            %test-getmail))
+            %test-getmail
+            %test-rspamd))
 
 (define %opensmtpd-os
   (simple-operating-system
@@ -579,3 +581,86 @@ (define %test-getmail
    (name "getmail")
    (description "Connect to a running Getmail server.")
    (value (run-getmail-test))))
+
+(define %rspamd-os
+  (simple-operating-system
+   (service dhcp-client-service-type)
+   (service rspamd-service-type)))
+
+(define (run-rspamd-test)
+  "Return a test of an OS running Rspamd service."
+
+  (define rspamd-ports
+    '((22664 . 11332)    ;; proxy worker
+      (22666 . 11333)    ;; normal worker
+      (22668 . 11334)    ;; web controller
+      (22670 . 11335)))  ;; fuzzy worker
+
+  (define vm
+    (virtual-machine
+     (operating-system (marionette-operating-system
+                        %rspamd-os
+                        #:imported-modules '((gnu services herd))))
+     (port-forwardings rspamd-ports)))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-64)
+                       (srfi srfi-11)
+                       (gnu build marionette)
+                       (web uri)
+                       (web client)
+                       (web response))
+
+          (define marionette
+            (make-marionette '(#$vm)))
+
+          (test-runner-current (system-test-runner #$output))
+          (test-begin "rspamd")
+
+          (test-assert "service is running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'rspamd))
+             marionette))
+
+
+          ;; Check mympd-service-type commit for reference
+          ;; TODO: For this test we need to authorize the controller to
+          ;; listen on other interfaces, e.g *:11334 instead of localhost:11334
+
+          ;; Check that we can access the web ui
+          (test-equal "http-get"
+            200
+            (begin
+              (let-values (((response text)
+                            (http-get "http://localhost:22668/"
+                                      #:decode-body? #t)))
+                (response-code response))))
+
+          (test-assert "rspamd socket ready"
+            (wait-for-unix-socket
+             "/var/lib/rspamd/rspamd.sock"
+             marionette))
+
+          (test-assert "rspamd pid ready"
+            (marionette-eval
+             '(file-exists? "/var/run/rspamd/rspamd.pid")
+             marionette))
+
+          (test-assert "rspamd log file"
+            (marionette-eval
+             '(file-exists? "/var/log/rspamd/rspamd.log")
+             marionette))
+
+          (test-end))))
+
+  (gexp->derivation "rspamd-test" test))
+
+(define %test-rspamd
+  (system-test
+   (name "rspamd")
+   (description "Send an email to a running rspamd server.")
+   (value (run-rspamd-test))))

base-commit: 2d4d147839b81ba8761c9e50cabe9b60025dc670
-- 
2.41.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]