[bug#74151] [PATCH 0/7] Add anonip system test.

[bug#74151] [PATCH 0/7] Add anonip system test.

[Maxim Cournoyer]

This was developed as part of investigating the source of bug #59181 ([berlin]
web services fail to start on reboot due to anonip).

There is no issue having anonip run and the system reboot (as should be
expected as /var/run is supposed to be cleared thus the named pipes should be
recreated every boot).  This points to a race with nginx that would "win"
creating its log files under /var/run/anonip/* before the anonip service does.

It'll need further investigating in that direction.

Maxim Cournoyer (7):
  doc: Use @table @code for anonip-configuration doc.
  services: anonip: Add 'debug?' configuration field.
  system/vm: Fix virtual-machine bug.
  tests: web: Have the retry-on-error throw on exhausted attempts.
  services: herd: Export 'eval-there' in API.
  build: marionette: Make it possible to reboot VM during tests.
  tests: Add anonip system test.

 doc/guix.texi            |  21 ++++---
 gnu/build/marionette.scm |   2 +-
 gnu/services/herd.scm    |   1 +
 gnu/services/web.scm     |   8 ++-
 gnu/system/vm.scm        |   9 ++-
 gnu/tests/web.scm        | 117 ++++++++++++++++++++++++++++++++++++++-
 6 files changed, 142 insertions(+), 16 deletions(-)


base-commit: 4009d1de954d694cb11af391d4113d29c5c1379d
-- 
2.46.0

[bug#74151] [PATCH 1/7] doc: Use @table @code for anonip-configuration doc.

[Maxim Cournoyer]

* doc/guix.texi (Log Rotation): Use @table @code for anonip-configuration doc.

Change-Id: Ib6f8136715ecabc81d53dc13a14f4b111302c5c9
---
 doc/guix.texi | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5768e195b0..b519b483fe 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -20788,30 +20788,30 @@ Log Rotation
 
 The following optional settings may be provided:
 
-@table @asis
-@item @code{skip-private?}
+@table @code
+@item skip-private?
 When @code{#true} do not mask addresses in private ranges.
 
-@item @code{column}
+@item column
 A 1-based indexed column number.  Assume IP address is in the specified
 column (default is 1).
 
-@item @code{replacement}
+@item replacement
 Replacement string in case address parsing fails, e.g. @code{"0.0.0.0"}.
 
-@item @code{ipv4mask}
+@item ipv4mask
 Number of bits to mask in IPv4 addresses.
 
-@item @code{ipv6mask}
+@item ipv6mask
 Number of bits to mask in IPv6 addresses.
 
-@item @code{increment}
+@item increment
 Increment the IP address by the given number.  By default this is zero.
 
-@item @code{delimiter}
+@item delimiter
 Log delimiter string.
 
-@item @code{regex}
+@item regex
 Regular expression for detecting IP addresses.  Use this instead of @code{column}.
 @end table
 @end deftp
-- 
2.46.0

[bug#74151] [PATCH 2/7] services: anonip: Add 'debug?' configuration field.

[Maxim Cournoyer]

* gnu/services/web.scm (<anonip-configuration>) [debug?]: New field.
(anonip-shepherd-service): Honor it.
* doc/guix.texi (Log Rotation) <anonip-configuration>: Document it.

Change-Id: Iaf57b5992808374b069a55c34a9adfdfe52b046c
---
 doc/guix.texi        | 3 +++
 gnu/services/web.scm | 8 ++++++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index b519b483fe..51a543d690 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -20789,6 +20789,9 @@ Log Rotation
 The following optional settings may be provided:
 
 @table @code
+@item debug?
+Print debug messages when @code{#true}.
+
 @item skip-private?
 When @code{#true} do not mask addresses in private ranges.
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index cc6f4e6d9b..cf3515bf70 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -220,6 +220,7 @@ (define-module (gnu services web)
             anonip-configuration-anonip
             anonip-configuration-input
             anonip-configuration-output
+            anonip-configuration-debug?
             anonip-configuration-skip-private?
             anonip-configuration-column
             anonip-configuration-replacement
@@ -1448,6 +1449,8 @@ (define-record-type* <anonip-configuration>
                      (default anonip))
   (input             anonip-configuration-input)       ;string
   (output            anonip-configuration-output)      ;string
+  (debug?            anonip-configuration-debug?       ;boolean
+                     (default #f))
   (skip-private?     anonip-configuration-skip-private? ;boolean
                      (default #f))
   (column            anonip-configuration-column       ;number
@@ -1503,14 +1506,15 @@ (define (anonip-shepherd-service config)
                                     "/bin/anonip")
                      (string-append "--input=" #$input)
                      (string-append "--output=" #$output))
+               (if #$(anonip-configuration-debug? config)
+                   '("--debug") (list))
                (if #$(anonip-configuration-skip-private? config)
                    '("--skip-private") (list))
                '#$(optional anonip-configuration-column "--column")
                '#$(optional anonip-configuration-ipv4mask "--ipv4mask")
                '#$(optional anonip-configuration-ipv6mask "--ipv6mask")
                '#$(optional anonip-configuration-increment "--increment")
-               '#$(optional anonip-configuration-replacement
-                            "--replacement")
+               '#$(optional anonip-configuration-replacement "--replacement")
                '#$(optional anonip-configuration-delimiter "--delimiter")
                '#$(optional anonip-configuration-regex "--regex"))
               ;; Run in a UTF-8 locale
-- 
2.46.0

[bug#74151] [PATCH 3/7] system/vm: Fix virtual-machine bug.

[Maxim Cournoyer]

The virtual-machine syntax would not accept a single operating-system field,
which was puzzling.

* gnu/system/vm.scm (virtual-machine): Add a pattern matching a single literal
'operating-system' field and value.

Change-Id: If207fd71df3a3f763b2e63229eafa82f63e80773
---
 gnu/system/vm.scm | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index a2743453e7..4a400056d7 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -422,10 +422,14 @@ (define-record-type* <virtual-machine> %virtual-machine
                     (default #f)))
 
 (define-syntax virtual-machine
-  (syntax-rules ()
+  (syntax-rules (operating-system)
     "Declare a virtual machine running the specified OS, with the given
 options."
-    ((_ os)                                       ;shortcut
+    ((_ (operating-system os))
+     ;; Also accept the long form (virtual-machine (operating-system os)), for
+     ;; correctness.
+     (%virtual-machine (operating-system os)))
+    ((_ os)                             ;shortcut
      (%virtual-machine (operating-system os)))
     ((_ fields ...)
      (%virtual-machine fields ...))))
-- 
2.46.0

[bug#74151] [PATCH 4/7] tests: web: Have the retry-on-error throw on exhausted attempts.

[Maxim Cournoyer]

Previously, it'd simply return the error arguments, which would not always
fail a 'test-assert' test, for example.

* gnu/tests/web.scm (retry-on-error): Re-throw the error when the attempts
have been exhausted.

Change-Id: Ic1468d8ba23f0f78377e90d19bccb2878dc56f0e
---
 gnu/tests/web.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/tests/web.scm b/gnu/tests/web.scm
index a071e05e1d..5c50322cef 100644
--- a/gnu/tests/web.scm
+++ b/gnu/tests/web.scm
@@ -83,7 +83,7 @@ (define retry-on-error
            return-value)
           ((#f . error-args)
            (if (>= attempt times)
-               error-args
+               (apply throw error-args)
                (begin
                  (sleep delay)
                  (loop (+ 1 attempt)))))))))
-- 
2.46.0

[bug#74151] [PATCH 5/7] services: herd: Export 'eval-there' in API.

[Maxim Cournoyer]

* gnu/services/herd.scm (gnu): Export 'eval-there', which is useful to
evaluate Scheme expressions.

Change-Id: Iff1db56e0847fc5886ac9fe594d677b363e7e6e7
---
 gnu/services/herd.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gnu/services/herd.scm b/gnu/services/herd.scm
index 4b47acf72a..c185fc2cec 100644
--- a/gnu/services/herd.scm
+++ b/gnu/services/herd.scm
@@ -54,6 +54,7 @@ (define-module (gnu services herd)
             with-shepherd-action
             current-service
             current-services
+            eval-there
             unload-services
             unload-service
             load-services
-- 
2.46.0

[bug#74151] [PATCH 6/7] build: marionette: Make it possible to reboot VM during tests.

[Maxim Cournoyer]

* gnu/build/marionette.scm (make-marionette): Add 'reconnect=1' socket
parameter.
* gnu/system/vm.scm (common-qemu-options): Remove '-no-reboot' option.

Change-Id: I5e100543ddddba0aea3ebe4e2f5cb8b0261c0d73
---
 gnu/build/marionette.scm | 2 +-
 gnu/system/vm.scm        | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/build/marionette.scm b/gnu/build/marionette.scm
index 0b0a8a70d8..af211eff43 100644
--- a/gnu/build/marionette.scm
+++ b/gnu/build/marionette.scm
@@ -108,7 +108,7 @@ (define* (make-marionette command
     (list "-nographic"
           "-monitor" (string-append "unix:" socket-directory "/monitor")
           "-chardev" (string-append "socket,id=repl,path=" socket-directory
-                                    "/repl")
+                                    "/repl,reconnect=1")
           "-chardev" (string-append "socket,id=qga,server=on,wait=off,path="
                                     socket-directory "/qemu-ga")
 
diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 4a400056d7..1e3f72c7b2 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -227,7 +227,6 @@ (define* (common-qemu-options image shared-fs
             '("-enable-kvm")
             '())
 
-     "-no-reboot"
      "-object" "rng-random,filename=/dev/urandom,id=guix-vm-rng"
      "-device" "virtio-rng-pci,rng=guix-vm-rng"
 
-- 
2.46.0

[bug#74151] [PATCH 7/7] tests: Add anonip system test.

[Maxim Cournoyer]

* gnu/tests/web.scm (%test-anonip): New test.
(%anonip-os): New variables.
(run-anonip-test): New procedure.

Change-Id: Ieed210a784dbdeee8a498e74b6c0e31cb72cd9b8
---
 gnu/tests/web.scm | 115 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)

diff --git a/gnu/tests/web.scm b/gnu/tests/web.scm
index 5c50322cef..f7a5659dc5 100644
--- a/gnu/tests/web.scm
+++ b/gnu/tests/web.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2017, 2018 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2018 Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
 ;;; Copyright © 2018 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,7 @@ (define-module (gnu tests web)
   #:use-module (gnu services networking)
   #:use-module (gnu services shepherd)
   #:use-module (gnu services mail)
+  #:use-module (gnu packages base)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages guile-xyz)
   #:use-module (gnu packages patchutils)
@@ -52,6 +54,7 @@ (define-module (gnu tests web)
             %test-php-fpm
             %test-hpcguix-web
             %test-tailon
+            %test-anonip
             %test-patchwork
             %test-agate))
 
@@ -509,6 +512,118 @@ (define %test-tailon
    (description "Connect to a running Tailon server.")
    (value (run-tailon-test))))
 
+\f
+;;;
+;;; Anonip
+;;;
+(define %anonip-os
+  ;; Operating system under test.
+  (simple-operating-system
+   (service anonip-service-type
+            (anonip-configuration
+             (input "/var/run/anonip/http.access.log")
+             (output "/var/log/anonip/http.access.log")
+             (debug? #t)))))
+
+(define (run-anonip-test)
+  (define os
+    (marionette-operating-system
+     %anonip-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     ;; We are interested in verifying if anonip still launches following a
+     ;; reboot; thus make the base image writable.
+     (volatile? #f)))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (ice-9 match)
+                       (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (test-runner-current (system-test-runner #$output))
+          (test-begin "anonip")
+
+          (test-assert "service is running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (wait-for-service 'anonip-/var/log/anonip/http.access.log))
+             marionette))
+
+          (test-assert "service can be restarted"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (restart-service 'anonip-/var/log/anonip/http.access.log)
+                (wait-for-service 'anonip-/var/log/anonip/http.access.log))
+             marionette))
+
+          (test-assert "ip addresses are anonymized"
+            (marionette-eval
+             '(begin
+                (use-modules (ice-9 textual-ports))
+                (call-with-output-file "/var/run/anonip/http.access.log"
+                  (lambda (port)
+                    (display "192.168.100.200 - - \
+[30/Oct/2024:14:57:44 +0100] GET /xxx.narinfo HTTP/1.1\" 200 1065 \
+\"-\" \"GNU Guile\"\n" port)
+                    (display "2001:0db8:85a3:0000:0000:8a2e:0370:7334 - - \
+[30/Oct/2024:14:57:44 +0100] \"GET /xxx.narinfo HTTP/1.1\" 200 1065 \
+\"-\" \"GNU Guile\"\n" port)))
+                (#$retry-on-error
+                 (lambda ()
+                   (call-with-input-file "/var/log/anonip/http.access.log"
+                     (lambda (port)
+                       (let ((content (get-string-all port)))
+                         ;; The expected values are taken from anonip's test
+                         ;; suite (see its test_module.py file).
+                         (or (and (string-contains content "192.168.96.0")
+                                  (string-contains content "2001:db8:85a0::"))
+                             (error "could not find expected anonymized IPs"
+                                    content))))))
+                 #:times 20
+                 #:delay 1))
+             marionette))
+
+          (test-assert "service is running after reboot"
+            (begin
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd))
+                  (eval-there '(begin
+                                 (use-modules (shepherd system))
+                                 (sync) ;ensure the log is fully written
+                                 (reboot))))
+               marionette)
+              ;; Note: a distinct marionette-eval call is needed here; if
+              ;; included in the previous one issuing the reboot,
+              ;; 'wait-for-service' would apparently run before the system had
+              ;; rebooted (and succeed), which would defeat the test.
+              (marionette-eval
+               '(begin
+                  (use-modules (gnu services herd))
+                  (wait-for-service 'anonip-/var/log/anonip/http.access.log))
+               marionette)))
+
+          (test-end))))
+
+  (gexp->derivation "anonip-test" test))
+
+(define %test-anonip
+  (system-test
+   (name "anonip")
+   (description "Anonymize logs via Anonip")
+   (value (run-anonip-test))))
+
 \f
 ;;;
 ;;; Patchwork
-- 
2.46.0