Adding a TOR system test - confusing results

[Chris Marusich <cmmarusich@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2452 bytes --]

Hi Guix,

In response to a recent email thread [1], I'd like to update our TOR
service to allow users to run TOR over a UNIX-domain socket.  I
discovered that TOR has no system tests, so I decided to start by
creating a basic system test to protect against regressions I might
accidentally introduce.

Unfortunately, the system test I've added behaves mysteriously.  It
succeeds when I think it obviously should not.  Could you please help me
understand what's going on?

Let's examine the problem.  I've attached two patches to this email.
They apply cleanly to 889e7fab3c04be98a59b880bf44fbdaa6ddf0a4e.

To begin, let's apply the first patch ("Add system tests for the TOR
service.").  Then run the test:

--8<---------------cut here---------------start------------->8---
$ make -j1 check-system TESTS=tor
[...]
# of expected passes      1
@ build-succeeded /gnu/store/zdh97qnh4b69yr23yp5js16hw9lvjhzm-tor-test.drv -
TOTAL: 1
PASS: /gnu/store/c3f0y7jbqi1qw8s63p3jbp5mrwl1bxnc-tor-test
--8<---------------cut here---------------end--------------->8---

It passed!  Great!  Now, let's make sure it also detects failures
correctly: Let's break TOR by applying the second patch ("For testing
purposes, break the TOR service.").  Then run it again:

--8<---------------cut here---------------start------------->8---
$ make -j1 check-system TESTS=tor
[...]
This is the GNU system.  Welcome.
komputilo login: exec of "/gnu/store/5jhlsv29n6kx76na1kkgfarznrl5kjjg-tor-0.3.3.9DOES-NOT-EXIST" failed: No such file or directory
QEMU runs as PID 4
connected to QEMU's monitor
read QEMU monitor prompt
connected to guest REPL
%%%% Starting test tor  (Writing full log to "tor.log")
marionette is ready
# of expected passes      1
@ build-succeeded /gnu/store/z8q9y8zivsipaq9fdj7yhhcvqgfq8ryl-tor-test.drv -
TOTAL: 1
PASS: /gnu/store/r84kh8x3d9ypqi6nl6r9xkqpk9i8jgk2-tor-test
[0] [env] marusich@garuda.local:~/guix
$ 
--8<---------------cut here---------------end--------------->8---

The test still passes, but I can't figure out why.  There is even an
error message in there, complaining that we can't exec the path to the
TOR program.  What is going on?  I've already tried rebuilding
everything from a clean slate, so that isn't the issue.

Any help would be greatly appreciated!  Thank you in advance,

Footnotes: 
[1]  https://lists.gnu.org/archive/html/help-guix/2018-07/msg00087.html

-- 
Chris

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Add-system-tests-for-the-TOR-service.patch --]
[-- Type: text/x-patch, Size: 2527 bytes --]

From 63f84210927bc45ad0d239b4b92aaf028fa5c5ac Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Sun, 22 Jul 2018 16:23:53 -0700
Subject: [PATCH 1/2] Add system tests for the TOR service.

---
 gnu/tests/networking.scm | 55 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 54 insertions(+), 1 deletion(-)

diff --git a/gnu/tests/networking.scm b/gnu/tests/networking.scm
index 323679e7f..1128e3448 100644
--- a/gnu/tests/networking.scm
+++ b/gnu/tests/networking.scm
@@ -30,7 +30,7 @@
   #:use-module (gnu packages bash)
   #:use-module (gnu packages networking)
   #:use-module (gnu services shepherd)
-  #:export (%test-inetd %test-openvswitch %test-dhcpd))
+  #:export (%test-inetd %test-openvswitch %test-dhcpd %test-tor))
 
 (define %inetd-os
   ;; Operating system with 2 inetd services.
@@ -339,3 +339,56 @@ subnet 192.168.1.0 netmask 255.255.255.0 {
    (name "dhcpd")
    (description "Test a running DHCP daemon configuration.")
    (value (run-dhcpd-test))))
+
+\f
+;;;
+;;; Services related to TOR
+;;;
+
+(define %tor-os
+  (simple-operating-system
+   (tor-service)))
+
+(define (run-tor-test)
+  (define os
+    (marionette-operating-system %tor-os
+                                 #:imported-modules '((gnu services herd))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (gnu build marionette)
+                       (ice-9 popen)
+                       (ice-9 rdelim)
+                       (srfi srfi-64))
+
+          (define marionette
+            (make-marionette (list #$(virtual-machine os))))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "tor")
+
+          (test-assert "tor is alive"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd)
+                             (srfi srfi-1))
+                (live-service-running
+                 (find (lambda (live)
+                         (memq 'tor
+                               (live-service-provision live)))
+                       (current-services))))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "tor-test" test))
+
+(define %test-tor
+  (system-test
+   (name "tor")
+   (description "Test a running TOR daemon configuration.")
+   (value (run-tor-test))))
-- 
2.18.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0002-For-testing-purposes-break-the-TOR-service.patch --]
[-- Type: text/x-patch, Size: 984 bytes --]

From f2ac689f2d1cc308eb7b4371130b4b2a5ea1a393 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Sun, 22 Jul 2018 16:38:23 -0700
Subject: [PATCH 2/2] For testing purposes, break the TOR service.

---
 gnu/services/networking.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/services/networking.scm b/gnu/services/networking.scm
index d5d0cf9d1..7996aae7e 100644
--- a/gnu/services/networking.scm
+++ b/gnu/services/networking.scm
@@ -657,7 +657,7 @@ HiddenServicePort ~a ~a~%"
                            (gnu system file-systems)))
 
                 (start #~(make-forkexec-constructor/container
-                          (list #$(file-append tor "/bin/tor") "-f" #$torrc)
+                          (list #$(file-append tor "DOES-NOT-EXIST") "-f" #$torrc)
 
                           #:mappings (list (file-system-mapping
                                             (source "/var/lib/tor")
-- 
2.18.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]