[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Bruno Victal]

It turns out that the issue with the xvnc test was due to using invoke
instead of system*.
Along the way, the GDM test was refactored to replace the ugly sleep
delay with a more adequate awaiting method.

Bruno Victal (4):
  tests: xvnc: Fix test.
  marionette: Allow passing custom OCR arguments.
  tests: gdm: Prefer OCR to delay.
  tests: xvnc: Group up GDM test.

 gnu/build/marionette.scm | 34 +++++++++++++++++-------
 gnu/tests/gdm.scm        | 19 ++++++++++----
 gnu/tests/vnc.scm        | 56 ++++++++++++++++++++--------------------
 3 files changed, 67 insertions(+), 42 deletions(-)


base-commit: 94ac93042f09b4ba68b7b64ed1feeebd3dab1ea4
-- 
2.39.2

[bug#64356] [PATCH 1/4] tests: xvnc: Fix test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Use system* instead of invoke.
---
 gnu/tests/vnc.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index 5c4bd43fa3..e59972eee4 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -142,11 +142,11 @@ (define (run-xvnc-test)
               (marionette-eval
                '(begin
                   ;; Check that DCONF_PROFILE is set...
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 pgrep gdm | head -n1 | xargs -I{} grep -Fq DCONF_PROFILE /proc/{}/environ")
 
                   ;; ... and that
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 sudo -E -u gdm env DCONF_PROFILE=/etc/dconf/profile/gdm dbus-run-session \
 gsettings get org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type \
 | grep -Fq nothing"))

base-commit: 94ac93042f09b4ba68b7b64ed1feeebd3dab1ea4
-- 
2.39.2

[bug#64356] [PATCH 2/4] marionette: Allow passing custom OCR arguments.

[Bruno Victal]

* gnu/build/marionette.scm (invoke-ocrad-ocr, invoke-tesseract-ocr)
(marionette-screen-text): New 'ocr-arguments' argument.
---
 gnu/build/marionette.scm | 34 +++++++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 9 deletions(-)

diff --git a/gnu/build/marionette.scm b/gnu/build/marionette.scm
index b8fba61d06..5621896198 100644
--- a/gnu/build/marionette.scm
+++ b/gnu/build/marionette.scm
@@ -287,23 +287,30 @@ (define (marionette-control command marionette)
      ;; The "quit" command terminates QEMU immediately, with no output.
      (unless (string=? command "quit") (wait-for-monitor-prompt monitor)))))
 
-(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad"))
+(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad") ocr-arguments)
   "Invoke the OCRAD command on image, and return the recognized text."
-  (let* ((pipe (open-pipe* OPEN_READ ocrad "-i" "-s" "10" image))
+  (let* ((arguments (or ocr-arguments
+                        "--invert --scale 10"))
+         (command (string-join (list ocrad ocr-arguments image)))
+         (pipe (open-input-pipe command))
          (text (get-string-all pipe)))
     (unless (zero? (close-pipe pipe))
       (error "'ocrad' failed" ocrad))
     text))
 
-(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
+(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract")
+                               ocr-arguments)
   "Invoke the TESSERACT command on IMAGE, and return the recognized text."
   (let* ((output-basename (tmpnam))
-         (output-basename* (string-append output-basename ".txt")))
+         (output-basename* (string-append output-basename ".txt"))
+         (arguments (cons* image output-basename
+                           (or (and=> ocr-arguments list)
+                               '()))))
     (dynamic-wind
       (const #t)
       (lambda ()
         (let ((exit-val (status:exit-val
-                         (system* tesseract image output-basename))))
+                         (apply system* tesseract arguments))))
           (unless (zero? exit-val)
             (error "'tesseract' failed" tesseract))
           (call-with-input-file output-basename* get-string-all)))
@@ -311,7 +318,8 @@ (define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
         (false-if-exception (delete-file output-basename))
         (false-if-exception (delete-file output-basename*))))))
 
-(define* (marionette-screen-text marionette #:key (ocr "ocrad"))
+(define* (marionette-screen-text marionette #:key (ocr "ocrad")
+                                 ocr-arguments)
   "Take a screenshot of MARIONETTE, perform optical character
 recognition (OCR), and return the text read from the screen as a string, along
 the screen dump image used.  Do this by invoking OCR, which should be the file
@@ -324,14 +332,19 @@ (define* (marionette-screen-text marionette #:key (ocr "ocrad"))
   ;; Process it via the OCR.
   (cond
    ((string-contains ocr "ocrad")
-    (values (invoke-ocrad-ocr image #:ocrad ocr) image))
+    (values (invoke-ocrad-ocr image
+                              #:ocrad ocr
+                              #:ocr-arguments ocr-arguments) image))
    ((string-contains ocr "tesseract")
-    (values (invoke-tesseract-ocr image #:tesseract ocr) image))
+    (values (invoke-tesseract-ocr image
+                                  #:tesseract ocr
+                                  #:ocr-arguments ocr-arguments) image))
    (else (error "unsupported ocr command"))))
 
 (define* (wait-for-screen-text marionette predicate
                                #:key
                                (ocr "ocrad")
+                               ocr-arguments
                                (timeout 30)
                                pre-action
                                post-action)
@@ -359,7 +372,10 @@ (define* (wait-for-screen-text marionette predicate
                  'ocr-text: last-text
                  'screendump: screendump-backup))
         (let* ((_ (and (procedure? pre-action) (pre-action)))
-               (text screendump (marionette-screen-text marionette #:ocr ocr))
+               (text screendump
+                     (marionette-screen-text marionette
+                                             #:ocr ocr
+                                             #:ocr-arguments ocr-arguments))
                (_ (and (procedure? post-action) (post-action)))
                (result (predicate text)))
           (cond (result
-- 
2.39.2

[bug#64356] [PATCH 3/4] tests: gdm: Prefer OCR to delay.

[Bruno Victal]

* gnu/tests/gdm.scm (run-gdm-test): Use wait-for-screen-text instead of sleep.
---
 gnu/tests/gdm.scm | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/gnu/tests/gdm.scm b/gnu/tests/gdm.scm
index ec1df4b797..77163bc3e0 100644
--- a/gnu/tests/gdm.scm
+++ b/gnu/tests/gdm.scm
@@ -19,6 +19,7 @@
 (define-module (gnu tests gdm)
   #:use-module (gnu tests)
   #:use-module (gnu packages freedesktop)
+  #:use-module (gnu packages ocr)
   #:use-module (gnu services)
   #:use-module (gnu services desktop)
   #:use-module (gnu services xorg)
@@ -57,6 +58,7 @@ (define* (run-gdm-test #:key wayland?)
       #~(begin
           (use-modules (gnu build marionette)
                        (ice-9 format)
+                       (srfi srfi-26)
                        (srfi srfi-64))
 
           (let ((marionette (make-marionette (list #$vm)))
@@ -73,11 +75,18 @@ (define* (run-gdm-test #:key wayland?)
                   (start-service 'xorg-server))
                marionette))
 
-            (test-assert "gdm ready"
-              (wait-for-file "/var/run/gdm/gdm.pid" marionette))
-
-            ;; waiting for gdm.pid is not enough, tests may still sporadically fail.
-            (sleep 1)
+            (test-group "gdm ready"
+              (test-assert "PID file present"
+                (wait-for-file "/var/run/gdm/gdm.pid" marionette))
+
+              ;; Waiting for gdm.pid is not enough, tests may still sporadically
+              ;; fail; ensure that the login screen is up.
+              ;; XXX: GNU Ocrad works but with '--invert' only.
+              (test-assert "login screen up"
+                (wait-for-screen-text marionette
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$(file-append ocrad "/bin/ocrad")
+                                      #:ocr-arguments "--invert")))
 
             (test-equal (string-append "session-type is " expected-session-type)
               expected-session-type
-- 
2.39.2

[bug#64356] [PATCH 4/4] tests: xvnc: Group up GDM test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Group up GDM test. Use GNU Ocrad when
possible.
---
 gnu/tests/vnc.scm | 52 +++++++++++++++++++++++------------------------
 1 file changed, 26 insertions(+), 26 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index e59972eee4..504cb544b1 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -152,43 +152,43 @@ (define (run-xvnc-test)
 | grep -Fq nothing"))
                marionette))
 
-            (test-assert "vnc lands on the gdm login screen"
+            (test-group "vnc lands on the gdm login screen"
               ;; This test runs vncviewer on the local VM and verifies that it
               ;; manages to access the GDM login screen (via XDMCP).
-              (begin
-                (define (ratpoison-abort)
-                  (marionette-control "sendkey ctrl-g" marionette))
-
-                (define (ratpoison-help)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "?" marionette)
-                  (sleep 1))            ;wait for help screen to appear
-
-                (define (ratpoison-exec command)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "!" marionette)
-                  (marionette-type (string-append command "\n") marionette))
-
-                ;; Wait until the ratpoison help screen can be displayed; this
-                ;; means the window manager is ready.
+              (define (ratpoison-abort)
+                (marionette-control "sendkey ctrl-g" marionette))
+
+              (define (ratpoison-help)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "?" marionette)
+                (sleep 1))            ;wait for help screen to appear
+
+              (define (ratpoison-exec command)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "!" marionette)
+                (marionette-type (string-append command "\n") marionette))
+
+              ;; Wait until the ratpoison help screen can be displayed; this
+              ;; means the window manager is ready.
+              ;; XXX: GNU Ocrad 0.28 yields poor results here, use tesseract
+              ;; instead.
+              (test-assert "window manager is ready"
                 (wait-for-screen-text marionette
                                       (cut string-contains <> "key bindings")
                                       #:ocr #$(file-append tesseract-ocr
                                                            "/bin/tesseract")
                                       #:pre-action ratpoison-help
-                                      #:post-action ratpoison-abort)
+                                      #:post-action ratpoison-abort))
 
                 ;; Run vncviewer and expect the GDM login screen (accessed via
                 ;; XDMCP).  This can take a while to appear on slower machines.
-                (ratpoison-exec "vncviewer localhost:5905")
-                ;; XXX: tesseract narrowly recognizes "Guix" as "uix" from the
-                ;; background image; ocrad fares worst.  Sadly, 'Username' is
-                ;; not recognized at all.
+              (ratpoison-exec "vncviewer localhost:5905")
+
+              (test-assert "GDM login screen ready"
                 (wait-for-screen-text marionette
-                                      (cut string-contains <> "uix")
-                                      #:ocr #$(file-append tesseract-ocr
-                                                           "/bin/tesseract")
-                                      #:timeout 120)))
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$(file-append ocrad "/bin/ocrad")
+                                      #:ocr-arguments "--invert")))
 
             (test-end)))))
 
-- 
2.39.2

[bug#64356] [PATCH v2 1/4] tests: xvnc: Fix test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Use system* instead of invoke.
---
 gnu/tests/vnc.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index 5c4bd43fa3..e59972eee4 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -142,11 +142,11 @@ (define (run-xvnc-test)
               (marionette-eval
                '(begin
                   ;; Check that DCONF_PROFILE is set...
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 pgrep gdm | head -n1 | xargs -I{} grep -Fq DCONF_PROFILE /proc/{}/environ")
 
                   ;; ... and that
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 sudo -E -u gdm env DCONF_PROFILE=/etc/dconf/profile/gdm dbus-run-session \
 gsettings get org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type \
 | grep -Fq nothing"))

base-commit: 94ac93042f09b4ba68b7b64ed1feeebd3dab1ea4
-- 
2.39.2

[bug#64356] [PATCH v2 2/4] marionette: Allow passing custom OCR arguments.

[Bruno Victal]

* gnu/build/marionette.scm (invoke-ocrad-ocr, invoke-tesseract-ocr)
(marionette-screen-text): New 'ocr-arguments' argument.
---
 gnu/build/marionette.scm | 34 +++++++++++++++++++++++++---------
 1 file changed, 25 insertions(+), 9 deletions(-)

diff --git a/gnu/build/marionette.scm b/gnu/build/marionette.scm
index b8fba61d06..5621896198 100644
--- a/gnu/build/marionette.scm
+++ b/gnu/build/marionette.scm
@@ -287,23 +287,30 @@ (define (marionette-control command marionette)
      ;; The "quit" command terminates QEMU immediately, with no output.
      (unless (string=? command "quit") (wait-for-monitor-prompt monitor)))))
 
-(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad"))
+(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad") ocr-arguments)
   "Invoke the OCRAD command on image, and return the recognized text."
-  (let* ((pipe (open-pipe* OPEN_READ ocrad "-i" "-s" "10" image))
+  (let* ((arguments (or ocr-arguments
+                        "--invert --scale 10"))
+         (command (string-join (list ocrad ocr-arguments image)))
+         (pipe (open-input-pipe command))
          (text (get-string-all pipe)))
     (unless (zero? (close-pipe pipe))
       (error "'ocrad' failed" ocrad))
     text))
 
-(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
+(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract")
+                               ocr-arguments)
   "Invoke the TESSERACT command on IMAGE, and return the recognized text."
   (let* ((output-basename (tmpnam))
-         (output-basename* (string-append output-basename ".txt")))
+         (output-basename* (string-append output-basename ".txt"))
+         (arguments (cons* image output-basename
+                           (or (and=> ocr-arguments list)
+                               '()))))
     (dynamic-wind
       (const #t)
       (lambda ()
         (let ((exit-val (status:exit-val
-                         (system* tesseract image output-basename))))
+                         (apply system* tesseract arguments))))
           (unless (zero? exit-val)
             (error "'tesseract' failed" tesseract))
           (call-with-input-file output-basename* get-string-all)))
@@ -311,7 +318,8 @@ (define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
         (false-if-exception (delete-file output-basename))
         (false-if-exception (delete-file output-basename*))))))
 
-(define* (marionette-screen-text marionette #:key (ocr "ocrad"))
+(define* (marionette-screen-text marionette #:key (ocr "ocrad")
+                                 ocr-arguments)
   "Take a screenshot of MARIONETTE, perform optical character
 recognition (OCR), and return the text read from the screen as a string, along
 the screen dump image used.  Do this by invoking OCR, which should be the file
@@ -324,14 +332,19 @@ (define* (marionette-screen-text marionette #:key (ocr "ocrad"))
   ;; Process it via the OCR.
   (cond
    ((string-contains ocr "ocrad")
-    (values (invoke-ocrad-ocr image #:ocrad ocr) image))
+    (values (invoke-ocrad-ocr image
+                              #:ocrad ocr
+                              #:ocr-arguments ocr-arguments) image))
    ((string-contains ocr "tesseract")
-    (values (invoke-tesseract-ocr image #:tesseract ocr) image))
+    (values (invoke-tesseract-ocr image
+                                  #:tesseract ocr
+                                  #:ocr-arguments ocr-arguments) image))
    (else (error "unsupported ocr command"))))
 
 (define* (wait-for-screen-text marionette predicate
                                #:key
                                (ocr "ocrad")
+                               ocr-arguments
                                (timeout 30)
                                pre-action
                                post-action)
@@ -359,7 +372,10 @@ (define* (wait-for-screen-text marionette predicate
                  'ocr-text: last-text
                  'screendump: screendump-backup))
         (let* ((_ (and (procedure? pre-action) (pre-action)))
-               (text screendump (marionette-screen-text marionette #:ocr ocr))
+               (text screendump
+                     (marionette-screen-text marionette
+                                             #:ocr ocr
+                                             #:ocr-arguments ocr-arguments))
                (_ (and (procedure? post-action) (post-action)))
                (result (predicate text)))
           (cond (result
-- 
2.39.2

[bug#64356] [PATCH v2 3/4] tests: gdm: Prefer OCR to delay.

[Bruno Victal]

* gnu/tests/gdm.scm (run-gdm-test): Use wait-for-screen-text instead of sleep.
---
 gnu/tests/gdm.scm | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/gnu/tests/gdm.scm b/gnu/tests/gdm.scm
index ec1df4b797..77163bc3e0 100644
--- a/gnu/tests/gdm.scm
+++ b/gnu/tests/gdm.scm
@@ -19,6 +19,7 @@
 (define-module (gnu tests gdm)
   #:use-module (gnu tests)
   #:use-module (gnu packages freedesktop)
+  #:use-module (gnu packages ocr)
   #:use-module (gnu services)
   #:use-module (gnu services desktop)
   #:use-module (gnu services xorg)
@@ -57,6 +58,7 @@ (define* (run-gdm-test #:key wayland?)
       #~(begin
           (use-modules (gnu build marionette)
                        (ice-9 format)
+                       (srfi srfi-26)
                        (srfi srfi-64))
 
           (let ((marionette (make-marionette (list #$vm)))
@@ -73,11 +75,18 @@ (define* (run-gdm-test #:key wayland?)
                   (start-service 'xorg-server))
                marionette))
 
-            (test-assert "gdm ready"
-              (wait-for-file "/var/run/gdm/gdm.pid" marionette))
-
-            ;; waiting for gdm.pid is not enough, tests may still sporadically fail.
-            (sleep 1)
+            (test-group "gdm ready"
+              (test-assert "PID file present"
+                (wait-for-file "/var/run/gdm/gdm.pid" marionette))
+
+              ;; Waiting for gdm.pid is not enough, tests may still sporadically
+              ;; fail; ensure that the login screen is up.
+              ;; XXX: GNU Ocrad works but with '--invert' only.
+              (test-assert "login screen up"
+                (wait-for-screen-text marionette
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$(file-append ocrad "/bin/ocrad")
+                                      #:ocr-arguments "--invert")))
 
             (test-equal (string-append "session-type is " expected-session-type)
               expected-session-type
-- 
2.39.2

[bug#64356] [PATCH v2 4/4] tests: xvnc: Group up GDM test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Group up GDM test. Use GNU Ocrad.
---
 gnu/tests/vnc.scm | 58 ++++++++++++++++++++++++-----------------------
 1 file changed, 30 insertions(+), 28 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index e59972eee4..a2c3e3480e 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -104,6 +104,8 @@ (define (run-xvnc-test)
               (operating-system os)
               (memory-size 1024)))
 
+  (define ocr (file-append ocrad "/bin/ocrad"))
+
   (define test
     (with-imported-modules (source-module-closure
                             '((gnu build marionette)
@@ -152,43 +154,43 @@ (define (run-xvnc-test)
 | grep -Fq nothing"))
                marionette))
 
-            (test-assert "vnc lands on the gdm login screen"
+            (test-group "vnc lands on the gdm login screen"
               ;; This test runs vncviewer on the local VM and verifies that it
               ;; manages to access the GDM login screen (via XDMCP).
-              (begin
-                (define (ratpoison-abort)
-                  (marionette-control "sendkey ctrl-g" marionette))
-
-                (define (ratpoison-help)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "?" marionette)
-                  (sleep 1))            ;wait for help screen to appear
-
-                (define (ratpoison-exec command)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "!" marionette)
-                  (marionette-type (string-append command "\n") marionette))
-
-                ;; Wait until the ratpoison help screen can be displayed; this
-                ;; means the window manager is ready.
+              (define (ratpoison-abort)
+                (marionette-control "sendkey ctrl-g" marionette))
+
+              (define (ratpoison-help)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "?" marionette)
+                (sleep 1))            ;wait for help screen to appear
+
+              (define (ratpoison-exec command)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "!" marionette)
+                (marionette-type (string-append command "\n") marionette))
+
+              ;; Wait until the ratpoison help screen can be displayed; this
+              ;; means the window manager is ready.
+              ;; XXX: The letters are half of the height preferred by
+              ;; GNU Ocrad, scale it by 2.
+              (test-assert "window manager is ready"
                 (wait-for-screen-text marionette
                                       (cut string-contains <> "key bindings")
-                                      #:ocr #$(file-append tesseract-ocr
-                                                           "/bin/tesseract")
+                                      #:ocr #$ocr
+                                      #:ocr-arguments "--scale=2"
                                       #:pre-action ratpoison-help
-                                      #:post-action ratpoison-abort)
+                                      #:post-action ratpoison-abort))
 
                 ;; Run vncviewer and expect the GDM login screen (accessed via
                 ;; XDMCP).  This can take a while to appear on slower machines.
-                (ratpoison-exec "vncviewer localhost:5905")
-                ;; XXX: tesseract narrowly recognizes "Guix" as "uix" from the
-                ;; background image; ocrad fares worst.  Sadly, 'Username' is
-                ;; not recognized at all.
+              (ratpoison-exec "vncviewer localhost:5905")
+
+              (test-assert "GDM login screen ready"
                 (wait-for-screen-text marionette
-                                      (cut string-contains <> "uix")
-                                      #:ocr #$(file-append tesseract-ocr
-                                                           "/bin/tesseract")
-                                      #:timeout 120)))
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$ocr
+                                      #:ocr-arguments "--invert")))
 
             (test-end)))))
 
-- 
2.39.2

[bug#64356] [PATCH v3 0/4] Fix GDM + VNC tests

[Bruno Victal]

Notable changes since v2:

* The ocr-arguments are passed as a list of strings.

Bruno Victal (4):
  tests: xvnc: Fix test.
  marionette: Allow passing custom OCR arguments.
  tests: gdm: Prefer OCR to delay.
  tests: xvnc: Group up GDM test.

 gnu/build/marionette.scm | 32 ++++++++++++++------
 gnu/tests/gdm.scm        | 19 ++++++++----
 gnu/tests/vnc.scm        | 63 +++++++++++++++++++++-------------------
 3 files changed, 70 insertions(+), 44 deletions(-)


base-commit: 8af22b493199a17f46351c2f3d9f6ee759e48564
-- 
2.39.2

[bug#64356] [PATCH v3 1/4] tests: xvnc: Fix test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Use system* instead of invoke.
---
 gnu/tests/vnc.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index 5c4bd43fa3..e59972eee4 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -142,11 +142,11 @@ (define (run-xvnc-test)
               (marionette-eval
                '(begin
                   ;; Check that DCONF_PROFILE is set...
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 pgrep gdm | head -n1 | xargs -I{} grep -Fq DCONF_PROFILE /proc/{}/environ")
 
                   ;; ... and that
-                  (invoke "/bin/sh" "-lc" "\
+                  (system* "/bin/sh" "-lc" "\
 sudo -E -u gdm env DCONF_PROFILE=/etc/dconf/profile/gdm dbus-run-session \
 gsettings get org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type \
 | grep -Fq nothing"))

base-commit: 8af22b493199a17f46351c2f3d9f6ee759e48564
-- 
2.39.2

[bug#64356] [PATCH v3 2/4] marionette: Allow passing custom OCR arguments.

[Bruno Victal]

* gnu/build/marionette.scm (invoke-ocrad-ocr, invoke-tesseract-ocr)
(marionette-screen-text): New 'ocr-arguments' argument.
---
 gnu/build/marionette.scm | 32 +++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)

diff --git a/gnu/build/marionette.scm b/gnu/build/marionette.scm
index b8fba61d06..df69d6d17e 100644
--- a/gnu/build/marionette.scm
+++ b/gnu/build/marionette.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2023 Bruno Victal <mirai@makinata.eu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -287,23 +288,27 @@ (define (marionette-control command marionette)
      ;; The "quit" command terminates QEMU immediately, with no output.
      (unless (string=? command "quit") (wait-for-monitor-prompt monitor)))))
 
-(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad"))
+(define* (invoke-ocrad-ocr image #:key (ocrad "ocrad")
+                           (ocr-arguments '("--invert" "--scale=10")))
   "Invoke the OCRAD command on image, and return the recognized text."
-  (let* ((pipe (open-pipe* OPEN_READ ocrad "-i" "-s" "10" image))
+  (let* ((command (string-join `(,ocrad ,@ocr-arguments ,image)))
+         (pipe (open-input-pipe command))
          (text (get-string-all pipe)))
     (unless (zero? (close-pipe pipe))
       (error "'ocrad' failed" ocrad))
     text))
 
-(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
+(define* (invoke-tesseract-ocr image #:key (tesseract "tesseract")
+                               (ocr-arguments '()))
   "Invoke the TESSERACT command on IMAGE, and return the recognized text."
   (let* ((output-basename (tmpnam))
-         (output-basename* (string-append output-basename ".txt")))
+         (output-basename* (string-append output-basename ".txt"))
+         (arguments (cons* image output-basename ocr-arguments)))
     (dynamic-wind
       (const #t)
       (lambda ()
         (let ((exit-val (status:exit-val
-                         (system* tesseract image output-basename))))
+                         (apply system* tesseract arguments))))
           (unless (zero? exit-val)
             (error "'tesseract' failed" tesseract))
           (call-with-input-file output-basename* get-string-all)))
@@ -311,7 +316,8 @@ (define* (invoke-tesseract-ocr image #:key (tesseract "tesseract"))
         (false-if-exception (delete-file output-basename))
         (false-if-exception (delete-file output-basename*))))))
 
-(define* (marionette-screen-text marionette #:key (ocr "ocrad"))
+(define* (marionette-screen-text marionette #:key (ocr "ocrad")
+                                 ocr-arguments)
   "Take a screenshot of MARIONETTE, perform optical character
 recognition (OCR), and return the text read from the screen as a string, along
 the screen dump image used.  Do this by invoking OCR, which should be the file
@@ -324,14 +330,19 @@ (define* (marionette-screen-text marionette #:key (ocr "ocrad"))
   ;; Process it via the OCR.
   (cond
    ((string-contains ocr "ocrad")
-    (values (invoke-ocrad-ocr image #:ocrad ocr) image))
+    (values (invoke-ocrad-ocr image
+                              #:ocrad ocr
+                              #:ocr-arguments ocr-arguments) image))
    ((string-contains ocr "tesseract")
-    (values (invoke-tesseract-ocr image #:tesseract ocr) image))
+    (values (invoke-tesseract-ocr image
+                                  #:tesseract ocr
+                                  #:ocr-arguments ocr-arguments) image))
    (else (error "unsupported ocr command"))))
 
 (define* (wait-for-screen-text marionette predicate
                                #:key
                                (ocr "ocrad")
+                               ocr-arguments
                                (timeout 30)
                                pre-action
                                post-action)
@@ -359,7 +370,10 @@ (define* (wait-for-screen-text marionette predicate
                  'ocr-text: last-text
                  'screendump: screendump-backup))
         (let* ((_ (and (procedure? pre-action) (pre-action)))
-               (text screendump (marionette-screen-text marionette #:ocr ocr))
+               (text screendump
+                     (marionette-screen-text marionette
+                                             #:ocr ocr
+                                             #:ocr-arguments ocr-arguments))
                (_ (and (procedure? post-action) (post-action)))
                (result (predicate text)))
           (cond (result
-- 
2.39.2

[bug#64356] [PATCH v3 3/4] tests: gdm: Prefer OCR to delay.

[Bruno Victal]

* gnu/tests/gdm.scm (run-gdm-test): Use wait-for-screen-text instead of sleep.
---
 gnu/tests/gdm.scm | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/gnu/tests/gdm.scm b/gnu/tests/gdm.scm
index ec1df4b797..06177d4080 100644
--- a/gnu/tests/gdm.scm
+++ b/gnu/tests/gdm.scm
@@ -19,6 +19,7 @@
 (define-module (gnu tests gdm)
   #:use-module (gnu tests)
   #:use-module (gnu packages freedesktop)
+  #:use-module (gnu packages ocr)
   #:use-module (gnu services)
   #:use-module (gnu services desktop)
   #:use-module (gnu services xorg)
@@ -57,6 +58,7 @@ (define* (run-gdm-test #:key wayland?)
       #~(begin
           (use-modules (gnu build marionette)
                        (ice-9 format)
+                       (srfi srfi-26)
                        (srfi srfi-64))
 
           (let ((marionette (make-marionette (list #$vm)))
@@ -73,11 +75,18 @@ (define* (run-gdm-test #:key wayland?)
                   (start-service 'xorg-server))
                marionette))
 
-            (test-assert "gdm ready"
-              (wait-for-file "/var/run/gdm/gdm.pid" marionette))
-
-            ;; waiting for gdm.pid is not enough, tests may still sporadically fail.
-            (sleep 1)
+            (test-group "gdm ready"
+              (test-assert "PID file present"
+                (wait-for-file "/var/run/gdm/gdm.pid" marionette))
+
+              ;; Waiting for gdm.pid is not enough, tests may still sporadically
+              ;; fail; ensure that the login screen is up.
+              ;; XXX: GNU Ocrad works but with '--invert' only.
+              (test-assert "login screen up"
+                (wait-for-screen-text marionette
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$(file-append ocrad "/bin/ocrad")
+                                      #:ocr-arguments '("--invert"))))
 
             (test-equal (string-append "session-type is " expected-session-type)
               expected-session-type
-- 
2.39.2

[bug#64356] [PATCH v3 4/4] tests: xvnc: Group up GDM test.

[Bruno Victal]

* gnu/tests/vnc.scm (run-xvnc-test): Group up GDM test. Use GNU Ocrad.
---
 gnu/tests/vnc.scm | 59 +++++++++++++++++++++++++----------------------
 1 file changed, 31 insertions(+), 28 deletions(-)

diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
index e59972eee4..40f8348db9 100644
--- a/gnu/tests/vnc.scm
+++ b/gnu/tests/vnc.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com>.
+;;; Copyright © 2023 Bruno Victal <mirai@makinata.eu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -104,6 +105,8 @@ (define (run-xvnc-test)
               (operating-system os)
               (memory-size 1024)))
 
+  (define ocr (file-append ocrad "/bin/ocrad"))
+
   (define test
     (with-imported-modules (source-module-closure
                             '((gnu build marionette)
@@ -152,43 +155,43 @@ (define (run-xvnc-test)
 | grep -Fq nothing"))
                marionette))
 
-            (test-assert "vnc lands on the gdm login screen"
+            (test-group "vnc lands on the gdm login screen"
               ;; This test runs vncviewer on the local VM and verifies that it
               ;; manages to access the GDM login screen (via XDMCP).
-              (begin
-                (define (ratpoison-abort)
-                  (marionette-control "sendkey ctrl-g" marionette))
-
-                (define (ratpoison-help)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "?" marionette)
-                  (sleep 1))            ;wait for help screen to appear
-
-                (define (ratpoison-exec command)
-                  (marionette-control "sendkey ctrl-t" marionette)
-                  (marionette-type "!" marionette)
-                  (marionette-type (string-append command "\n") marionette))
-
-                ;; Wait until the ratpoison help screen can be displayed; this
-                ;; means the window manager is ready.
+              (define (ratpoison-abort)
+                (marionette-control "sendkey ctrl-g" marionette))
+
+              (define (ratpoison-help)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "?" marionette)
+                (sleep 1))            ;wait for help screen to appear
+
+              (define (ratpoison-exec command)
+                (marionette-control "sendkey ctrl-t" marionette)
+                (marionette-type "!" marionette)
+                (marionette-type (string-append command "\n") marionette))
+
+              ;; Wait until the ratpoison help screen can be displayed; this
+              ;; means the window manager is ready.
+              ;; XXX: The letters are half of the height preferred by
+              ;; GNU Ocrad, scale it by 2.
+              (test-assert "window manager is ready"
                 (wait-for-screen-text marionette
                                       (cut string-contains <> "key bindings")
-                                      #:ocr #$(file-append tesseract-ocr
-                                                           "/bin/tesseract")
+                                      #:ocr #$ocr
+                                      #:ocr-arguments '("--scale=2")
                                       #:pre-action ratpoison-help
-                                      #:post-action ratpoison-abort)
+                                      #:post-action ratpoison-abort))
 
                 ;; Run vncviewer and expect the GDM login screen (accessed via
                 ;; XDMCP).  This can take a while to appear on slower machines.
-                (ratpoison-exec "vncviewer localhost:5905")
-                ;; XXX: tesseract narrowly recognizes "Guix" as "uix" from the
-                ;; background image; ocrad fares worst.  Sadly, 'Username' is
-                ;; not recognized at all.
+              (ratpoison-exec "vncviewer localhost:5905")
+
+              (test-assert "GDM login screen ready"
                 (wait-for-screen-text marionette
-                                      (cut string-contains <> "uix")
-                                      #:ocr #$(file-append tesseract-ocr
-                                                           "/bin/tesseract")
-                                      #:timeout 120)))
+                                      (cut string-contains <> "Guix")
+                                      #:ocr #$ocr
+                                      #:ocr-arguments '("--invert"))))
 
             (test-end)))))
 
-- 
2.39.2

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi Bruno!

Bruno Victal <mirai@makinata.eu> writes:

> It turns out that the issue with the xvnc test was due to using invoke
> instead of system*.
> Along the way, the GDM test was refactored to replace the ugly sleep
> delay with a more adequate awaiting method.
>
> Bruno Victal (4):
>   tests: xvnc: Fix test.
>   marionette: Allow passing custom OCR arguments.
>   tests: gdm: Prefer OCR to delay.
>   tests: xvnc: Group up GDM test.
>
>  gnu/build/marionette.scm | 34 +++++++++++++++++-------
>  gnu/tests/gdm.scm        | 19 ++++++++++----
>  gnu/tests/vnc.scm        | 56 ++++++++++++++++++++--------------------
>  3 files changed, 67 insertions(+), 42 deletions(-)

So the xvnc service is not broken per se?  I thought I had a black
screen on GDM, when using the (xdmcp? #t) to login via the GDM screen.
Maybe just a fluke in the midst of upgrades?  I'll double check, and if
there's indeed no issue, look deeper into this promising series!

-- 
Thanks,
Maxim

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

> * gnu/tests/vnc.scm (run-xvnc-test): Use system* instead of invoke.
> ---
>  gnu/tests/vnc.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
> index 5c4bd43fa3..e59972eee4 100644
> --- a/gnu/tests/vnc.scm
> +++ b/gnu/tests/vnc.scm
> @@ -142,11 +142,11 @@ (define (run-xvnc-test)
>                (marionette-eval
>                 '(begin
>                    ;; Check that DCONF_PROFILE is set...
> -                  (invoke "/bin/sh" "-lc" "\
> +                  (system* "/bin/sh" "-lc" "\
>  pgrep gdm | head -n1 | xargs -I{} grep -Fq DCONF_PROFILE /proc/{}/environ")
>  
>                    ;; ... and that
> -                  (invoke "/bin/sh" "-lc" "\
> +                  (system* "/bin/sh" "-lc" "\
>  sudo -E -u gdm env DCONF_PROFILE=/etc/dconf/profile/gdm dbus-run-session \
>  gsettings get org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type \
>  | grep -Fq nothing"))

Using system* instead of invoke would silence the errors as it doesn't
raise on errors.

The real cause of the problem was a09c7da ("tests: Fork and exec a new
Guile for the marionette REPL."), which I've fixed in 1edbadc6 ("tests:
xvnc: Fix 'gdm auto-suspend is disabled' test.").

Thanks for the report!

-- 
Thanks,
Maxim

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

> * gnu/tests/gdm.scm (run-gdm-test): Use wait-for-screen-text instead of sleep.
> ---
>  gnu/tests/gdm.scm | 19 ++++++++++++++-----
>  1 file changed, 14 insertions(+), 5 deletions(-)
>
> diff --git a/gnu/tests/gdm.scm b/gnu/tests/gdm.scm
> index ec1df4b797..06177d4080 100644
> --- a/gnu/tests/gdm.scm
> +++ b/gnu/tests/gdm.scm
> @@ -19,6 +19,7 @@
>  (define-module (gnu tests gdm)
>    #:use-module (gnu tests)
>    #:use-module (gnu packages freedesktop)
> +  #:use-module (gnu packages ocr)
>    #:use-module (gnu services)
>    #:use-module (gnu services desktop)
>    #:use-module (gnu services xorg)
> @@ -57,6 +58,7 @@ (define* (run-gdm-test #:key wayland?)
>        #~(begin
>            (use-modules (gnu build marionette)
>                         (ice-9 format)
> +                       (srfi srfi-26)
>                         (srfi srfi-64))
>  
>            (let ((marionette (make-marionette (list #$vm)))
> @@ -73,11 +75,18 @@ (define* (run-gdm-test #:key wayland?)
>                    (start-service 'xorg-server))
>                 marionette))
>  
> -            (test-assert "gdm ready"
> -              (wait-for-file "/var/run/gdm/gdm.pid" marionette))
> -
> -            ;; waiting for gdm.pid is not enough, tests may still sporadically fail.
> -            (sleep 1)
> +            (test-group "gdm ready"
> +              (test-assert "PID file present"
> +                (wait-for-file "/var/run/gdm/gdm.pid" marionette))
> +
> +              ;; Waiting for gdm.pid is not enough, tests may still sporadically
> +              ;; fail; ensure that the login screen is up.
> +              ;; XXX: GNU Ocrad works but with '--invert' only.
> +              (test-assert "login screen up"
> +                (wait-for-screen-text marionette
> +                                      (cut string-contains <> "Guix")
> +                                      #:ocr #$(file-append ocrad "/bin/ocrad")
> +                                      #:ocr-arguments '("--invert"))))

I modified it to use #:timeout 120, to accommodate older, slow systems
like mine.

-- 
Thanks,
Maxim

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

> * gnu/tests/vnc.scm (run-xvnc-test): Group up GDM test. Use GNU Ocrad.
> ---
>  gnu/tests/vnc.scm | 59 +++++++++++++++++++++++++----------------------
>  1 file changed, 31 insertions(+), 28 deletions(-)
>
> diff --git a/gnu/tests/vnc.scm b/gnu/tests/vnc.scm
> index e59972eee4..40f8348db9 100644
> --- a/gnu/tests/vnc.scm
> +++ b/gnu/tests/vnc.scm
> @@ -1,5 +1,6 @@

[...]

>  
>                  ;; Run vncviewer and expect the GDM login screen (accessed via
>                  ;; XDMCP).  This can take a while to appear on slower machines.
> -                (ratpoison-exec "vncviewer localhost:5905")
> -                ;; XXX: tesseract narrowly recognizes "Guix" as "uix" from the
> -                ;; background image; ocrad fares worst.  Sadly, 'Username' is
> -                ;; not recognized at all.

I've kept most of the comment; otherwise it'd be hard to understand for
a newcomer that Guix is matched from the background (e.g. when the
background changes and breaks this assumption...).

> +              (ratpoison-exec "vncviewer localhost:5905")
> +
> +              (test-assert "GDM login screen ready"
>                  (wait-for-screen-text marionette
> -                                      (cut string-contains <> "uix")
> -                                      #:ocr #$(file-append tesseract-ocr
> -                                                           "/bin/tesseract")
> -                                      #:timeout 120)))
> +                                      (cut string-contains <> "Guix")
> +                                      #:ocr #$ocr
> +                                      #:ocr-arguments '("--invert"))))

I've preserved the timeout, which is important for slow systems (I added
a comment).

-- 
Thanks,
Maxim

[bug#64356] [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

[...]

> -(define* (marionette-screen-text marionette #:key (ocr "ocrad"))
> +(define* (marionette-screen-text marionette #:key (ocr "ocrad")
> +                                 ocr-arguments)
>    "Take a screenshot of MARIONETTE, perform optical character
>  recognition (OCR), and return the text read from the screen as a string, along
>  the screen dump image used.  Do this by invoking OCR, which should be the file
> @@ -324,14 +330,19 @@ (define* (marionette-screen-text marionette #:key (ocr "ocrad"))
>    ;; Process it via the OCR.
>    (cond
>     ((string-contains ocr "ocrad")
> -    (values (invoke-ocrad-ocr image #:ocrad ocr) image))
> +    (values (invoke-ocrad-ocr image
> +                              #:ocrad ocr
> +                              #:ocr-arguments ocr-arguments) image))
>     ((string-contains ocr "tesseract")
> -    (values (invoke-tesseract-ocr image #:tesseract ocr) image))
> +    (values (invoke-tesseract-ocr image
> +                                  #:tesseract ocr
> +                                  #:ocr-arguments ocr-arguments) image))
>     (else (error "unsupported ocr command"))))
>  
>  (define* (wait-for-screen-text marionette predicate
>                                 #:key
>                                 (ocr "ocrad")
> +                               ocr-arguments

Care must be taken here to avoid overwriting the default arguments of
the 'invoke-ocrad-ocr' and 'invoke-tesseract-ocr' procedures.  I've
handled this by extracting a %default-ocrad-arguments and or'ing the
argument here with it in this procedure, like:

--8<---------------cut here---------------start------------->8---
@@ -324,14 +334,22 @@ (define* (marionette-screen-text marionette #:key (ocr "ocrad"))
   ;; Process it via the OCR.
   (cond
    ((string-contains ocr "ocrad")
-    (values (invoke-ocrad-ocr image #:ocrad ocr) image))
+    (values (invoke-ocrad-ocr image
+                              #:ocrad ocr
+                              #:ocr-arguments
+                              (or ocr-arguments %default-ocrad-arguments))
+            image))
    ((string-contains ocr "tesseract")
-    (values (invoke-tesseract-ocr image #:tesseract ocr) image))
+    (values (invoke-tesseract-ocr image
+                                  #:tesseract ocr
+                                  #:ocr-arguments (or ocr-arguments '()))
+            image))
    (else (error "unsupported ocr command"))))
 
 (define* (wait-for-screen-text marionette predicate
                                #:key
                                (ocr "ocrad")
--8<---------------cut here---------------end--------------->8---

Otherwise, LGTM.

-- 
Thanks,
Maxim

bug#64356: [PATCH 0/4] Fix GDM and VNC tests

[Maxim Cournoyer]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Bruno!
>
> Bruno Victal <mirai@makinata.eu> writes:
>
>> It turns out that the issue with the xvnc test was due to using invoke
>> instead of system*.
>> Along the way, the GDM test was refactored to replace the ugly sleep
>> delay with a more adequate awaiting method.
>>
>> Bruno Victal (4):
>>   tests: xvnc: Fix test.
>>   marionette: Allow passing custom OCR arguments.
>>   tests: gdm: Prefer OCR to delay.
>>   tests: xvnc: Group up GDM test.

Applied, with some modifications mentioned in the thread.

Thanks a lot!

-- 
Maxim