all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
@ 2022-10-07 20:47 Maxime Devos
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:47 UTC (permalink / raw)
  To: 58365


[-- Attachment #1.1.1: Type: text/plain, Size: 1589 bytes --]

Fixes: <https://issues.guix.gnu.org/58337>

guile-build-system can't guess where the tests are located, so currently 
it's opt-in, via #:test-script.  After this patch series, it is 
currently only used by guile-ffi-fftw, but other guile-build-system 
packages that use SRFI-64 could potentially use it too (e.g. 
guile-srfi-117, which is not packages or even released yet).

* First commit: package a test driver, for nicer output and for proper 
exit status.  I copied test-driver.scm to aux-files such that 
test-driver.scm can be changed without worrying about rebuilds.  OTOH, 
it isn't modified that often, so that could be left out.

Also, copying test-driver.scm causes the following:

guix build: warning: failed to load '(gnu packages aux-files test-driver)':
no code for module (gnu packages aux-files test-driver)
hint: File 
`/home/antipode/source-code/guix-master/gnu/packages/aux-files/test-driver.scm' 
should probably start with:

      (define-module (gnu packages aux-files test-driver))

Whatever code does this loading, should probably be modified to not load 
things under aux-files.

Going by "git log", the upstream of this test driver is Guix itself.

* Second commit: add support for #:tests?, using the test driver.
* Third commit: use it in guile-ffi-fftw.
* Fourth and fifth commit: cleanup of guile-ffi-fftw.
* Sixth commit: adjust guile-ac-d-bus to new 'check' phase

Only the initial commit is required for adding support for tests.

I'll leave building dependencies to <https://qa.guix.gnu.org/>.

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver.
  2022-10-07 20:47 [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Maxime Devos
@ 2022-10-07 20:53 ` Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 2/6] build-system/guile: Run SRFI-64 tests Maxime Devos
                     ` (5 more replies)
  0 siblings, 6 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

A copy is made of test-driver.scm to avoid potentially surprising rebuilds
when changes are made.

* gnu/packages/guile.scm (guile-test-driver): New variable.
* gnu/packages/aux-files/test-driver.scm: New file.
* Makefile.am (AUX_FILES): Register it.
* build-aux/test-driver.scm: Add a note.
---
 Makefile.am                            |   3 +-
 build-aux/test-driver.scm              |   2 +
 gnu/packages/aux-files/test-driver.scm | 284 +++++++++++++++++++++++++
 gnu/packages/guile.scm                 |  39 +++-
 4 files changed, 326 insertions(+), 2 deletions(-)
 create mode 100755 gnu/packages/aux-files/test-driver.scm

diff --git a/Makefile.am b/Makefile.am
index bfabf0bf2e..e1f1a4573e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -427,7 +427,8 @@ AUX_FILES =						\
   gnu/packages/aux-files/python/sanity-check.py		\
   gnu/packages/aux-files/python/sitecustomize.py	\
   gnu/packages/aux-files/renpy/renpy.in	\
-  gnu/packages/aux-files/run-in-namespace.c
+  gnu/packages/aux-files/run-in-namespace.c \
+  gnu/packages/aux-files/test-driver.scm
 
 # Templates, examples.
 EXAMPLES =					\
diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
index 1cdd4ff8f7..7ff8d45031 100755
--- a/build-aux/test-driver.scm
+++ b/build-aux/test-driver.scm
@@ -2,6 +2,8 @@
 exec guile --no-auto-compile -e main -s "$0" "$@"
 !#
 ;;;; test-driver.scm - Guile test driver for Automake testsuite harness
+;;;; When update this code, consider updating
+;;;; gnu/packages/aux-files/test-driver.scm as well.
 
 (define script-version "2021-02-02.05") ;UTC
 
diff --git a/gnu/packages/aux-files/test-driver.scm b/gnu/packages/aux-files/test-driver.scm
new file mode 100755
index 0000000000..1cdd4ff8f7
--- /dev/null
+++ b/gnu/packages/aux-files/test-driver.scm
@@ -0,0 +1,284 @@
+#!/bin/sh
+exec guile --no-auto-compile -e main -s "$0" "$@"
+!#
+;;;; test-driver.scm - Guile test driver for Automake testsuite harness
+
+(define script-version "2021-02-02.05") ;UTC
+
+;;; Copyright © 2015, 2016 Mathieu Lirzin <mthl@gnu.org>
+;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; This script provides a Guile test driver using the SRFI-64 Scheme API for
+;;; test suites.  SRFI-64 is distributed with Guile since version 2.0.9.
+;;;
+;;;; Code:
+
+(use-modules (ice-9 format)
+             (ice-9 getopt-long)
+             (ice-9 pretty-print)
+             (ice-9 regex)
+             (srfi srfi-1)
+             (srfi srfi-19)
+             (srfi srfi-26)
+             (srfi srfi-64))
+
+(define (show-help)
+  (display "Usage:
+   test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+               [--expect-failure={yes|no}] [--color-tests={yes|no}]
+               [--select=REGEXP] [--exclude=REGEXP] [--errors-only={yes|no}]
+               [--enable-hard-errors={yes|no}] [--brief={yes|no}}]
+               [--show-duration={yes|no}] [--]
+               TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
+The '--test-name' option is mandatory.  The '--select' and '--exclude' options
+allow selecting or excluding individual test cases via a regexp, respectively.
+The '--errors-only' option can be set to \"yes\" to limit the logged test case
+metadata to only those test cases that failed.  When set to \"yes\", the
+'--brief' option disables printing the individual test case result to the
+console.  When '--show-duration' is set to \"yes\", the time elapsed per test
+case is shown.\n"))
+
+(define %options
+  '((test-name                 (value #t))
+    (log-file                  (value #t))
+    (trs-file                  (value #t))
+    (select                    (value #t))
+    (exclude                   (value #t))
+    (errors-only               (value #t))
+    (color-tests               (value #t))
+    (expect-failure            (value #t)) ;XXX: not implemented yet
+    (enable-hard-errors        (value #t)) ;not implemented in SRFI-64
+    (brief                     (value #t))
+    (show-duration             (value #t))
+    (help    (single-char #\h) (value #f))
+    (version (single-char #\V) (value #f))))
+
+(define (option->boolean options key)
+  "Return #t if the value associated with KEY in OPTIONS is \"yes\"."
+  (and=> (option-ref options key #f) (cut string=? <> "yes")))
+
+(define* (test-display field value  #:optional (port (current-output-port))
+                       #:key pretty?)
+  "Display \"FIELD: VALUE\\n\" on PORT."
+  (if pretty?
+      (begin
+        (format port "~A:~%" field)
+        (pretty-print value port #:per-line-prefix "+ "))
+      (format port "~A: ~S~%" field value)))
+
+(define* (result->string symbol #:key colorize?)
+  "Return SYMBOL as an upper case string.  Use colors when COLORIZE is #t."
+  (let ((result (string-upcase (symbol->string symbol))))
+    (if colorize?
+        (string-append (case symbol
+                         ((pass)       "^[[0;32m")  ;green
+                         ((xfail)      "^[[1;32m")  ;light green
+                         ((skip)       "^[[1;34m")  ;blue
+                         ((fail xpass) "^[[0;31m")  ;red
+                         ((error)      "^[[0;35m")) ;magenta
+                       result
+                       "^[[m")          ;no color
+        result)))
+
+\f
+;;;
+;;; SRFI 64 custom test runner.
+;;;
+
+(define* (test-runner-gnu test-name #:key color? brief? errors-only?
+                          show-duration?
+                          (out-port (current-output-port))
+                          (trs-port (%make-void-port "w"))
+                          select exclude)
+  "Return an custom SRFI-64 test runner.  TEST-NAME is a string specifying the
+file name of the current the test.  COLOR? specifies whether to use colors.
+When BRIEF? is true, the individual test cases results are masked and only the
+summary is shown.  ERRORS-ONLY? reduces the amount of test case metadata
+logged to only that of the failed test cases.  OUT-PORT and TRS-PORT must be
+output ports.  OUT-PORT defaults to the current output port, while TRS-PORT
+defaults to a void port, which means no TRS output is logged.  SELECT and
+EXCLUDE may take a regular expression to select or exclude individual test
+cases based on their names."
+
+  (define test-cases-start-time (make-hash-table))
+
+  (define (test-on-test-begin-gnu runner)
+    ;; Procedure called at the start of an individual test case, before the
+    ;; test expression (and expected value) are evaluated.
+    (let ((test-case-name (test-runner-test-name runner))
+          (start-time     (current-time time-monotonic)))
+      (hash-set! test-cases-start-time test-case-name start-time)))
+
+  (define (test-skipped? runner)
+    (eq? 'skip (test-result-kind runner)))
+
+  (define (test-failed? runner)
+    (not (or (test-passed? runner)
+             (test-skipped? runner))))
+
+  (define (test-on-test-end-gnu runner)
+    ;; Procedure called at the end of an individual test case, when the result
+    ;; of the test is available.
+    (let* ((results (test-result-alist runner))
+           (result? (cut assq <> results))
+           (result  (cut assq-ref results <>))
+           (test-case-name (test-runner-test-name runner))
+           (start (hash-ref test-cases-start-time test-case-name))
+           (end (current-time time-monotonic))
+           (time-elapsed (time-difference end start))
+           (time-elapsed-seconds (+ (time-second time-elapsed)
+                                    (* 1e-9 (time-nanosecond time-elapsed)))))
+      (unless (or brief? (and errors-only? (test-skipped? runner)))
+        ;; Display the result of each test case on the console.
+        (format out-port "~a: ~a - ~a ~@[[~,3fs]~]~%"
+                (result->string (test-result-kind runner) #:colorize? color?)
+                test-name test-case-name
+                (and show-duration? time-elapsed-seconds)))
+
+      (unless (and errors-only? (not (test-failed? runner)))
+        (format #t "test-name: ~A~%" (result 'test-name))
+        (format #t "location: ~A~%"
+                (string-append (result 'source-file) ":"
+                               (number->string (result 'source-line))))
+        (test-display "source" (result 'source-form) #:pretty? #t)
+        (when (result? 'expected-value)
+          (test-display "expected-value" (result 'expected-value)))
+        (when (result? 'expected-error)
+          (test-display "expected-error" (result 'expected-error) #:pretty? #t))
+        (when (result? 'actual-value)
+          (test-display "actual-value" (result 'actual-value)))
+        (when (result? 'actual-error)
+          (test-display "actual-error" (result 'actual-error) #:pretty? #t))
+        (format #t "result: ~a~%" (result->string (result 'result-kind)))
+        (newline))
+
+      (format trs-port ":test-result: ~A ~A [~,3fs]~%"
+              (result->string (test-result-kind runner))
+              (test-runner-test-name runner) time-elapsed-seconds)))
+
+  (define (test-on-group-end-gnu runner)
+    ;; Procedure called by a 'test-end', including at the end of a test-group.
+    (let ((fail (or (positive? (test-runner-fail-count runner))
+                    (positive? (test-runner-xpass-count runner))))
+          (skip (or (positive? (test-runner-skip-count runner))
+                    (positive? (test-runner-xfail-count runner)))))
+      ;; XXX: The global results need some refinements for XPASS.
+      (format trs-port ":global-test-result: ~A~%"
+              (if fail "FAIL" (if skip "SKIP" "PASS")))
+      (format trs-port ":recheck: ~A~%"
+              (if fail "yes" "no"))
+      (format trs-port ":copy-in-global-log: ~A~%"
+              (if (or fail skip) "yes" "no"))
+      (when brief?
+        ;; Display the global test group result on the console.
+        (format out-port "~A: ~A~%"
+                (result->string (if fail 'fail (if skip 'skip 'pass))
+                                #:colorize? color?)
+                test-name))
+      #f))
+
+  (let ((runner (test-runner-null)))
+    (test-runner-on-test-begin! runner test-on-test-begin-gnu)
+    (test-runner-on-test-end! runner test-on-test-end-gnu)
+    (test-runner-on-group-end! runner test-on-group-end-gnu)
+    (test-runner-on-bad-end-name! runner test-on-bad-end-name-simple)
+    runner))
+
+\f
+;;;
+;;; SRFI 64 test specifiers.
+;;;
+(define (test-match-name* regexp)
+  "Return a test specifier that matches a test name against REGEXP."
+  (lambda (runner)
+    (string-match regexp (test-runner-test-name runner))))
+
+(define (test-match-name*/negated regexp)
+  "Return a negated test specifier version of test-match-name*."
+  (lambda (runner)
+    (not (string-match regexp (test-runner-test-name runner)))))
+
+;;; XXX: test-match-all is a syntax, which isn't convenient to use with a list
+;;; of test specifiers computed at run time.  Copy this SRFI 64 internal
+;;; definition here, which is the procedural equivalent of 'test-match-all'.
+(define (%test-match-all . pred-list)
+  (lambda (runner)
+    (let ((result #t))
+      (let loop ((l pred-list))
+	(if (null? l)
+	    result
+	    (begin
+	      (if (not ((car l) runner))
+		  (set! result #f))
+	      (loop (cdr l))))))))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (main . args)
+  (let* ((opts   (getopt-long (command-line) %options))
+         (option (cut option-ref opts <> <>)))
+    (cond
+     ((option 'help #f)    (show-help))
+     ((option 'version #f) (format #t "test-driver.scm ~A~%" script-version))
+     (else
+      (let* ((log (and=> (option 'log-file #f) (cut open-file <> "w0")))
+             (trs (and=> (option 'trs-file #f) (cut open-file <> "wl")))
+             (out (duplicate-port (current-output-port) "wl"))
+             (test-name (option 'test-name #f))
+             (select (option 'select #f))
+             (exclude (option 'exclude #f))
+             (test-specifiers (filter-map
+                               identity
+                               (list (and=> select test-match-name*)
+                                     (and=> exclude test-match-name*/negated))))
+             (test-specifier (apply %test-match-all test-specifiers))
+             (color-tests (if (assoc 'color-tests opts)
+                              (option->boolean opts 'color-tests)
+                              #t)))
+        (when log
+          (redirect-port log (current-output-port))
+          (redirect-port log (current-warning-port))
+          (redirect-port log (current-error-port)))
+        (test-with-runner
+            (test-runner-gnu test-name
+                             #:color? color-tests
+                             #:brief? (option->boolean opts 'brief)
+                             #:errors-only? (option->boolean opts 'errors-only)
+                             #:show-duration? (option->boolean
+                                               opts 'show-duration)
+                             #:out-port out #:trs-port trs)
+          (test-apply test-specifier
+                      (lambda _
+                        (load-from-path test-name))))
+        (and=> log close-port)
+        (and=> trs close-port)
+        (close-port out))))
+    (exit 0)))
+
+;;; Local Variables:
+;;; eval: (add-hook 'write-file-functions 'time-stamp)
+;;; time-stamp-start: "(define script-version \""
+;;; time-stamp-format: "%:y-%02m-%02d.%02H"
+;;; time-stamp-time-zone: "UTC"
+;;; time-stamp-end: "\") ;UTC"
+;;; End:
+
+;;;; test-driver.scm ends here.
diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index fcdf75051c..b847ee6be4 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -16,7 +16,7 @@
 ;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2019 Taylan Kammer <taylan.kammer@gmail.com>
 ;;; Copyright © 2020, 2021, 2022 Efraim Flashner <efraim@flashner.co.il>
-;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
+;;; Copyright © 2021, 2022 Maxime Devos <maximedevos@telenet.be>
 ;;; Copyright © 2021 Timothy Sample <samplet@ngyro.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -60,9 +60,11 @@ (define-module (gnu packages guile)
   #:use-module (gnu packages version-control)
   #:use-module (guix packages)
   #:use-module (guix download)
+  #:use-module (guix gexp)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system guile)
+  #:use-module (guix build-system trivial)
   #:use-module (guix deprecation)
   #:use-module (guix utils))
 
@@ -961,4 +963,39 @@ (define-public guile-lzma
 libraries, like Guile-zlib.")
     (license license:gpl3+)))
 
+(define-public guile-test-driver
+  (package
+    (name "guile-test-driver")
+    ;; (define script-version ...) in the source code
+    (version "2021-01-02.05")
+    ;; 'search-auxiliary-file' could be used here, but that causes warnings.
+    (source (local-file "../../gnu/packages/aux-files/test-driver.scm"))
+    (build-system gnu-build-system)
+    (inputs (list guile-3.0))
+    (arguments
+     (list #:phases
+           #~(modify-phases %standard-phases
+               (delete 'configure)
+               (delete 'check)
+               (delete 'install) ; no point in separating build and install
+               (replace 'build
+                 (lambda _
+                   (define destination
+                     (string-append #$output "/bin/test-driver.scm"))
+                   (mkdir-p (dirname destination))
+                   (copy-file #$source destination)
+                   (substitute* destination
+                     (("/bin/sh")
+                      ;; Reference to Guile will be patched by patch-shebangs.
+                      "/bin/guile \\")
+                     (("^exec guile(.*)$") "--no-auto-compile -e main -s\n"))
+                   (chmod destination #o500))))))
+    (home-page "https://www.gnu.org/software/guix")
+    (synopsis "Guile test driver for SRFI-64 test suites")
+    (description "This package, also known as @file{test-driver.scm}, provides
+a Guile test driver using the SRFI-64 Scheme API for test suites.  Unlike the
+default test runner, its output is consistent with other test drivers used
+by Automake.")
+    (license license:gpl3+)))
+
 ;;; guile.scm ends here

base-commit: 31a56967e2869c916b7a5e8ee570e8e10f0210a5
prerequisite-patch-id: 2712efb97bf33985fd0658e4dd8e936dc08be5fe
prerequisite-patch-id: 9d2409b480a8bff0fef029b4b095922d4957e06f
prerequisite-patch-id: 51a32abca3efec1ba67ead59b8694c5ea3129ad3
prerequisite-patch-id: 9092927761a340c07a99f5f3ed314a6add04cdee
prerequisite-patch-id: d0af09fbd5ee0ef60bdee53b87d729e46c1db2ca
prerequisite-patch-id: 4fee177b2d8c9478c6a7b8ce1ca9072942f39863
prerequisite-patch-id: c2b101598fa5b6f93470ae41d51a983dcb931b04
prerequisite-patch-id: 8fe65a852a4463203ea6b92abb3968bd819475b1
prerequisite-patch-id: 7585c78056095ec991615cbbe877e06f713aada9
prerequisite-patch-id: a9fb1bf1718ad8de6fc26d97a7dc5baf41dc38bd
prerequisite-patch-id: 1e5c9ef57d1df286042f4ae3eb420394c8b3b045
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 2/6] build-system/guile: Run SRFI-64 tests.
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
@ 2022-10-07 20:53   ` Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 3/6] guile-ffi-fftw: Respect #:tests? Maxime Devos
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

The build system needs a hint for the location of the test suite(s),
but aside from that tests are now run automatically.  I expect having
multiple test suites to be the exception, so I used 'test-script'
instead of 'test-scripts'.

Fixes: <https://issues.guix.gnu.org/58337>

* guix/build-system/guile.scm (guile-build): Pass on 'tests?', 'test-script'
and 'test-arguments'.
(guile-cross-build): Likewise.
* guix/build/guile-build-system.scm (guile-check): New phase.
(%standard-phases)[check]: Use it.
* doc/guix.texi (Build Systems)[guile-build-system]: Document new behaviour.
---
 doc/guix.texi                     | 15 ++++++++---
 guix/build-system/guile.scm       | 13 +++++++++
 guix/build/guile-build-system.scm | 44 +++++++++++++++++++++++++++++--
 3 files changed, 67 insertions(+), 5 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 08dab8e5b1..858bcec96f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9022,13 +9022,22 @@ code and that are so lean that they don't even have a makefile, let alone a
 @file{configure} script.  It compiles Scheme code using @command{guild
 compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and
 installs the @file{.scm} and @file{.go} files in the right place.  It also
-installs documentation.
+installs documentation and runs SRFI-64 test suites.
+
+Test suites cannot be found automatically; the location of the test
+suite can be set with the @code{#:test-script} argument (as a string).
+If there are multiple test scripts, this argument can be set to a staged
+list of locations.  Sometimes some tests need to be skipped, if so,
+@code{#:test-arguments} can be a staged list with extra arguments to
+pass to the test driver (see @code{test-driver.scm --help} for details).
 
 This build system supports cross-compilation by using the
 @option{--target} option of @samp{guild compile}.
 
-Packages built with @code{guile-build-system} must provide a Guile package in
-their @code{native-inputs} field.
+Packages built with @code{guile-build-system} must provide a Guile
+package in their @code{native-inputs} field.  Additionally, if
+@code{#:test-script} is used, a test driver (usually
+@code{guile-test-driver}) must be provided.
 @end defvr
 
 @defvr {Scheme Variable} julia-build-system
diff --git a/guix/build-system/guile.scm b/guix/build-system/guile.scm
index 36a88e181a..75d62fe403 100644
--- a/guix/build-system/guile.scm
+++ b/guix/build-system/guile.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018-2019, 2021-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -78,6 +79,9 @@ (define %compile-flags
 
 (define* (guile-build name inputs
                       #:key source
+                      (tests? #true)
+                      (test-script #false)
+                      (test-arguments ''())
                       (guile #f)
                       (phases '%standard-phases)
                       (outputs '("out"))
@@ -98,6 +102,9 @@ (define builder
 
           (guile-build #:name #$name
                        #:source #+source
+                       #:tests? #$tests?
+                       #:test-script #$test-script
+                       #:test-arguments #$test-arguments
                        #:source-directory #$source-directory
                        #:scheme-file-regexp #$scheme-file-regexp
                        #:not-compiled-file-regexp #$not-compiled-file-regexp
@@ -122,6 +129,9 @@ (define* (guile-cross-build name
                             build-inputs target-inputs host-inputs
                             (guile #f)
                             source
+                            (tests? #false)
+                            (test-script #false)
+                            (test-arguments ''())
                             (outputs '("out"))
                             (search-paths '())
                             (native-search-paths '())
@@ -149,6 +159,9 @@ (define %outputs
             #$(outputs->gexp outputs))
 
           (guile-build #:source #+source
+                       #:tests? #$tests?
+                       #:test-script #$test-script
+                       #:test-arguments #$test-arguments
                        #:system #$system
                        #:target #$target
                        #:outputs %outputs
diff --git a/guix/build/guile-build-system.scm b/guix/build/guile-build-system.scm
index 32a431d347..838cb13089 100644
--- a/guix/build/guile-build-system.scm
+++ b/guix/build/guile-build-system.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -19,8 +20,10 @@
 (define-module (guix build guile-build-system)
   #:use-module ((guix build gnu-build-system) #:prefix gnu:)
   #:use-module (guix build utils)
+  #:use-module (rnrs exceptions)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-34)
   #:use-module (ice-9 match)
   #:use-module (ice-9 popen)
   #:use-module (ice-9 rdelim)
@@ -29,7 +32,8 @@ (define-module (guix build guile-build-system)
   #:use-module (guix build utils)
   #:export (target-guile-effective-version
             %standard-phases
-            guile-build))
+            guile-build
+            guile-check))
 
 (define* (target-guile-effective-version #:optional guile)
   "Return the effective version of GUILE or whichever 'guile' is in $PATH.
@@ -201,6 +205,42 @@ (define* (build #:key outputs inputs native-inputs
      source-files))
     #t))
 
+(define* (guile-check #:key tests? test-script (test-arguments '())
+                      (source-directory ".")
+                      native-inputs inputs
+                      #:allow-other-keys)
+  (when tests?
+    ;; Let Guile find the source code of newly compiled modules,
+    ;; otherwise the modules won't be found even if Guile knows
+    ;; where the compiled code is.
+    (setenv "GUILE_LOAD_PATH"
+            (string-append source-directory
+                           (match (getenv "GUILE_LOAD_PATH")
+                             (#f "")
+                             (path (string-append ":" path)))))
+    (for-each
+     (lambda (test-script)
+       (guard (c ((invoke-error? c)
+                  (when (equal? (list (invoke-error-exit-status c)
+                                      (invoke-error-term-signal c)
+                                      (invoke-error-stop-signal c))
+                                '(127 #false #false))
+                    (display "hint: Make sure 'guile-test-driver' is in\
+ 'native-inputs'.\n"
+                             (current-error-port)))
+                  (raise-continuable c)))
+         (apply invoke "test-driver.scm"
+                (string-append "--test-name=" test-script) test-arguments)))
+     (match test-script
+       ;; Tests can be separated over multiple files.
+       ((? list? test-scripts) test-scripts)
+       ((? string? test-script) (list test-script))
+       (#false
+        (format (current-error-port)
+                "warning: location of test suite is unknown; not running\
+ tests~%")
+        '())))))
+
 (define* (install-documentation #:key outputs
                                 (documentation-file-regexp
                                  %documentation-file-regexp)
@@ -222,7 +262,7 @@ (define %standard-phases
     (replace 'build build)
     (add-after 'build 'install-documentation
       install-documentation)
-    (delete 'check)
+    (replace 'check guile-check)
     (delete 'strip)
     (delete 'validate-runpath)
     (delete 'install)))
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 3/6] guile-ffi-fftw: Respect #:tests?.
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 2/6] build-system/guile: Run SRFI-64 tests Maxime Devos
@ 2022-10-07 20:53   ` Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 4/6] guile-ffi-fftw: Modernise style Maxime Devos
                     ` (3 subsequent siblings)
  5 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[arguments]<#:test-script>: Set it.
[arguments]{#:phases}{check}: Remove custom phase.
[native-inputs]: Add native-input required by uncustomised 'check' phase.
---
 gnu/packages/guile-xyz.scm | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 56f0bb8acc..2ab5683d19 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3682,6 +3682,7 @@ (define-public guile-ffi-fftw
       (build-system guile-build-system)
       (arguments
        `(#:source-directory "mod"
+         #:test-script "../test/test-ffi-fftw.scm"
          #:phases
          (modify-phases %standard-phases
            (add-after 'unpack 'prepare-build
@@ -3689,11 +3690,8 @@ (define-public guile-ffi-fftw
                (substitute* "mod/ffi/fftw.scm"
                  (("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
                   (format #f "\"~a/lib\"" (assoc-ref inputs "fftw"))))
-               #t))
-           (add-after 'build 'check
-             (lambda _
-               (invoke "guile" "-L" "mod"
-                       "-s" "test/test-ffi-fftw.scm"))))))
+               #t)))))
+      (native-inputs (list guile-test-driver))
       (inputs
        (list fftw guile-2.2))
       (home-page "https://github.com/lloda/guile-ffi-fftw/")
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 4/6] guile-ffi-fftw: Modernise style.
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 2/6] build-system/guile: Run SRFI-64 tests Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 3/6] guile-ffi-fftw: Respect #:tests? Maxime Devos
@ 2022-10-07 20:53   ` Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 5/6] guile-ffi-fftw: Update to new Guile version Maxime Devos
                     ` (2 subsequent siblings)
  5 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

The 'object->string' change is more a personal thing.
"guix style" adds modify-phases on the same line as #:phases, causing the
80-columns limit to be exceeded, so it has been partially ignored.

* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[arguments]<#:test-script>: Set it.
[arguments]{#:phases}{check}: Remove custom phase.
[native-inputs]: Add native-input required by uncustomised 'check' phase.
---
 gnu/packages/guile-xyz.scm | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 2ab5683d19..a664e37968 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3681,22 +3681,23 @@ (define-public guile-ffi-fftw
                   "08j40a5p6a8pgvhffmzb5rfdnrav2mksy3gfjkdqy93jfj1z5afg"))))
       (build-system guile-build-system)
       (arguments
-       `(#:source-directory "mod"
-         #:test-script "../test/test-ffi-fftw.scm"
-         #:phases
-         (modify-phases %standard-phases
-           (add-after 'unpack 'prepare-build
-             (lambda* (#:key inputs #:allow-other-keys)
-               (substitute* "mod/ffi/fftw.scm"
-                 (("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
-                  (format #f "\"~a/lib\"" (assoc-ref inputs "fftw"))))
-               #t)))))
+       (list #:source-directory "mod"
+             #:test-script "../test/test-ffi-fftw.scm"
+             #:phases
+             #~(modify-phases %standard-phases
+                 (add-after 'unpack 'prepare-build
+                   (lambda* (#:key inputs #:allow-other-keys)
+                     (substitute* "mod/ffi/fftw.scm"
+                       (("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
+                        (object->string
+                         (dirname
+                          (search-input-file inputs "lib/libfftw3.so"))))))))))
       (native-inputs (list guile-test-driver))
-      (inputs
-       (list fftw guile-2.2))
+      (inputs (list fftw guile-2.2))
       (home-page "https://github.com/lloda/guile-ffi-fftw/")
       (synopsis "Access FFTW through Guile's FFI")
-      (description "This is a minimal set of Guile FFI bindings for the FFTW
+      (description
+       "This is a minimal set of Guile FFI bindings for the FFTW
 library's ‘guru interface’.  It provides two functions: @code{fftw-dft! rank
 sign in out} and @code{fftw-dft rank sign in}.  These bindings being minimal,
 there is no support for computing & reusing plans, or split r/i transforms, or
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 5/6] guile-ffi-fftw: Update to new Guile version.
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
                     ` (2 preceding siblings ...)
  2022-10-07 20:53   ` [bug#58365] [PATCH 4/6] guile-ffi-fftw: Modernise style Maxime Devos
@ 2022-10-07 20:53   ` Maxime Devos
  2022-10-07 20:53   ` [bug#58365] [PATCH 6/6] guile-ac-d-bus: Don't duplicate 'check' phase Maxime Devos
  2022-10-18 12:36   ` [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Ludovic Courtès
  5 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

It seems to build (including tests) fine and there are no dependents, so it
appears safe to update to a newer Guile version.

* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[inputs]{guile-2.2}: Replace by guile-3.0.
---
 gnu/packages/guile-xyz.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index a664e37968..711e2eade9 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3693,7 +3693,7 @@ (define-public guile-ffi-fftw
                          (dirname
                           (search-input-file inputs "lib/libfftw3.so"))))))))))
       (native-inputs (list guile-test-driver))
-      (inputs (list fftw guile-2.2))
+      (inputs (list fftw guile-3.0))
       (home-page "https://github.com/lloda/guile-ffi-fftw/")
       (synopsis "Access FFTW through Guile's FFI")
       (description
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 6/6] guile-ac-d-bus: Don't duplicate 'check' phase.
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
                     ` (3 preceding siblings ...)
  2022-10-07 20:53   ` [bug#58365] [PATCH 5/6] guile-ffi-fftw: Update to new Guile version Maxime Devos
@ 2022-10-07 20:53   ` Maxime Devos
  2022-10-18 12:36   ` [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Ludovic Courtès
  5 siblings, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2022-10-07 20:53 UTC (permalink / raw)
  To: 58365; +Cc: Maxime Devos

Now guile-build-system has a 'check' phase, the 'check' phase is duplicated.
It's harmless, but let's not.

* gnu/packages/guile-xyz.scm
(guile-ac-d-bus)[arguments]<#:phases>{check}: Instead of adding the phase,
replace it.
---
 gnu/packages/guile-xyz.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 711e2eade9..0339bf8281 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3911,7 +3911,7 @@ (define (guile-sls->sls file)
             (lambda _
               (with-directory-excursion "docs"
                 (invoke "makeinfo" "ac-d-bus"))))
-          (add-after 'build-doc 'check
+          (replace 'check
             (lambda* (#:key (tests? #t) #:allow-other-keys)
               (when tests?
                 ;; There is no locale for the ö character, which crashes
-- 
2.37.3





^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
                     ` (4 preceding siblings ...)
  2022-10-07 20:53   ` [bug#58365] [PATCH 6/6] guile-ac-d-bus: Don't duplicate 'check' phase Maxime Devos
@ 2022-10-18 12:36   ` Ludovic Courtès
  2022-10-18 13:30     ` Maxime Devos
  5 siblings, 1 reply; 13+ messages in thread
From: Ludovic Courtès @ 2022-10-18 12:36 UTC (permalink / raw)
  To: Maxime Devos; +Cc: 58365

Hello,

Maxime Devos <maximedevos@telenet.be> skribis:

> A copy is made of test-driver.scm to avoid potentially surprising rebuilds
> when changes are made.
>
> * gnu/packages/guile.scm (guile-test-driver): New variable.
> * gnu/packages/aux-files/test-driver.scm: New file.
> * Makefile.am (AUX_FILES): Register it.
> * build-aux/test-driver.scm: Add a note.

I very much like the idea of supporting #:tests? in
‘guile-build-system’, but I’m skeptical about this approach.

The reason I’m skeptical is because ‘test-driver.scm’ is written as an
Automake test driver (it follows the “protocol” defined by Automake) for
tests that use SRFI-64—that’s a specific kind of tool, and one approach
to writing test suites in Guile.

I guess all I’m saying is that I doubt this is widely applicable, which
the diff seems to confirm:

>  4 files changed, 326 insertions(+), 2 deletions(-)

Perhaps at this stage there really isn’t much we can factorize, after
all.  It would be nice to fix this, but that’s probably work to be done
upstream—for example, by adding a “guild test” command.  We can even
beta-test such a command as an external project before including it in
Guile proper.

WDYT?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2022-10-18 12:36   ` [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Ludovic Courtès
@ 2022-10-18 13:30     ` Maxime Devos
  2022-10-18 15:44       ` Ludovic Courtès
  0 siblings, 1 reply; 13+ messages in thread
From: Maxime Devos @ 2022-10-18 13:30 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 58365


[-- Attachment #1.1.1: Type: text/plain, Size: 4905 bytes --]



On 18-10-2022 14:36, Ludovic Courtès wrote:
> Hello,
> 
> Maxime Devos <maximedevos@telenet.be> skribis:
> 
>> A copy is made of test-driver.scm to avoid potentially surprising rebuilds
>> when changes are made.
>>
>> * gnu/packages/guile.scm (guile-test-driver): New variable.
>> * gnu/packages/aux-files/test-driver.scm: New file.
>> * Makefile.am (AUX_FILES): Register it.
>> * build-aux/test-driver.scm: Add a note.
> 
> I very much like the idea of supporting #:tests? in
> ‘guile-build-system’, but I’m skeptical about this approach.
> 
> The reason I’m skeptical is because ‘test-driver.scm’ is written as an
> Automake test driver (it follows the “protocol” defined by Automake)

It is perfectly usable outside Automake, as this patch series 
demonstrates -- it does not depend on any Automake details.

Also, the alternative would be to do "guile -l the-test-suite.scm" 
directly (without a test runner) -- while sometimes this works, 
sometimes the SRFI-64 implementation really cares that there is actually 
some test runner (I got a failure for some unpackaged Scheme library 
when running it without test-driver.scm), and additionally IMO the 
output of "guile -l the-test-suite.scm" isn't very nice, the 
test-driver.scm has much more nice output and is more consistent with 
other packages' test output.

(A third alternative would be to ignore the tests altogether, but 
running tests is good and I have some patches for adding support for the 
tests.)

> for tests that use SRFI-64—

That's the standard Scheme API for test suites in Scheme -- widely 
applicable.  (There's also SRFI-78, but that's currently unsupported by 
Guile, so it's currently not relevant for guile-build-system.)

> that’s a specific kind of tool, and one approach
> to writing test suites in Guile.

It's the only approach to my knowledge, aside from 'let's just put a 
bunch of (when wrong? (error ...)) in a script and do 'guile -l 
the-script.scm' or reimplement our own thing, but that's not a nice 
approach, I would rather make it possible to use the more-standard and 
nicer (even though it's somewhat stateful) SRFI-64.

Also, I don't see what's wrong with 'specific kind of tools' -- should 
we remove 'nml' then because it's super-specific to openttd?  And even 
if there exists another tool that would fit here, why would that matter? 
I mean, the 'gcc' approach to compiling C wasn't rejected in Guix even 
though it's just 'one' approach to compiling C -- for example, 'clang' 
is another approach.

If someone has another approach to running SRFI-64 test suites in 
guile-build-system, they can propose it and it can be discussed which 
one would be better for guile-build-system.

Additionally, a nitpick -- test-driver.scm is an approach to running 
(SRFI-64) test suites, not for writing them, for writing test suites, 
SRFI-64 is an approach.

> I guess all I’m saying is that I doubt this is widely applicable, which
> the diff seems to confirm:
> 
>>   4 files changed, 326 insertions(+), 2 deletions(-)

I don't how you get from the diff that it isn't widely applicable -- it 
is applicable to every package that has a SRFI-64 test suite and uses 
guile-build-system.  I just didn't check every guile-build-system thing 
for whether it has a SRFI-64 test suite.  And once more Scheme libraries 
are packaged (especially Scheme libraries not written with Guile in 
mind, where gnu-build-system is likely inapplicable), presumably we 
would get more packages where this patch series can run tests.

Additionally, have you considered that one possible reason for lack of 
SRFI-64 test suites in guile-build-system packages, is that previously 
there was no good method to run those test suites, so (for someone using 
Guix and Guile) writing them would be pointless?

> Perhaps at this stage there really isn’t much we can factorize, after
> all.  It would be nice to fix this,

I don't see any problem to fix or anything to factorize.  What is this 
'this' you are referring to?

> but that’s probably work to be done
> upstream—for example, by adding a “guild test” command.

Going by "git log build-aux/test-driver.scm", Guix is the upstream, not 
Guile, though I suppose it could eventually be moved to Guile if desired 
by both.

> We can even
> beta-test such a command as an external project before including it in
> Guile proper. >
> WDYT?

I think it already has seen plenty of testing -- in Guix, in 
scheme-gnunet, I think I've also seen it used in the GnuTLS bindings, 
likely by most other Guile programs that use gnu-build-system and have a 
test suite as well.

I mean, it existed since 2015 according to "git log", that's 7 years of 
testing, seems plenty to me, and it also has been used by various projects.

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2022-10-18 13:30     ` Maxime Devos
@ 2022-10-18 15:44       ` Ludovic Courtès
  2023-11-20  0:34         ` Maxime Devos
  0 siblings, 1 reply; 13+ messages in thread
From: Ludovic Courtès @ 2022-10-18 15:44 UTC (permalink / raw)
  To: Maxime Devos; +Cc: 58365

Hi Maxime,

That’s quite a wall of text that you sent.  :-)

Let me try to clarify what I think can be a fruitful way forward.
Again, I agree with the goal you have here, I’m just proposing a
different strategy.

To me, the problem that we have is lack of a standard way to run tests
in Guile packages that don’t use Autotools.  That, in turn, leads to
duplication in package definitions—whether it’s ‘check’ phases in Guix
or something similar in Debian and other distros.

Instead of addressing it downstream (in Guix), my suggestion would be to
address it upstream—that is, to promote one way to run SRFI-64 test
suites, for example with a new “guild test” command.

That “guild test” command would probably be very similar to
‘build-aux/test-driver.scm’.  As you note, this script is battle-tested
and provides useful functionality.  If we would turn it into a (scripts
test) module, that ‘guild’ would pick up, then we’d have a good start.
With proper documentation, programmers who use Guile would have an
incentive to use this method; packagers would benefit because the
default ‘check’ phase would boil down to invoking “guild test”.

I hope this clarifies my proposal.  WDYT?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2022-10-18 15:44       ` Ludovic Courtès
@ 2023-11-20  0:34         ` Maxime Devos
  2023-11-20  0:41           ` Maxime Devos
  2023-12-03  9:17           ` Efraim Flashner
  0 siblings, 2 replies; 13+ messages in thread
From: Maxime Devos @ 2023-11-20  0:34 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 58365


[-- Attachment #1.1.1: Type: text/plain, Size: 4237 bytes --]

Hi,

Op 18-10-2022 om 17:44 schreef Ludovic Courtès:
> Hi Maxime,
> 
> That’s quite a wall of text that you sent.  :-)

Going by the definition at
‘https://en.wikipedia.org/wiki/Wikipedia:Wall_of_text’, it isn't -- it 
is not intentionally disruptive (neither unintentionally disruptive), it 
is not an attempt at overwhelmening, it is not irrelevant and AFAICT it 
is not due to an unawareness of good practices.

The only ‘wall-of-text’ property it might have is its length.

So, you're apparently ignoring all what I wrote previously?

> Let me try to clarify what I think can be a fruitful way forward.
> Again, I agree with the goal you have here, I’m just proposing a
> different strategy.
> 
> To me, the problem that we have is lack of a standard way to run tests
> in Guile packages that don’t use Autotools.  That, in turn, leads to
> duplication in package definitions—whether it’s ‘check’ phases in Guix
> or something similar in Debian and other distros.

Err, this is the same for Autotools Guile packages -- Autotools Guile 
packages have to copy test-driver.scm too (or do (exit 1)-style tests, 
but those aren't really the tests this patch series).  You can drop the 
qualifier ‘

Also, there is, in fact a standard way to run tests in Guile packages: 
copy the test-driver.scm (and in case of Autotools, parts of the 
Makefile.am from another Guile package).

While this does sound like _a_ problem, it is not a problem this patch 
series is about and hence not _the_ problem (see later for details).

> Instead of addressing it downstream (in Guix), my suggestion would be to
> address it upstream—that is, to promote one way to run SRFI-64 test
> suites, for example with a new “guild test” command.

But Guix is the upstream (I mentioned this in the first e-mail *) -- 
AFAICT, Guix has the latest version of test-driver.scm, and AFAICT Guix 
is where test-driver.scm + parts of Makefile.am is copied from.

(*) and also right in the e-mail you were responding to:

> Going by "git log build-aux/test-driver.scm", Guix is the upstream, not 
> Guile, though I suppose it could eventually be moved to Guile if desired 
> by both.

See, I can make claims too.  The difference is that you multiply times 
claimed things _without_ evidence whereas when I claimed the contrary 
_with_ evidence.

... did you even read it?  My ‘wall of text’ wasn't for show.

> That “guild test” command would probably be very similar to
> ‘build-aux/test-driver.scm’.  As you note, this script is battle-tested
> and provides useful functionality.  If we would turn it into a (scripts
> test) module, that ‘guild’ would pick up, then we’d have a good start.
> With proper documentation, programmers who use Guile would have an
> incentive to use this method; packagers would benefit because the
> default ‘check’ phase would boil down to invoking “guild test”.
> 
> I hope this clarifies my proposal.  WDYT?

I think that is a clear proposal, I'm not interested in implementing it, 
I consider it out-of-scope and something that could be done separately 
from this patch series, and I won't do it, no matter that you are 
presenting it as if it were natural for me to do so -- I'm not paid 
enough for this and don't want your money.

Finding who should be the upstream of test-driver.scm or finding a new 
home for test-driver.scm is not at all the point of this patch series -- 
the point is to support #:tests? in guile-build-system, and for that a 
test driver is needed, and look, there is a test driver right here in 
the Guix repo, let's package it so that it becomes available to the 
build/test code of guile-build-system (and as a bonus, it makes 
test-driver.scm more discoverable).

IOW, going back to your ‘I'm just proposing a different strategy’ 
comment -- you aren't proposing a different strategy for the patch 
series, you are proposing something supplemental, and that ‘something 
supplemental’ is something for guile-devel@, not guix@.

Best regards (except to Ludo (*)),
Maxime Devos

(*) because of the unconstructive and disrespectful PMs, and other reasons

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2023-11-20  0:34         ` Maxime Devos
@ 2023-11-20  0:41           ` Maxime Devos
  2023-12-03  9:17           ` Efraim Flashner
  1 sibling, 0 replies; 13+ messages in thread
From: Maxime Devos @ 2023-11-20  0:41 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 58365


[-- Attachment #1.1.1: Type: text/plain, Size: 324 bytes --]

(Some context I forgot to include:)

Around the same time at the last previous message  on technical matters 
(https://issues.guix.gnu.org/58365#9), Ludo' sent a PM on some 
non-technical matters, and I answered to the PM instead of 58365, 
leaving the technical matters message unreplied.  So, now, at last, a reply.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system
  2023-11-20  0:34         ` Maxime Devos
  2023-11-20  0:41           ` Maxime Devos
@ 2023-12-03  9:17           ` Efraim Flashner
  1 sibling, 0 replies; 13+ messages in thread
From: Efraim Flashner @ 2023-12-03  9:17 UTC (permalink / raw)
  To: Maxime Devos; +Cc: Ludovic Courtès, 58365

[-- Attachment #1: Type: text/plain, Size: 6135 bytes --]

Top posting to make sure it is read:

Maxime: I understand this has been going back and forth for over a year
now and has been very frustrating, but parts of this email are
unnecessary.  As a maintainer I haven't done enough to try to ease
tensions and find an equitable solution and for that I apologize.

Ludo: Based on the publicly available emails I'm not seeing why this was
held in limbo for so long.  For plenty of packages we have a policy of
writing a patch, submitting it upstream if possible, and carrying it
until we can drop it.  Currently I'm not seeing this as so much
different as either that case or some of the other files we carry in
gnu/packages/aux-files.

To everyone: Please keep in mind if you feel the need to remove the
mailing list from the reply chain if it is because it is something that
really, truly, doesn't need to be sent to everyone or because you'd
rather have it hidden from view.

I am liberally removing parts of the email as I respond below:

On Mon, Nov 20, 2023 at 01:34:12AM +0100, Maxime Devos wrote:
> Hi,
> 
> Op 18-10-2022 om 17:44 schreef Ludovic Courtès:
> > Let me try to clarify what I think can be a fruitful way forward.
> > Again, I agree with the goal you have here, I’m just proposing a
> > different strategy.

Noted

> > To me, the problem that we have is lack of a standard way to run tests
> > in Guile packages that don’t use Autotools.  That, in turn, leads to
> > duplication in package definitions—whether it’s ‘check’ phases in Guix
> > or something similar in Debian and other distros.

Noted

> Err, this is the same for Autotools Guile packages -- Autotools Guile
> packages have to copy test-driver.scm too (or do (exit 1)-style tests, but
> those aren't really the tests this patch series).  You can drop the
> qualifier ‘
> 
> Also, there is, in fact a standard way to run tests in Guile packages: copy
> the test-driver.scm (and in case of Autotools, parts of the Makefile.am from
> another Guile package).
> 
> While this does sound like _a_ problem, it is not a problem this patch
> series is about and hence not _the_ problem (see later for details).

Noted.

Referring back to duplicated code above from Ludo, I wonder about using
the test-driver script by default vs adding it per-package, but I don't
have a good handle on how many packages are waiting to use it.

> > Instead of addressing it downstream (in Guix), my suggestion would be to
> > address it upstream—that is, to promote one way to run SRFI-64 test
> > suites, for example with a new “guild test” command.
> 
> But Guix is the upstream (I mentioned this in the first e-mail *) -- AFAICT,
> Guix has the latest version of test-driver.scm, and AFAICT Guix is where
> test-driver.scm + parts of Makefile.am is copied from.
> 
> (*) and also right in the e-mail you were responding to:

Sounds like Guix is the upstream of test-driver.scm and guile, as the
upstream language in this build-system, could absorb it and make it
available through 'guild test'.

> > Going by "git log build-aux/test-driver.scm", Guix is the upstream, not
> > Guile, though I suppose it could eventually be moved to Guile if desired
> > by both.
>
> > That “guild test” command would probably be very similar to
> > ‘build-aux/test-driver.scm’.  As you note, this script is battle-tested
> > and provides useful functionality.  If we would turn it into a (scripts
> > test) module, that ‘guild’ would pick up, then we’d have a good start.
> > With proper documentation, programmers who use Guile would have an
> > incentive to use this method; packagers would benefit because the
> > default ‘check’ phase would boil down to invoking “guild test”.
> > 
> > I hope this clarifies my proposal.  WDYT?

Sounds like a good long-term plan.  That said, it is something to work
out at a later date, not in this patch series.

> I think that is a clear proposal, I'm not interested in implementing it, I
> consider it out-of-scope and something that could be done separately from
> this patch series.

Noted

> Finding who should be the upstream of test-driver.scm or finding a new home
> for test-driver.scm is not at all the point of this patch series -- the
> point is to support #:tests? in guile-build-system, and for that a test
> driver is needed, and look, there is a test driver right here in the Guix
> repo, let's package it so that it becomes available to the build/test code
> of guile-build-system (and as a bonus, it makes test-driver.scm more
> discoverable).

Sounds reasonable.

> IOW, going back to your ‘I'm just proposing a different strategy’ comment --
> you aren't proposing a different strategy for the patch series, you are
> proposing something supplemental, and that ‘something supplemental’ is
> something for guile-devel@, not guix@.

I ignored this the first time through because it has to do with the
guile-build-system and that's not something I've ever looked at.

It sounds like the best *long term* option is to move this upstream to
guile or to package it separately in its own repository.  Seeing as it's
been a year since this thread started I don't see that as a reason to
hold anything up.

In the immediate term, it looks like this patch set (which I haven't
tested, but noticed the patches seem to be in the wrong order) would
bring testing to the guile packages, which everyone agrees is a Good
Thing™.  Lets get it fixed up and merged and any other bits can be
worked on at a later date.  If no one else comes forward to make sure
that all the pieces work the way they're supposed to I will take care of
it myself in about a week.

In the mean time, please try to keep the conversation at least civil,
and remember that we're all bound by the Code of Conduct while involved
with Guix.


-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2023-12-03  9:18 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-07 20:47 [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Maxime Devos
2022-10-07 20:53 ` [bug#58365] [PATCH 1/6] gnu: Add guile-test-driver Maxime Devos
2022-10-07 20:53   ` [bug#58365] [PATCH 2/6] build-system/guile: Run SRFI-64 tests Maxime Devos
2022-10-07 20:53   ` [bug#58365] [PATCH 3/6] guile-ffi-fftw: Respect #:tests? Maxime Devos
2022-10-07 20:53   ` [bug#58365] [PATCH 4/6] guile-ffi-fftw: Modernise style Maxime Devos
2022-10-07 20:53   ` [bug#58365] [PATCH 5/6] guile-ffi-fftw: Update to new Guile version Maxime Devos
2022-10-07 20:53   ` [bug#58365] [PATCH 6/6] guile-ac-d-bus: Don't duplicate 'check' phase Maxime Devos
2022-10-18 12:36   ` [bug#58365] [PATCH 0/6] Support #:tests? in guile-build-system Ludovic Courtès
2022-10-18 13:30     ` Maxime Devos
2022-10-18 15:44       ` Ludovic Courtès
2023-11-20  0:34         ` Maxime Devos
2023-11-20  0:41           ` Maxime Devos
2023-12-03  9:17           ` Efraim Flashner

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.