[bug#42146] [PATCH core-updates 1/?] build: substitute: Don't fail silently.

[Maxim Cournoyer <maxim.cournoyer@gmail.com>]

Hi,

Bruno Victal <mirai@makinata.eu> writes:

> Hi Ludo’,
>
> On 2023-10-19 21:49, Ludovic Courtès wrote:
>>> +(test-group "substitute*"
>> 
>> I’d avoid groups: they’re not super useful and the output with the
>> Automake driver is terrible.
>
> I personally like using groups for structuring the logic of the tests.
>
> The output issue sounds either like a bug to me that should be fixed
> or a place for improvement.

Ludo, could you refresh my memory about what is wrong with the Automake
SRFI-64 driver when it comes to tests?  I think it used to masks the
tests in its output, but I'm not seeing this now?

Consider the following tests, where the "elm->package-name" and
"infer-elm-package-name" tests are in the "round trip" test group itself
nested in the "elm->package-name and infer-elm-package-name" test group.

tests/configuration.scm contains a "duplicated/conflicting entries" test
group that contains "duplicate sanitizer", "duplicate serializer" and
"conflicting use of serializer + empty-serializer":

--8<---------------cut here---------------start------------->8---
make check TESTS='tests/services/configuration.scm tests/elm.scm' \
  SCM_LOG_DRIVER_FLAGS="--brief=no"
--8<---------------cut here---------------end--------------->8---

PASS: tests/services/configuration.scm - default value, no serialization
PASS: tests/services/configuration.scm - wrong type for a field
PASS: tests/services/configuration.scm - default value, custom serializer
PASS: tests/services/configuration.scm - no default value, provided
PASS: tests/services/configuration.scm - no default value, not provided
PASS: tests/services/configuration.scm - serialize-configuration
PASS: tests/services/configuration.scm - serialize-configuration [deprecated]
PASS: tests/services/configuration.scm - serialize-configuration with no-serialization
PASS: tests/services/configuration.scm - serialize-configuration with prefix
PASS: tests/services/configuration.scm - default value, sanitizer
PASS: tests/services/configuration.scm - string value, sanitized to number
PASS: tests/services/configuration.scm - default value, serializer literal
PASS: tests/services/configuration.scm - empty-serializer as literal
PASS: tests/services/configuration.scm - empty-serializer as procedure
PASS: tests/services/configuration.scm - default value, sanitizer, permutation
PASS: tests/services/configuration.scm - default value, serializer, permutation
PASS: tests/services/configuration.scm - string value sanitized to number, permutation
PASS: tests/services/configuration.scm - default value, sanitizer, permutation 2
PASS: tests/services/configuration.scm - default value, serializer, permutation 2
PASS: tests/services/configuration.scm - duplicate sanitizer
PASS: tests/services/configuration.scm - duplicate serializer
PASS: tests/services/configuration.scm - conflicting use of serializer + empty-serializer
PASS: tests/services/configuration.scm - Mix of bare serializer and new syntax
PASS: tests/services/configuration.scm - Mix of bare serializer and new syntax, permutation)
PASS: tests/services/configuration.scm - maybe value serialization
PASS: tests/services/configuration.scm - maybe value serialization of the instance
PASS: tests/services/configuration.scm - maybe value serialization of the instance, unspecified
PASS: tests/services/configuration.scm - symbol maybe value serialization, unspecified
PASS: tests/services/configuration.scm - maybe value without serialization no procedure bound
PASS: tests/services/configuration.scm - maybe type, no default
PASS: tests/services/configuration.scm - maybe type, with default
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infer-elm-package-name
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm->package-name
PASS: tests/elm.scm - infers other name
PASS: tests/elm.scm - infered name round-trips
PASS: tests/elm.scm - elm
PASS: tests/elm.scm - guile
PASS: tests/elm.scm - gcc-toolchain
PASS: tests/elm.scm - font-adobe-source-sans-pro
PASS: tests/elm.scm - (elm->guix-package "elm/core")
PASS: tests/elm.scm - (elm-recursive-import "elm-guix/demo")
============================================================================
Testsuite summary for GNU Guix 1.3.0.48706-b42e6315-dirty
============================================================================
# TOTAL: 85
# PASS:  85
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

Observation: the output says nothing about the groups, but at least the
nested tests are correctly listed.

-- 
Thanks,
Maxim