Package gmsh: best practise with test-suite?

Package gmsh: best practise with test-suite?

[zimoun]

Hi Guix,

I am working on packaging GetDP [1] the companion finite element solver of Gmsh.

[1] http://getdp.info/

GetDP depends on some part of Gmsh. So I have a working
gmsh-minimal-no-x package, turning off FLTK, OpenCascade and
non-necessary features.

Then the full test suite fails. As expected. :-)
Now they are all turned off with "#:tests? #f".
Which is okish... but I am not fully satisfied.

Well, the issue is that the tests are generated with this piece of CMake:

  include(CTest)
  file(GLOB_RECURSE TESTFILES
       tutorial/*.geo demos/*.geo benchmarks/?d/*.geo benchmarks/extrude/*.geo)
  foreach(TESTFILE ${TESTFILES})
    # use relative path for cygwin/mingw (the pure win exe built with the mingw
    # compilers does not understand a full cygwin-style path)
    FILE(RELATIVE_PATH TEST ${CMAKE_CURRENT_BINARY_DIR} ${TESTFILE})
    add_test(${TEST} ./gmsh ${TEST} -3 -nopopup -o ./tmp.msh)
  endforeach()

What is the better?
 a- Patch the CMakeLists.txt to generate only the right tests
 b- Turn off all the tests

What do you think?

Thank you in advance for any comment.


All the best,
simon

Re: Package gmsh: best practise with test-suite?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 637 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> What is the better?
>  a- Patch the CMakeLists.txt to generate only the right tests
>  b- Turn off all the tests

There may be a third option, run only the tests that are known to pass.

I know very little about CMake, but I searched for ctest, which is
something mentioned in the CMakeLists.txt file, and it brought up this
[1] which mentions --exclude-regex and --label-exclude. Maybe these
could somehow be passed in, or maybe set through environment variables?

1: http://bima.astro.umd.edu/nemo/man_html/ctest.1.html

If that's possible, it might be less britle than patching the file.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Package gmsh: best practise with test-suite?

[zimoun]

Hi Chris,

Thank you for your tip.

On Mon, 11 Feb 2019 at 20:38, Christopher Baines <mail@cbaines.net> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:
>
> > What is the better?
> >  a- Patch the CMakeLists.txt to generate only the right tests
> >  b- Turn off all the tests
>
> There may be a third option, run only the tests that are known to pass.
>
> I know very little about CMake, but I searched for ctest, which is
> something mentioned in the CMakeLists.txt file, and it brought up this
> [1] which mentions --exclude-regex and --label-exclude. Maybe these
> could somehow be passed in, or maybe set through environment variables?

From my understanding, the cmake-build-system calls "make test" and
not "ctest". Right?

If I understand well, the idea should be to turn off the all tests
("#:tests? #f") and to add a final phase calling "ctest" with the
correct tests. Does this make sense?


All the best,
simon

Re: Package gmsh: best practise with test-suite?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> On Mon, 11 Feb 2019 at 20:38, Christopher Baines <mail@cbaines.net> wrote:
>> zimoun <zimon.toutoune@gmail.com> writes:
>>
>> > What is the better?
>> >  a- Patch the CMakeLists.txt to generate only the right tests
>> >  b- Turn off all the tests
>>
>> There may be a third option, run only the tests that are known to pass.
>>
>> I know very little about CMake, but I searched for ctest, which is
>> something mentioned in the CMakeLists.txt file, and it brought up this
>> [1] which mentions --exclude-regex and --label-exclude. Maybe these
>> could somehow be passed in, or maybe set through environment variables?
>
> From my understanding, the cmake-build-system calls "make test" and
> not "ctest". Right?

I think CMake generates the Makefile, so I would guess that "make test"
might end up running "ctest".

> If I understand well, the idea should be to turn off the all tests
> ("#:tests? #f") and to add a final phase calling "ctest" with the
> correct tests. Does this make sense?

That would work, but in my opinion a neater approach would be to keep
#:tests? as #t, and replace the 'check phase to call ctest with the
required arguments. Then #:tests? still works as an argument. Something
like...

(arguments
 '(#:phases
   (modify-phases %standard-phases
     (replace 'check
       (lambda* (#:key tests? #:allow-other-keys)
         (when tests?
           (invoke "ctest" "--exclude-regex" "..."))
         #t)))))

Hope that helps,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Package gmsh: best practise with test-suite?

[Ricardo Wurmus]

Christopher Baines <mail@cbaines.net> writes:

>> If I understand well, the idea should be to turn off the all tests
>> ("#:tests? #f") and to add a final phase calling "ctest" with the
>> correct tests. Does this make sense?
>
> That would work, but in my opinion a neater approach would be to keep
> #:tests? as #t, and replace the 'check phase to call ctest with the
> required arguments. Then #:tests? still works as an argument. Something
> like...
>
> (arguments
>  '(#:phases
>    (modify-phases %standard-phases
>      (replace 'check
>        (lambda* (#:key tests? #:allow-other-keys)
>          (when tests?
>            (invoke "ctest" "--exclude-regex" "..."))
>          #t)))))

You can also set test arguments in an earlier phase as is done in
“dune-common”, for example.

--8<---------------cut here---------------start------------->8---
         (add-after 'unpack 'disable-failing-tests
           (lambda _
             (setenv "ARGS"
                     (string-append "--exclude-regex '("
                                    (string-join
                                     (list
                                      "remoteindicestest"
                                      "remoteindicestest-mpi-2"
                                      "syncertest"
                                      "syncertest-mpi-2"
                                      "variablesizecommunicatortest"
                                      "variablesizecommunicatortest-mpi-2"
                                      "arithmetictestsuitetest"
                                      "assertandreturntest"
                                      "assertandreturntest_ndebug"
                                      "concept"
                                      "debugaligntest"
                                      "mpicollectivecommunication"
                                      "mpicollectivecommunication-mpi-2"
                                      "mpiguardtest"
                                      "mpiguardtest-mpi-2"
                                      "mpihelpertest"
                                      "mpihelpertest-mpi-2"
                                      "mpihelpertest2"
                                      "mpihelpertest2-mpi-2")
                                     "|")
                                    ")'"))
             #t))
--8<---------------cut here---------------end--------------->8---

-- 
Ricardo