all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test.
@ 2022-04-22 16:20 Paul A. Patience
  2022-04-22 16:30 ` [bug#55067] [PATCH v2] " Paul A. Patience
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Paul A. Patience @ 2022-04-22 16:20 UTC (permalink / raw)
  To: 55067; +Cc: Paul A. Patience

* gnu/packages/maths.scm (nomad-optimizer)[arguments]: Disable the
examples/advanced/library/PSDMads test.
---
It turns out the PSDMads test fails randomly.
I've disabled it.

 gnu/packages/maths.scm | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f4778179d6..26488a614c 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -2008,6 +2008,9 @@ (define-public nomad-optimizer
          (add-after 'fix-sources-for-build 'fix-sources-for-tests
            (lambda _
              (substitute* "examples/CMakeLists.txt"
+               ;; This test passes only sometimes.
+               (("^ +add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/advanced/library/PSDMads\\)\n")
+                "")
                ;; examples/basic/batch/example3 is accidentally omitted.
                (("^(add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/basic/batch/example)2(\\)\n)"
                  _ prefix suffix)
@@ -2052,10 +2055,6 @@ (define-public nomad-optimizer
               "examples/advanced/library/exampleSuggestAndObserve/cache0.txt")

              (let ((builddir (string-append (getcwd) "/../build")))
-               ;; For some reason, omitting this fix causes the
-               ;; examples/advanced/library/PSDMads test to fail, even though
-               ;; it doesn't seem to reference any part of the FixedVariable
-               ;; test.
                (let ((dir "examples/advanced/library/FixedVariable"))
                  (substitute* (string-append dir "/fixedVariable.cpp")
                    (("^( +std::string sExe = ).*" _ prefix)
--
2.35.1






^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [bug#55067] [PATCH v2] gnu: nomad-optimizer: Disable failing test.
  2022-04-22 16:20 [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test Paul A. Patience
@ 2022-04-22 16:30 ` Paul A. Patience
  2022-04-22 17:25 ` [bug#55067] [PATCH] " Maxime Devos
  2022-04-25 19:10 ` [bug#55067] [PATCH v3] " Paul A. Patience
  2 siblings, 0 replies; 7+ messages in thread
From: Paul A. Patience @ 2022-04-22 16:30 UTC (permalink / raw)
  To: 55067; +Cc: Paul A. Patience

* gnu/packages/maths.scm (nomad-optimizer)[arguments]: Disable the
examples/advanced/library/PSDMads test.
---
Just updating the copyright header too.

 gnu/packages/maths.scm | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f4778179d6..190b62fca1 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -45,7 +45,7 @@
 ;;; Copyright © 2021 Gerd Heber <gerd.heber@gmail.com>
 ;;; Copyright © 2021 Franck Pérignon <franck.perignon@univ-grenoble-alpes.fr>
 ;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
-;;; Copyright © 2021 Paul A. Patience <paul@apatience.com>
+;;; Copyright © 2021-2022 Paul A. Patience <paul@apatience.com>
 ;;; Copyright © 2021 Ivan Gankevich <i.gankevich@spbu.ru>
 ;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me>
 ;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
@@ -2008,6 +2008,9 @@ (define-public nomad-optimizer
          (add-after 'fix-sources-for-build 'fix-sources-for-tests
            (lambda _
              (substitute* "examples/CMakeLists.txt"
+               ;; This test passes only sometimes.
+               (("^ +add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/advanced/library/PSDMads\\)\n")
+                "")
                ;; examples/basic/batch/example3 is accidentally omitted.
                (("^(add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/basic/batch/example)2(\\)\n)"
                  _ prefix suffix)
@@ -2052,10 +2055,6 @@ (define-public nomad-optimizer
               "examples/advanced/library/exampleSuggestAndObserve/cache0.txt")

              (let ((builddir (string-append (getcwd) "/../build")))
-               ;; For some reason, omitting this fix causes the
-               ;; examples/advanced/library/PSDMads test to fail, even though
-               ;; it doesn't seem to reference any part of the FixedVariable
-               ;; test.
                (let ((dir "examples/advanced/library/FixedVariable"))
                  (substitute* (string-append dir "/fixedVariable.cpp")
                    (("^( +std::string sExe = ).*" _ prefix)
--
2.35.1






^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test.
  2022-04-22 16:20 [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test Paul A. Patience
  2022-04-22 16:30 ` [bug#55067] [PATCH v2] " Paul A. Patience
@ 2022-04-22 17:25 ` Maxime Devos
  2022-04-22 19:02   ` Paul A. Patience
  2022-04-25 19:10 ` [bug#55067] [PATCH v3] " Paul A. Patience
  2 siblings, 1 reply; 7+ messages in thread
From: Maxime Devos @ 2022-04-22 17:25 UTC (permalink / raw)
  To: Paul A. Patience, 55067

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

Paul A. Patience schreef op vr 22-04-2022 om 16:20 [+0000]:
> ---
> It turns out the PSDMads test fails randomly.

Sounds like a bug, preferably it would 100% reliably report the
underlying problem, whatever it is.

> I've disabled it.

I don't think this is a good reason to disable the test.
Disabling tests whenever they fail defeats the point of tests.

Cf. with, say, clinical trials for testing safety of new medical
treatments.  If some drug fails one of the tests (sometimes or always),
then one cannot simply ignore the test.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test.
  2022-04-22 17:25 ` [bug#55067] [PATCH] " Maxime Devos
@ 2022-04-22 19:02   ` Paul A. Patience
  2022-04-22 21:37     ` Maxime Devos
  0 siblings, 1 reply; 7+ messages in thread
From: Paul A. Patience @ 2022-04-22 19:02 UTC (permalink / raw)
  To: Maxime Devos; +Cc: 55067


On 2022-04-22 13:25:20-04:00, Maxime Devos wrote:
>> It turns out the PSDMads test fails randomly.
>
> Sounds like a bug, preferably it would 100% reliably report the
> underlying problem, whatever it is.

I agree with you, and I have already reported the issue to the
developers (along with some other issues that I addressed in the package
definition).

>> I've disabled it.
>
> I don't think this is a good reason to disable the test.
> Disabling tests whenever they fail defeats the point of tests.

This test failure wasn't getting detected in the past due to a missing
"set -o pipefail" in the test runner, which could explain why it was
missed by the developers.
I don't know enough to fix the bug myself, but I have a suspicion it is
a bug in the test rather than in the library.

Best regards,
Paul





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test.
  2022-04-22 19:02   ` Paul A. Patience
@ 2022-04-22 21:37     ` Maxime Devos
  0 siblings, 0 replies; 7+ messages in thread
From: Maxime Devos @ 2022-04-22 21:37 UTC (permalink / raw)
  To: Paul A. Patience; +Cc: 55067

[-- Attachment #1: Type: text/plain, Size: 696 bytes --]

Paul A. Patience schreef op vr 22-04-2022 om 19:02 [+0000]:
> On 2022-04-22 13:25:20-04:00, Maxime Devos wrote:
> > > It turns out the PSDMads test fails randomly.
> > 
> > Sounds like a bug, preferably it would 100% reliably report the
> > underlying problem, whatever it is.
> 
> I agree with you, and I have already reported the issue to the
> developers (along with some other issues that I addressed in the package
> definition).

Then I recommend adding a link to the upstream report (in a comment),
such that at the next update we know where to check if it has been
resolved, and to avoid giving the impression that just skipping tests
is acceptable.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [bug#55067] [PATCH v3] gnu: nomad-optimizer: Disable failing test.
  2022-04-22 16:20 [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test Paul A. Patience
  2022-04-22 16:30 ` [bug#55067] [PATCH v2] " Paul A. Patience
  2022-04-22 17:25 ` [bug#55067] [PATCH] " Maxime Devos
@ 2022-04-25 19:10 ` Paul A. Patience
  2022-04-26  8:45   ` bug#55067: [PATCH] " Mathieu Othacehe
  2 siblings, 1 reply; 7+ messages in thread
From: Paul A. Patience @ 2022-04-25 19:10 UTC (permalink / raw)
  To: 55067; +Cc: Paul A. Patience

* gnu/packages/maths.scm (nomad-optimizer)[arguments]: Disable the
examples/advanced/library/PSDMads test.
---
I've added a link to the issue I opened upstream regarding this failing test,
as suggested by Maxime Devos.

 gnu/packages/maths.scm | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f4778179d6..448cdb4d73 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -45,7 +45,7 @@
 ;;; Copyright © 2021 Gerd Heber <gerd.heber@gmail.com>
 ;;; Copyright © 2021 Franck Pérignon <franck.perignon@univ-grenoble-alpes.fr>
 ;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
-;;; Copyright © 2021 Paul A. Patience <paul@apatience.com>
+;;; Copyright © 2021-2022 Paul A. Patience <paul@apatience.com>
 ;;; Copyright © 2021 Ivan Gankevich <i.gankevich@spbu.ru>
 ;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me>
 ;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
@@ -2008,6 +2008,10 @@ (define-public nomad-optimizer
          (add-after 'fix-sources-for-build 'fix-sources-for-tests
            (lambda _
              (substitute* "examples/CMakeLists.txt"
+               ;; This test passes only sometimes.
+               ;; See https://github.com/bbopt/nomad/issues/72.
+               (("^ +add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/advanced/library/PSDMads\\)\n")
+                "")
                ;; examples/basic/batch/example3 is accidentally omitted.
                (("^(add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/basic/batch/example)2(\\)\n)"
                  _ prefix suffix)
@@ -2052,10 +2056,6 @@ (define-public nomad-optimizer
               "examples/advanced/library/exampleSuggestAndObserve/cache0.txt")

              (let ((builddir (string-append (getcwd) "/../build")))
-               ;; For some reason, omitting this fix causes the
-               ;; examples/advanced/library/PSDMads test to fail, even though
-               ;; it doesn't seem to reference any part of the FixedVariable
-               ;; test.
                (let ((dir "examples/advanced/library/FixedVariable"))
                  (substitute* (string-append dir "/fixedVariable.cpp")
                    (("^( +std::string sExe = ).*" _ prefix)
--
2.35.1






^ permalink raw reply related	[flat|nested] 7+ messages in thread

* bug#55067: [PATCH] gnu: nomad-optimizer: Disable failing test.
  2022-04-25 19:10 ` [bug#55067] [PATCH v3] " Paul A. Patience
@ 2022-04-26  8:45   ` Mathieu Othacehe
  0 siblings, 0 replies; 7+ messages in thread
From: Mathieu Othacehe @ 2022-04-26  8:45 UTC (permalink / raw)
  To: Paul A. Patience; +Cc: 55067-done


> * gnu/packages/maths.scm (nomad-optimizer)[arguments]: Disable the
> examples/advanced/library/PSDMads test.

Pushed, thanks Paul & Maxime!

Mathieu




^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2022-04-26  8:46 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-04-22 16:20 [bug#55067] [PATCH] gnu: nomad-optimizer: Disable failing test Paul A. Patience
2022-04-22 16:30 ` [bug#55067] [PATCH v2] " Paul A. Patience
2022-04-22 17:25 ` [bug#55067] [PATCH] " Maxime Devos
2022-04-22 19:02   ` Paul A. Patience
2022-04-22 21:37     ` Maxime Devos
2022-04-25 19:10 ` [bug#55067] [PATCH v3] " Paul A. Patience
2022-04-26  8:45   ` bug#55067: [PATCH] " Mathieu Othacehe

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.