[bug#70936] [PATCH gnome-team 0/3] Fix some broken tests.

[bug#70936] [PATCH gnome-team 0/3] Fix some broken tests.

[Liliana Marie Prikler]

QA seems to dislike at least some major packages with our recent changes.
It's quite possible that some of them are flaky tests.

Liliana Marie Prikler (3):
  gnu: gst-plugins-bad: Ignore elements_netsim test.
  gnu: qtbase: Ignore tst_qsqlthread.
  gnu: qemu: Disable more tests.

 gnu/packages/gstreamer.scm      | 4 ++++
 gnu/packages/qt.scm             | 3 +++
 gnu/packages/virtualization.scm | 7 ++++++-
 3 files changed, 13 insertions(+), 1 deletion(-)


base-commit: 9185efbc6cdaf8f7d83bf8fc90f44e25131b9f88
-- 
2.41.0

[bug#70936] [PATCH gnome-team 1/3] gnu: gst-plugins-bad: Ignore elements_netsim test.

[Liliana Marie Prikler]

* gnu/packages/gstreamer.scm (gst-plugins-bad)[adjust-tests]: Add
“elements/netsim.c” to the list of ignored test files.
---
 gnu/packages/gstreamer.scm | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/gnu/packages/gstreamer.scm b/gnu/packages/gstreamer.scm
index 2705e245ea..ad08285181 100644
--- a/gnu/packages/gstreamer.scm
+++ b/gnu/packages/gstreamer.scm
@@ -796,6 +796,10 @@ (define-public gst-plugins-bad
                   ;; The 'elements_curlhttpsrc' test sometimes times out.
                   ((".*'elements/curlhttpsrc\\.c'.*") "")
 
+                  ;; Unexpected critical/warning, see
+                  ;; <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3000>
+                  ((".*'elements/netsim\\.c'.*") "")
+
                   ;; TODO: Figure out why this test fails on riscv64-linux.
                   #$@(if (target-riscv64?)
                          `((("'elements/viewfinderbin\\.c'\\].*],")
-- 
2.41.0

[bug#70936] [PATCH gnome-team 1/3] gnu: gst-plugins-bad: Ignore elements_netsim test.

[Maxim Cournoyer]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * gnu/packages/gstreamer.scm (gst-plugins-bad)[adjust-tests]: Add
> “elements/netsim.c” to the list of ignored test files.

Reviewed-by: Maxim Cournoyer <maxim.cournoyer@gmail>

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Liliana Marie Prikler]

* gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to the list of
ignored tests.
---
 gnu/packages/qt.scm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 13f54fcc5d..1416c3a36c 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -899,6 +899,9 @@ (define-public qtbase
                        ;; TODO: when core-updates is merged, check again.
                        "tst_selftests"
 
+                       ;; The 'tst_qsqlthread' test sometimes fails.
+                       "tst_qsqlthread"
+
                        ;; The 'tst_qsystemsemaphore' test sometimes fails.
                        "tst_qsystemsemaphore"
                        ;; The 'tst_moc' test fails with "'fi.exists()' returned FALSE".
-- 
2.41.0

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Maxim Cournoyer]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to the list of
> ignored tests.
> ---
>  gnu/packages/qt.scm | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
> index 13f54fcc5d..1416c3a36c 100644
> --- a/gnu/packages/qt.scm
> +++ b/gnu/packages/qt.scm
> @@ -899,6 +899,9 @@ (define-public qtbase
>                         ;; TODO: when core-updates is merged, check again.
>                         "tst_selftests"
>  
> +                       ;; The 'tst_qsqlthread' test sometimes fails.
> +                       "tst_qsqlthread"
> +

This was reported by myself [0] and is currently skipped only for
target-ppc64le?.  We should probably skip it for every architecture if
you've encountered it on x86_64 (I'm assuming).

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Maxim Cournoyer]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to the list of
> ignored tests.
> ---
>  gnu/packages/qt.scm | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
> index 13f54fcc5d..1416c3a36c 100644
> --- a/gnu/packages/qt.scm
> +++ b/gnu/packages/qt.scm
> @@ -899,6 +899,9 @@ (define-public qtbase
>                         ;; TODO: when core-updates is merged, check again.
>                         "tst_selftests"
>  
> +                       ;; The 'tst_qsqlthread' test sometimes fails.
> +                       "tst_qsqlthread"
> +

This was reported by myself [0] and is currently skipped only for
target-ppc64le?.  We should probably skip it for every architecture if
you've encountered it on x86_64 (I'm assuming).

[0]  https://bugreports.qt.io/browse/QTBUG-117114?jql=text%20~%20%22tst_qsqlthread%22

Could you adjust and send a v2?

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Liliana Marie Prikler]

Am Dienstag, dem 14.05.2024 um 21:34 -0400 schrieb Maxim Cournoyer:
> Hi Liliana,
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> 
> > * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to the
> > list of
> > ignored tests.
> > ---
> >  gnu/packages/qt.scm | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
> > index 13f54fcc5d..1416c3a36c 100644
> > --- a/gnu/packages/qt.scm
> > +++ b/gnu/packages/qt.scm
> > @@ -899,6 +899,9 @@ (define-public qtbase
> >                         ;; TODO: when core-updates is merged, check
> > again.
> >                         "tst_selftests"
> >  
> > +                       ;; The 'tst_qsqlthread' test sometimes
> > fails.
> > +                       "tst_qsqlthread"
> > +
> 
> This was reported by myself [0] and is currently skipped only for
> target-ppc64le?.  We should probably skip it for every architecture
> if you've encountered it on x86_64 (I'm assuming).
> 
> [0] 
> https://bugreports.qt.io/browse/QTBUG-117114?jql=text%20~%20%22tst_qsqlthread%22
Should I rebase gnome-team on master again, so that I can drop the
condition?

Cheers

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Maxim Cournoyer]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Am Dienstag, dem 14.05.2024 um 21:34 -0400 schrieb Maxim Cournoyer:
>> Hi Liliana,
>> 
>> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
>> 
>> > * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to the
>> > list of
>> > ignored tests.
>> > ---
>> >  gnu/packages/qt.scm | 3 +++
>> >  1 file changed, 3 insertions(+)
>> > 
>> > diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
>> > index 13f54fcc5d..1416c3a36c 100644
>> > --- a/gnu/packages/qt.scm
>> > +++ b/gnu/packages/qt.scm
>> > @@ -899,6 +899,9 @@ (define-public qtbase
>> >                         ;; TODO: when core-updates is merged, check
>> > again.
>> >                         "tst_selftests"
>> >  
>> > +                       ;; The 'tst_qsqlthread' test sometimes
>> > fails.
>> > +                       "tst_qsqlthread"
>> > +
>> 
>> This was reported by myself [0] and is currently skipped only for
>> target-ppc64le?.  We should probably skip it for every architecture
>> if you've encountered it on x86_64 (I'm assuming).
>> 
>> [0] 
>> https://bugreports.qt.io/browse/QTBUG-117114?jql=text%20~%20%22tst_qsqlthread%22
> Should I rebase gnome-team on master again, so that I can drop the
> condition?

Sounds reasonable, if you didn't have that conditional skip on
gnome-team already.

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Liliana Marie Prikler]

Am Donnerstag, dem 16.05.2024 um 08:44 -0400 schrieb Maxim Cournoyer:
> Hi Liliana,
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> 
> > Am Dienstag, dem 14.05.2024 um 21:34 -0400 schrieb Maxim Cournoyer:
> > > Hi Liliana,
> > > 
> > > Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> > > 
> > > > * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to
> > > > the
> > > > list of
> > > > ignored tests.
> > > > ---
> > > >  gnu/packages/qt.scm | 3 +++
> > > >  1 file changed, 3 insertions(+)
> > > > 
> > > > diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
> > > > index 13f54fcc5d..1416c3a36c 100644
> > > > --- a/gnu/packages/qt.scm
> > > > +++ b/gnu/packages/qt.scm
> > > > @@ -899,6 +899,9 @@ (define-public qtbase
> > > >                         ;; TODO: when core-updates is merged,
> > > > check
> > > > again.
> > > >                         "tst_selftests"
> > > >  
> > > > +                       ;; The 'tst_qsqlthread' test sometimes
> > > > fails.
> > > > +                       "tst_qsqlthread"
> > > > +
> > > 
> > > This was reported by myself [0] and is currently skipped only for
> > > target-ppc64le?.  We should probably skip it for every
> > > architecture
> > > if you've encountered it on x86_64 (I'm assuming).
> > > 
> > > [0] 
> > > https://bugreports.qt.io/browse/QTBUG-117114?jql=text%20~%20%22tst_qsqlthread%22
> > Should I rebase gnome-team on master again, so that I can drop the
> > condition?
> 
> Sounds reasonable, if you didn't have that conditional skip on
> gnome-team already.
I don't see it even after the rebase… is that on master yet or
elsewhere?

Anyhow, I think I forgot to link the bug report appropriately before
pushing, will do so when CI lights green.

Cheers

[bug#70936] [PATCH gnome-team 2/3] gnu: qtbase: Ignore tst_qsqlthread.

[Maxim Cournoyer]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Am Donnerstag, dem 16.05.2024 um 08:44 -0400 schrieb Maxim Cournoyer:
>> Hi Liliana,
>> 
>> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
>> 
>> > Am Dienstag, dem 14.05.2024 um 21:34 -0400 schrieb Maxim Cournoyer:
>> > > Hi Liliana,
>> > > 
>> > > Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
>> > > 
>> > > > * gnu/packages/qt.scm (qtbase)[check]: Add “tst_qsqlthread” to
>> > > > the
>> > > > list of
>> > > > ignored tests.
>> > > > ---
>> > > >  gnu/packages/qt.scm | 3 +++
>> > > >  1 file changed, 3 insertions(+)
>> > > > 
>> > > > diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
>> > > > index 13f54fcc5d..1416c3a36c 100644
>> > > > --- a/gnu/packages/qt.scm
>> > > > +++ b/gnu/packages/qt.scm
>> > > > @@ -899,6 +899,9 @@ (define-public qtbase
>> > > >                         ;; TODO: when core-updates is merged,
>> > > > check
>> > > > again.
>> > > >                         "tst_selftests"
>> > > >  
>> > > > +                       ;; The 'tst_qsqlthread' test sometimes
>> > > > fails.
>> > > > +                       "tst_qsqlthread"
>> > > > +
>> > > 
>> > > This was reported by myself [0] and is currently skipped only for
>> > > target-ppc64le?.  We should probably skip it for every
>> > > architecture
>> > > if you've encountered it on x86_64 (I'm assuming).
>> > > 
>> > > [0] 
>> > > https://bugreports.qt.io/browse/QTBUG-117114?jql=text%20~%20%22tst_qsqlthread%22
>> > Should I rebase gnome-team on master again, so that I can drop the
>> > condition?
>> 
>> Sounds reasonable, if you didn't have that conditional skip on
>> gnome-team already.
> I don't see it even after the rebase… is that on master yet or
> elsewhere?
>
> Anyhow, I think I forgot to link the bug report appropriately before
> pushing, will do so when CI lights green.

It's from commit 625fa5745e3, dated 26th of March 2024.

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Liliana Marie Prikler]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

* gnu/packages/virtualization.scm (qemu)[disable-unusable-tests]: Also disable
qtest/ahci-test and qemu-iotests/copy-before-write.
---
 gnu/packages/virtualization.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 34cccd6550..5389af62f0 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -305,7 +305,12 @@ (define-public qemu
               (substitute* "tests/qtest/meson.build"
                 ;; These tests fail to get the expected number of tests
                 ;; on arm platforms.
-                (("'arm-cpu-features',") ""))))
+                (("'arm-cpu-features',") "")
+                ;; XXX: Why does this (sometimes) fail?
+                (("\\['ahci-test'\\]") "[]"))
+              ;; This test appears to be missing permissions.
+              ;; Very sus 🤔
+              (delete-file "tests/qemu-iotests/tests/copy-before-write")))
           #$@(if (target-riscv64?)
                  '((add-after 'unpack 'disable-some-tests
                      (lambda _
-- 
2.41.0

[bug#70936] [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Maxim Cournoyer]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * gnu/packages/virtualization.scm (qemu)[disable-unusable-tests]: Also disable
> qtest/ahci-test and qemu-iotests/copy-before-write.
> ---
>  gnu/packages/virtualization.scm | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
> index 34cccd6550..5389af62f0 100644
> --- a/gnu/packages/virtualization.scm
> +++ b/gnu/packages/virtualization.scm
> @@ -305,7 +305,12 @@ (define-public qemu
>                (substitute* "tests/qtest/meson.build"
>                  ;; These tests fail to get the expected number of tests
>                  ;; on arm platforms.
> -                (("'arm-cpu-features',") ""))))
> +                (("'arm-cpu-features',") "")
> +                ;; XXX: Why does this (sometimes) fail?

Someone reported it already upstream:
<https://gitlab.com/qemu-project/qemu/-/issues/2121>.  Please add a
reference to it.

> +                (("\\['ahci-test'\\]") "[]"))
> +              ;; This test appears to be missing permissions.
> +              ;; Very sus 🤔

How can it be missing permissions if it is sometimes working?  Perhaps
more something racy where the file hasn't been created yet.  I'd
personally rather not see emojis or comments of dubious utility ("Very
suspect") in the source.  Perhaps add a short excerpt of the error for
the next curious person.

> +              (delete-file "tests/qemu-iotests/tests/copy-before-write")))
>            #$@(if (target-riscv64?)
>                   '((add-after 'unpack 'disable-some-tests
>                       (lambda _

Otherwise, LGTM!

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Maxim Cournoyer]

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> * gnu/packages/virtualization.scm (qemu)[disable-unusable-tests]: Also disable
> qtest/ahci-test and qemu-iotests/copy-before-write.
> ---
>  gnu/packages/virtualization.scm | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
> index 34cccd6550..5389af62f0 100644
> --- a/gnu/packages/virtualization.scm
> +++ b/gnu/packages/virtualization.scm
> @@ -305,7 +305,12 @@ (define-public qemu
>                (substitute* "tests/qtest/meson.build"
>                  ;; These tests fail to get the expected number of tests
>                  ;; on arm platforms.
> -                (("'arm-cpu-features',") ""))))
> +                (("'arm-cpu-features',") "")
> +                ;; XXX: Why does this (sometimes) fail?

Someone reported it already upstream:
<https://gitlab.com/qemu-project/qemu/-/issues/2121>.  Please add a
reference to it.

> +                (("\\['ahci-test'\\]") "[]"))
> +              ;; This test appears to be missing permissions.
> +              ;; Very sus 🤔

How can it be missing permissions if it is sometimes working?  Perhaps
more something racy where the file hasn't been created yet.  I'd
personally rather not see emojis or comments of dubious utility ("Very
suspect") in the source.  Perhaps add a short excerpt of the error for
the next curious person.

> +              (delete-file "tests/qemu-iotests/tests/copy-before-write")))
>            #$@(if (target-riscv64?)
>                   '((add-after 'unpack 'disable-some-tests
>                       (lambda _

Otherwise, LGTM, with the added reference to the upstream issue.  Thanks
for taking the time to address flaky tests!  They are a real drag when
left unchecked.

-- 
Thanks,
Maxim

[bug#70936] [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Maxim Cournoyer]

Hello!

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
>
>> * gnu/packages/virtualization.scm (qemu)[disable-unusable-tests]: Also disable
>> qtest/ahci-test and qemu-iotests/copy-before-write.
>> ---
>>  gnu/packages/virtualization.scm | 7 ++++++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
>> index 34cccd6550..5389af62f0 100644
>> --- a/gnu/packages/virtualization.scm
>> +++ b/gnu/packages/virtualization.scm
>> @@ -305,7 +305,12 @@ (define-public qemu
>>                (substitute* "tests/qtest/meson.build"
>>                  ;; These tests fail to get the expected number of tests
>>                  ;; on arm platforms.
>> -                (("'arm-cpu-features',") ""))))
>> +                (("'arm-cpu-features',") "")
>> +                ;; XXX: Why does this (sometimes) fail?
>
> Someone reported it already upstream:
> <https://gitlab.com/qemu-project/qemu/-/issues/2121>.  Please add a
> reference to it.
>
>> +                (("\\['ahci-test'\\]") "[]"))
>> +              ;; This test appears to be missing permissions.
>> +              ;; Very sus 🤔
>
> How can it be missing permissions if it is sometimes working?  Perhaps
> more something racy where the file hasn't been created yet.  I'd
> personally rather not see emojis or comments of dubious utility ("Very
> suspect") in the source.  Perhaps add a short excerpt of the error for
> the next curious person.
>
>> +              (delete-file "tests/qemu-iotests/tests/copy-before-write")))
>>            #$@(if (target-riscv64?)
>>                   '((add-after 'unpack 'disable-some-tests
>>                       (lambda _
>
> Otherwise, LGTM, with the added reference to the upstream issue.  Thanks
> for taking the time to address flaky tests!  They are a real drag when
> left unchecked.

Ping, in case it had fallen in cracks :-).

-- 
Thanks,
Maxim

bug#70936: [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Liliana Marie Prikler]

Am Sonntag, dem 23.06.2024 um 23:14 -0400 schrieb Maxim Cournoyer:
> Ping, in case it had fallen in cracks :-).
It hasn't, I just forgot to mark it as pushed.

Cheers

[bug#70936] [PATCH gnome-team 3/3] gnu: qemu: Disable more tests.

[Maxim Cournoyer]

Hello,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Am Sonntag, dem 23.06.2024 um 23:14 -0400 schrieb Maxim Cournoyer:
>> Ping, in case it had fallen in cracks :-).
> It hasn't, I just forgot to mark it as pushed.

Excellent.

-- 
Thanks,
Maxim