python-duckdb stuck in its tests

python-duckdb stuck in its tests

[Andreas Enge]

Hello the Python team,

python-duckdb sticks out on the build farm, since it apparently gets blocked
in its tests after spending quite some CPU time for building, and finally
it times out after a day. It seems to happen consistently over several
evaluations. The last lines are:
tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas0] PASSED [  8%]
tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas1] PASSED [  8%]
tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas0] PASSED [  8%]
tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas1] 

(see https://bordeaux.guix.gnu.org/build/56359106-e402-49d2-8c6b-2f35de90b7da/log)

Could you maybe have a look?

Andreas

Re: python-duckdb stuck in its tests

[Sharlatan Hellseher]

Hi Andreas,

It looks like updating to 1.0.0 has not issue wit passing test on my
local checkout after applying this patch

--8<---------------cut here---------------start------------->8---
@@ -23334,20 +23334,24 @@ (define-public python-chevron
 (define-public python-duckdb
   (package
     (name "python-duckdb")
-    (version "0.8.1")
+    (version "1.0.0")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "duckdb" version))
               (sha256
                (base32
-                "1sgfmii5xlkbx3hzyjxg80gl2ni1rxpabahl4gww9by2mgs3fkd5"))))
+                "0lyl6di1c7j31i2mk384j711kzyyf9rjd3nqx5mbgmf7gfvmk852"))))
     (build-system pyproject-build-system)
     (arguments
      (list
       #:test-flags
       '(list "--ignore=tests/slow/test_h2oai_arrow.py"
-             ;; Don't install anything, thank you.
-             "-k" "not test_install_non_existent_extension")
+             "-k" (string-append
+                   ;; Don't install anything, thank you.
+                   "not test_install_non_existent_extension"
+                   ;; assert not ["error: duckdb failed to find <..>
+                   ;; site-packages/duckdb/__init__.py
+                   " and not test_generated_stubs"))
       #:phases
       #~(modify-phases %standard-phases
           ;; Tests need this
--8<---------------cut here---------------end--------------->8---

But Sanity check is not happy with:
--8<---------------cut here---------------start------------->8---
starting phase `sanity-check'
validating 'duckdb'
/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages
...checking requirements: OK
...trying to load module adbc_driver_duckdb: ERROR:
Traceback (most recent call last):
  File "/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py",
line 73, in <module>
    importlib.import_module(name)
  File "/gnu/store/4ncpi13lpwj8fk3j7adgnr5mi90dz311-python-3.10.7/lib/python3.10/importlib/__init__.py",
line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages/adbc_driver_duckdb/__init__.py",
line 24, in <module>
    import adbc_driver_manager
ModuleNotFoundError: No module named 'adbc_driver_manager'
...trying to load module duckdb: OK
...trying to load module duckdb-stubs: OK
error: in phase 'sanity-check': uncaught exception:
%exception #<&invoke-error program: "python" arguments:
("/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py"
"/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages")
exit-status: 1 term-signal: #f stop-signal: #f>
phase `sanity-check' failed after 0.2 seconds
command "python"
"/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py"
"/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages"
failed with status 1
builder for `/gnu/store/8i8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv'
failed with exit code 1
build of /gnu/store/8i8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv
failed
View build log at
'/var/log/guix/drvs/8i/8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv.gz'.
--8<---------------cut here---------------end--------------->8---

Oleg

On Sun, 30 Jun 2024 at 09:23, Andreas Enge <andreas@enge.fr> wrote:
>
> Hello the Python team,
>
> python-duckdb sticks out on the build farm, since it apparently gets blocked
> in its tests after spending quite some CPU time for building, and finally
> it times out after a day. It seems to happen consistently over several
> evaluations. The last lines are:
> tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas0] PASSED [  8%]
> tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas1] PASSED [  8%]
> tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas0] PASSED [  8%]
> tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas1]
>
> (see https://bordeaux.guix.gnu.org/build/56359106-e402-49d2-8c6b-2f35de90b7da/log)
>
> Could you maybe have a look?
>
> Andreas
>


-- 
VCS: https://github.incerto.xyz/; https://git.sr.ht/~hellseher/
GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5

… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

Re: python-duckdb stuck in its tests

[Sharlatan Hellseher]

Hi,

I've pushed update togather with https://issues.guix.gnu.org/71480. It
 fixed the build, check and sanity check phases in
ce98c3436c57e7b366a3ec06c47a7e8919c990fb.

Thanks,
Oleg

On Sun, 30 Jun 2024 at 11:02, Sharlatan Hellseher <sharlatanus@gmail.com> wrote:
>
> Hi Andreas,
>
> It looks like updating to 1.0.0 has not issue wit passing test on my
> local checkout after applying this patch
>
> --8<---------------cut here---------------start------------->8---
> @@ -23334,20 +23334,24 @@ (define-public python-chevron
>  (define-public python-duckdb
>    (package
>      (name "python-duckdb")
> -    (version "0.8.1")
> +    (version "1.0.0")
>      (source (origin
>                (method url-fetch)
>                (uri (pypi-uri "duckdb" version))
>                (sha256
>                 (base32
> -                "1sgfmii5xlkbx3hzyjxg80gl2ni1rxpabahl4gww9by2mgs3fkd5"))))
> +                "0lyl6di1c7j31i2mk384j711kzyyf9rjd3nqx5mbgmf7gfvmk852"))))
>      (build-system pyproject-build-system)
>      (arguments
>       (list
>        #:test-flags
>        '(list "--ignore=tests/slow/test_h2oai_arrow.py"
> -             ;; Don't install anything, thank you.
> -             "-k" "not test_install_non_existent_extension")
> +             "-k" (string-append
> +                   ;; Don't install anything, thank you.
> +                   "not test_install_non_existent_extension"
> +                   ;; assert not ["error: duckdb failed to find <..>
> +                   ;; site-packages/duckdb/__init__.py
> +                   " and not test_generated_stubs"))
>        #:phases
>        #~(modify-phases %standard-phases
>            ;; Tests need this
> --8<---------------cut here---------------end--------------->8---
>
> But Sanity check is not happy with:
> --8<---------------cut here---------------start------------->8---
> starting phase `sanity-check'
> validating 'duckdb'
> /gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages
> ...checking requirements: OK
> ...trying to load module adbc_driver_duckdb: ERROR:
> Traceback (most recent call last):
>   File "/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py",
> line 73, in <module>
>     importlib.import_module(name)
>   File "/gnu/store/4ncpi13lpwj8fk3j7adgnr5mi90dz311-python-3.10.7/lib/python3.10/importlib/__init__.py",
> line 126, in import_module
>     return _bootstrap._gcd_import(name[level:], package, level)
>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
>   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
>   File "/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages/adbc_driver_duckdb/__init__.py",
> line 24, in <module>
>     import adbc_driver_manager
> ModuleNotFoundError: No module named 'adbc_driver_manager'
> ...trying to load module duckdb: OK
> ...trying to load module duckdb-stubs: OK
> error: in phase 'sanity-check': uncaught exception:
> %exception #<&invoke-error program: "python" arguments:
> ("/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py"
> "/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages")
> exit-status: 1 term-signal: #f stop-signal: #f>
> phase `sanity-check' failed after 0.2 seconds
> command "python"
> "/gnu/store/iqsjkp55pcx5bfcp2jm9yj5rlx9a0whd-sanity-check.py"
> "/gnu/store/cvr5g1ivssavn3v5bhnbnpkm2zpwyj2s-python-duckdb-1.0.0/lib/python3.10/site-packages"
> failed with status 1
> builder for `/gnu/store/8i8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv'
> failed with exit code 1
> build of /gnu/store/8i8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv
> failed
> View build log at
> '/var/log/guix/drvs/8i/8z6fynmyg35x48dln3lgl1z1vqiygy-python-duckdb-1.0.0.drv.gz'.
> --8<---------------cut here---------------end--------------->8---
>
> Oleg
>
> On Sun, 30 Jun 2024 at 09:23, Andreas Enge <andreas@enge.fr> wrote:
> >
> > Hello the Python team,
> >
> > python-duckdb sticks out on the build farm, since it apparently gets blocked
> > in its tests after spending quite some CPU time for building, and finally
> > it times out after a day. It seems to happen consistently over several
> > evaluations. The last lines are:
> > tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas0] PASSED [  8%]
> > tests/fast/test_multithread.py::TestDuckMultithread::test_transaction[pandas1] PASSED [  8%]
> > tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas0] PASSED [  8%]
> > tests/fast/test_multithread.py::TestDuckMultithread::test_df_append[pandas1]
> >
> > (see https://bordeaux.guix.gnu.org/build/56359106-e402-49d2-8c6b-2f35de90b7da/log)
> >
> > Could you maybe have a look?
> >
> > Andreas
> >
>
>
> --
> VCS: https://github.incerto.xyz/; https://git.sr.ht/~hellseher/
> GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5
>
> … наш разум - превосходная объяснительная машина которая способна
> найти смысл почти в чем угодно, истолковать любой феномен, но
> совершенно не в состоянии принять мысль о непредсказуемости.



-- 
VCS: https://github.incerto.xyz/; https://git.sr.ht/~hellseher/
GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5

… наш разум - превосходная объяснительная машина которая способна
найти смысл почти в чем угодно, истолковать любой феномен, но
совершенно не в состоянии принять мысль о непредсказуемости.

Re: python-duckdb stuck in its tests

[Andreas Enge]

Am Mon, Jul 08, 2024 at 11:41:12PM +0100 schrieb Sharlatan Hellseher:
> I've pushed update togather with https://issues.guix.gnu.org/71480. It
>  fixed the build, check and sanity check phases in
> ce98c3436c57e7b366a3ec06c47a7e8919c990fb.

Excellent, thanks a lot!

Andreas