[bug#75400] [PATCH] Fixing python-spacy

[bug#75400] [PATCH] Fixing python-spacy

[Alice BRENON]

[-- Attachment #1: Type: text/plain, Size: 3112 bytes --]

Hi all,

The package `python-spacy` seems to have been broken at least partially by the
recent changes in python tooling. The attached patch is a very short first step
into fixing it. This message doesn't hope to get it accepted but is rather meant
to open the discussion on the topic.

- the first issue is naturally the new required dependencies in `native-inputs`
  mentioned in bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many
  python packages commit these past weeks. Surprisingly, I managed to compile
  `python-spacy` by adding only `python-wheel`, without `python-setuptools`.
  What makes both required? Are we sure that both were required in each and
  every python package recently modified?

- the second issue which I can't link with the changes in our tooling is caused
  by two packages which tests have broken: `python-aws-xray-sdk` and
  `python-jose`. First one seems to have a problem in sqlalchemy version (it
  seems to be using too recent a syntax; which is weird considering the fact
  that it uses version 2 and that it used to work — `python-aws-xray-sdk` hasn't
  been modified since cec543e7 on april the 27th 2023!). Second one now has
  trouble checking several certificates for mere formatting reasons (expected
  and actual PEMs are almost the same, only newlines are different; fixing the
  code of the test itself, another one breaks somewhere else. It seems that
  these source codes aren't sturdy enough and don't work at all in the package
  environment guix provides.

- these packages are dependencies of `python-moto`. Removing them from the
  dependencies, many tests break as expected for lack of these modules, but
  disabling them isn't enough: I played a game of whack-a-mole yesterday trying
  to get the tests to pass. Strangely, whereas testing with hypothesis runs all
  the tests and then reports all that have failed, fixing the one reported and
  running the tests again doesn't help: new tests break in different files that
  were apparently fine before… I had three iterations of this process, each one
  consuming half and hour on my machine. As far as I am concerned, these tests
  are unreliable and don't bring any additional confidence to the package. I
  have disabled them to test the fix for `python-spacy` and have been able to
  compile it and use the package.

Appart from that, everything looks ok. `python-moto` appears as a dependency of
`python-smart-open` which itself is both a direct dependency of `python-spacy`
and a dependency of `python-pathy` which is also a dependency of `python-spacy`.

python-smart-open -> python-moto
python-pathy -> python-smart-open
python-spacy -> python-smart-open, python-pathy

I understand that disabling the tests on a package entirely isn't a very sound
practice for a distribution so I doubt this is an acceptable fix. As I
understand the situation the next steps will be to sort things between
`python-aws-xray-sdk` and `python-sqlalchemy` and to understand why the tests in
`python-jose` have suddenly broken so bad.

Best,

Alice

[-- Attachment #2: 0001-gnu-python-spacy-Fix-build.patch --]
[-- Type: text/x-patch, Size: 1216 bytes --]

From 9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922 Mon Sep 17 00:00:00 2001
Message-ID: <9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922.1736155491.git.alice.brenon@ens-lyon.fr>
From: Alice BRENON <alice.brenon@ens-lyon.fr>
Date: Sat, 4 Jan 2025 23:01:50 +0100
Subject: [PATCH] gnu: python-spacy: Fix build.

* gnu/packages/machine-learning.scm (python-spacy): Fix build.
[native-inputs]: Add python-wheel

Change-Id: I6550f6dc0cb1fb88fc733f6471f1831c4ba95977
---
 gnu/packages/machine-learning.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c7d336b8b3..1617ef917e 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -1282,7 +1282,7 @@ (define-public python-spacy
                              python-typing-extensions
                              python-wasabi))
     (native-inputs
-     (list python-cython python-pytest python-mock))
+     (list python-cython python-pytest python-mock python-wheel))
     (home-page "https://spacy.io")
     (synopsis "Natural Language Processing (NLP) in Python")
     (description

base-commit: 7a7c01b393ecb20dff142b9ea9bf8317e994bef2
-- 
2.47.1

bug#75400: [PATCH] Fixing python-spacy

[Ludovic Courtès]

Hello Alice,

Alice BRENON <alice.brenon@ens-lyon.fr> skribis:

>>From 9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922 Mon Sep 17 00:00:00 2001
> Message-ID: <9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922.1736155491.git.alice.brenon@ens-lyon.fr>
> From: Alice BRENON <alice.brenon@ens-lyon.fr>
> Date: Sat, 4 Jan 2025 23:01:50 +0100
> Subject: [PATCH] gnu: python-spacy: Fix build.
>
> * gnu/packages/machine-learning.scm (python-spacy): Fix build.
> [native-inputs]: Add python-wheel
>
> Change-Id: I6550f6dc0cb1fb88fc733f6471f1831c4ba95977

Applied.  Thanks for the debugging session and thorough analysis.

Cc’ing the Python team for feedback on your other comments:

  https://issues.guix.gnu.org/75400

Ludo’.

[bug#75400] [PATCH] Fixing python-spacy

[Lars-Dominik Braun]

Hi Alice,

via:

> Cc’ing the Python team for feedback on your other comments:
> 
>   https://issues.guix.gnu.org/75400

> - the first issue is naturally the new required dependencies in `native-inputs`
>   mentioned in bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many
>   python packages commit these past weeks. Surprisingly, I managed to compile
>   `python-spacy` by adding only `python-wheel`, without `python-setuptools`.
>   What makes both required? Are we sure that both were required in each and
>   every python package recently modified?

in short: yes. I can’t tell for sure for this exact case because
the output of `guix graph` is too big for `xdot` to handle, but it’s
likely python-setuptools is implicitly propagated through some dependency
(perhaps a bug like https://issues.guix.gnu.org/25235) to python-spacy
and thus it looks like it’s “not required”.

Lars

[bug#75400] [PATCH] Fixing python-spacy

[Alice BRENON]

Hi Lars!

Thanks for the great insight. Yeah, I tried generating that graph when I started
the investigation because at first I couldn't understand where the crashing
version of `python-moto` was still being used… ^^' I got a PNG over 25Mo. I
ended up browsing the dot graph in text directly. Ok, so I think I'll make
another patch to add setuptools (can I send the patch to this closed issue or
will I need to open a new one?).

Any opinion on all the broken tests, anyone?

Alice

Le Tue, 7 Jan 2025 15:18:32 +0100,
Lars-Dominik Braun <lars@6xq.net> a écrit :

> Hi Alice,
> 
> via:
> 
> > Cc’ing the Python team for feedback on your other comments:
> > 
> >   https://issues.guix.gnu.org/75400  
> 
> > - the first issue is naturally the new required dependencies in
> > `native-inputs` mentioned in
> > bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many python
> > packages commit these past weeks. Surprisingly, I managed to
> > compile `python-spacy` by adding only `python-wheel`, without
> > `python-setuptools`. What makes both required? Are we sure that
> > both were required in each and every python package recently
> > modified?  
> 
> in short: yes. I can’t tell for sure for this exact case because
> the output of `guix graph` is too big for `xdot` to handle, but it’s
> likely python-setuptools is implicitly propagated through some
> dependency (perhaps a bug
> like https://issues.guix.gnu.org/25235) to python-spacy and thus it looks like it’s “not required”.
> 
> Lars
> 

[bug#75400] [PATCH] Fixing python-spacy

[Ludovic Courtès]

Alice BRENON <alice.brenon@ens-lyon.fr> skribis:

> Thanks for the great insight. Yeah, I tried generating that graph when I started
> the investigation because at first I couldn't understand where the crashing
> version of `python-moto` was still being used… ^^' I got a PNG over 25Mo.

Note that you use, say, ‘guix graph -M3’ to cut the graph at distance 3.
Or you can use things like ‘guix graph --path python-spacy python-moto’.

Ludo’.

[bug#75400] [PATCH] Fixing python-spacy

[jgart via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 258 bytes --]

> Note that you use, say, ‘guix graph -M3’ to cut the graph at distance 3.


Hi, should the CLI documentation for guix graph mention that graphviz is the default backend?

For example, when calling `guix graph --list-backends`?

all best,

jgart

[-- Attachment #2: Type: text/html, Size: 510 bytes --]