[bug#63139] [PATCH python 00/20] Updates for pyproject-build-system, poetry, python-yubikey-manager

[bug#63139] [PATCH python 00/20] Updates for pyproject-build-system, poetry, python-yubikey-manager

[John Kehayias via Guix-patches via]

Hi Guix and Python team,

Here is a patch series where the original goal was to fix python-yubikey-manager on core-updates and then ballooned to a bit more. This should be done in a feature branch for Python.

Mostly this was to fix/update the needed dependencies, though it may be possible to do this in a more minimal way just for that package fix. Anyway, I tried to generate this series in a way that each patch continues to fix things, but due to the complicated dependencies this may not be perfect.

A few notes:

1. Most of the series is pretty trivial, quick fixes/updates, some new packages.

2. What isn't is a few cases of failing tests which weren't immediately obvious to me and likely were some network access and/or build environment details. Some could be worked around maybe if someone wants to try (e.g. in python-virtualenv). I did enable more tests along the way though (like for poetry), so on the whole I think this is a step forward.

3. The dependents tend to be maybe 10s, a few in the hundreds, and then about 3k for python-filelock. Until we get to pyproject-build-system updates:

4. I believe it was poetry that needed a newer python-pypa-build module, which then touches all pyproject-build-system (about 6k packages). This isn't strictly necessary as we could have a newer and separate package for leafs to use rather than in the build system as well, but I figured might as well do it sooner rather than later. At least the packages up to python-yubikey-manager built with this along with some random others.

5. On that note, I did not complete this change as I wanted some feedback on the bootstraping. I've added python-pyproject-hooks which should deprecate pep517, but currently it also needs python-pypa-build. I've made the older python-pypa-build a -bootstrap package to build this and the newer version of itself as well. So I did not deprecate pep517 yet.

   Also, python-wheel was a propagated-input in pep517 which is not needed in pyproject-hooks. However, I saw at least some packages will then need that as an input to build; so I kept it for pyproject-hooks to ease testing. It should be removed and added as an input as needed (no idea if that is just a few or a lot of the tree).

Okay, I think those are my notes. We should see what other things are ready to be made into this feature branch for Python. One brought to my attention recently is https://issues.guix.gnu.org/63044 though I have not looked at it.

Thanks!
John

John Kehayias (20):
  gnu: Add python-installer.
  gnu: Add python-pyproject-hooks.
  gnu: Add python-rapidfuzz.
  gnu: python-crashtest: Update to 0.4.1.
  gnu: python-cleo: Update to 2.0.1.
  gnu: Add python-deepdiff.
  gnu: python-platformdirs: Update to 3.2.0.
  gnu: python-filelock: Update to 3.12.0.
  gnu: python-distlib: Update to 0.3.6.
  gnu: python-virtualenv: Update to 20.22.0.
  gnu: python-pkginfo: Update to 1.9.6.
  gnu: python-jsonschema: Update to 4.17.3.
  gnu: python-dulwich: Update to 0.21.3.
  gnu: Update python-pypa-build to 1.0.0.
  gnu: poetry: Fix build.
  gnu: Add python-poetry-plugin-export.
  gnu: python-pyscard: Update to 2.0.7.
  gnu: python-fido2: Update to 1.1.1 and enable tests.
  gnu: Add python-makefun.
  gnu: python-yubikey-manager: Update to 5.1.0 and enable tests.

 gnu/packages/python-build.scm   |  80 +++++++-
 gnu/packages/python-xyz.scm     | 319 +++++++++++++++++++++++++-------
 gnu/packages/security-token.scm |  61 +++---
 3 files changed, 370 insertions(+), 90 deletions(-)


base-commit: aecc6e70587f8412cbbb9b2c13141de4f534518e
--
2.39.2

[bug#63139] [PATCH python 01/20] gnu: Add python-installer.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-installer): New variable.
---
 gnu/packages/python-xyz.scm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index e2d082091f..83beb1c8ef 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -1329,6 +1329,24 @@ (define-public python-d2to1
 distribute/setuptools setup.py script.")
     (license license:bsd-2)))

+(define-public python-installer
+  (package
+    (name "python-installer")
+    (version "0.7.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "installer" version))
+              (sha256
+               (base32
+                "0cdnqh3a3amw8k4s1pzfjh0hpvzw4pczgl702s1b16r82qqkwvd2"))))
+    (build-system pyproject-build-system)
+    (native-inputs (list python-flit-core python-pytest))
+    (home-page "https://installer.readthedocs.io/")
+    (synopsis "A library for installing Python wheels.")
+    (description
+     "This package provides a library for installing Python wheels.")
+    (license license:expat)))
+
 (define-public python-rawkit
   (package
     (name "python-rawkit")
--
2.39.2

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[John Kehayias via Guix-patches via]

* gnu/packages/python-build.scm (python-pyproject-hooks): New variable.
---
 gnu/packages/python-build.scm | 41 +++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 70719c44d4..45b9f52b68 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -10,6 +10,7 @@
 ;;; Copyright © 2020, 2021, 2022, 2023 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2022 Garek Dyszel <garekdyszel@disroot.org>
 ;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
+;;; Copyright © 2023 John Kehayias <john.kehayias@protonmail.com>

 ;;;
 ;;; This file is part of GNU Guix.
@@ -381,6 +382,46 @@ (define-public python-pep517-bootstrap
       "Wrappers to build Python packages using PEP 517 hooks.")
      (license license:expat))))

+(define-public python-pyproject-hooks
+  (package
+    (name "python-pyproject-hooks")
+    (version "1.0.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "pyproject_hooks" version))
+              (sha256
+               (base32
+                "1xaf4sharvacqlav6w3b38nc4j0rzg0p4axi7zamanbzp6cb4wgj"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:tests? #f ; disabled to avoid cycle
+           #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'use-toml-instead-of-tomli
+                 ;; Using toml instead of tomli eases bootstrapping.
+                 (lambda _
+                   (substitute* "pyproject.toml"
+                     (("\"tomli.*")
+                      "\"toml\","))))
+               (replace 'build
+                 (lambda _
+                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
+               (replace 'install
+                 (lambda _
+                   (let ((whl (car (find-files "dist" "\\.whl$"))))
+                     (invoke "pip" "--no-cache-dir" "--no-input"
+                             "install" "--no-deps" "--prefix" #$output whl)))))))
+    (native-inputs (list python-flit-core-bootstrap python-pypa-build-bootstrap))
+    ;; XXX: python-wheel isn't needed but replicate the pep517 package or else
+    ;; fix packages which will then need python-whell as an input.
+    (propagated-inputs (list python-toml python-wheel))
+    (home-page "https://pyproject-hooks.readthedocs.io")
+    (synopsis "Wrappers to call pyproject.toml-based build backend hooks")
+    (description "This is a low-level library for calling build-backends in
+@code{pyproject.toml}-based project.  It provides the basic functionality to
+help write tooling that generates distribution files from Python projects.")
+    (license license:expat)))
+
 (define-public python-pyparsing
   (package
     (name "python-pyparsing")
--
2.39.2

[bug#63139] [PATCH python 03/20] gnu: Add python-rapidfuzz.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-rapidfuzz): New variable.
---
 gnu/packages/python-xyz.scm | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 83beb1c8ef..4d1f291e68 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17793,6 +17793,39 @@ (define-public python-pylev
 @url{http://en.wikipedia.org/wiki/Levenshtein_distance}.")
     (license license:bsd-3)))

+(define-public python-rapidfuzz
+  (package
+    (name "python-rapidfuzz")
+    (version "2.15.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "rapidfuzz" version))
+              (sha256
+               (base32
+                "1xh0mkbhgnrwgwhrlnmypwwig3ww23fdffh0245akbiprb13f8fn"))))
+    (build-system python-build-system)
+    (native-inputs (list python-hypothesis python-numpy python-pytest
+                         python-scikit-build))
+    (home-page "https://github.com/maxbachmann/RapidFuzz")
+    (synopsis "Rapid fuzzy string matching using various string metrics")
+    (description
+     "RapidFuzz is a fast string matching library for Python and
+C++, which is using the string similarity calculations from FuzzyWuzzy.
+However there are a couple of aspects that set RapidFuzz apart from
+FuzzyWuzzy:
+@itemize
+@item It is MIT licensed so it can be used whichever License you might want to
+choose for your project, while you're forced to adopt the GPL license when
+using FuzzyWuzzy.
+@item It provides many @code{string_metrics} like @code{hamming} or
+@code{jaro_winkler}, which are not included in FuzzyWuzzy.
+@item It is mostly written in C++ and on top of this comes with a lot of
+algorithmic improvements to make string matching even faster, while still
+providing the same results.  For detailed benchmarks check the documentation.
+@item Fixes multiple bugs in the partial_ratio implementation.
+@end itemize")
+    (license license:expat)))
+
 (define-public python-cleo
   (package
     (name "python-cleo")
--
2.39.2

[bug#63139] [PATCH python 04/20] gnu: python-crashtest: Update to 0.4.1.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-crashtest): Update to 0.4.1.
---
 gnu/packages/python-xyz.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 4d1f291e68..5aac1de1af 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -12133,13 +12133,13 @@ (define-public python-crccheck
 (define-public python-crashtest
   (package
     (name "python-crashtest")
-    (version "0.3.1")
+    (version "0.4.1")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "crashtest" version))
         (sha256
-          (base32 "1p9p7mn8x2j9psc4jxab98897v4i9s4fliyfw8rp8v4bx1n7pjj2"))))
+          (base32 "1kphcr9af50p37j9v5s8p8qblxy8fmi6s1s8yqlx9yzb2vrv3mw0"))))
     (build-system python-build-system)
     (home-page "https://github.com/sdispater/crashtest")
     (synopsis "Manage Python errors with ease")
--
2.39.2

[bug#63139] [PATCH python 05/20] gnu: python-cleo: Update to 2.0.1.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-cleo): Update to 2.0.1.
[propagated-inputs]: Remove python-backpack, python-clikit, python-pastel, and
python-pylev.  Add python-rapidfuzz.
[native-inputs]: Add python-crashtest.
---
 gnu/packages/python-xyz.scm | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5aac1de1af..e94f8067f3 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17829,19 +17829,16 @@ (define-public python-rapidfuzz
 (define-public python-cleo
   (package
     (name "python-cleo")
-    (version "0.8.1")
+    (version "2.0.1")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "cleo" version))
               (sha256
                (base32
-                "17q6wi3q280kxmxzb2iwnnqih2xbljn18v0bjx2ip18p079j43ix"))))
+                "1iayl7s1mrdjd6zc78vmcli3q5i4j5p9lj5yrs2i1hb360gjwjzb"))))
     (build-system python-build-system)
-    (native-inputs
-     (list ;; For testing
-           python-mock python-pytest-mock python-pytest))
-    (propagated-inputs
-     (list python-backpack python-clikit python-pastel python-pylev))
+    (native-inputs (list python-crashtest))
+    (propagated-inputs (list python-rapidfuzz))
     (home-page "https://github.com/sdispater/cleo")
     (synopsis "Command-line arguments library for Python")
     (description
--
2.39.2

[bug#63139] [PATCH python 06/20] gnu: Add python-deepdiff.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-deepdiff): New variable.
---
 gnu/packages/python-xyz.scm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index e94f8067f3..95eea4ecf2 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -18072,6 +18072,33 @@ (define-public python-cachy
 database, file, dict stores.  Cachy supports python versions 2.7+ and 3.2+.")
     (license license:expat)))

+(define-public python-deepdiff
+  (package
+    (name "python-deepdiff")
+    (version "6.3.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "deepdiff" version))
+              (sha256
+               (base32
+                "0i5nnb3nppi2vgbhiakpxiagyhx7l1f50hzcl8fcgica4bkz2fva"))))
+    (build-system pyproject-build-system)
+    (propagated-inputs (list python-ordered-set))
+    (native-inputs (list python-dateutil
+                         python-click
+                         python-jsonpickle
+                         python-numpy
+                         python-pytest
+                         python-pyyaml))
+    (home-page "https://zepworks.com")
+    (synopsis
+     "Deep difference of any Python objects")
+    (description
+     "Deep Difference and Search of any Python object/data.  Recreate objects
+by adding adding deltas to each other.  Also includes DeepSearch for searching
+for objects within other objects and DeepHash to hash any object.")
+    (license license:expat)))
+
 (define-public poetry
   (package
     (name "poetry")
--
2.39.2

[bug#63139] [PATCH python 07/20] gnu: python-platformdirs: Update to 3.2.0.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-platformdirs): Update to 3.2.0.
[build-system]: Switch to pyproject-build-system.
[arguments]: Remove 'check' phase.
[native-inputs]: Add python-hatch-vcs and python-hatchling.  Remove
python-setuptools-scm.
---
 gnu/packages/python-xyz.scm | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 95eea4ecf2..46ceb1583b 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -29928,23 +29928,20 @@ (define-public python-stripe
 (define-public python-platformdirs
   (package
     (name "python-platformdirs")
-    (version "2.4.1")
+    (version "3.2.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "platformdirs" version))
        (sha256
-        (base32 "1njz0h4iky8iglrb85cd07hpa3lp1a2dfr934dj65hxwzvfk61j4"))))
-    (build-system python-build-system)
-    (arguments
-     '(#:phases (modify-phases %standard-phases
-                  (replace 'check
-                    (lambda* (#:key tests? #:allow-other-keys)
-                      (when tests?
-                        (invoke "pytest" "-vv")))))))
+        (base32 "021bdjqy29vvyl1qkcha07mdf0sr6gdqkxqg6mwzj9bz7753idnm"))))
+    (build-system pyproject-build-system)
     (native-inputs
-     (list python-appdirs python-pytest python-pytest-mock
-           python-setuptools-scm))
+     (list python-appdirs
+           python-hatch-vcs
+           python-hatchling
+           python-pytest
+           python-pytest-mock))
     (home-page "https://github.com/platformdirs/platformdirs")
     (synopsis "Determine the appropriate platform-specific directories")
     (description "When writing applications, finding the right location to
--
2.39.2

[bug#63139] [PATCH python 08/20] gnu: python-filelock: Update to 3.12.0.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-filelock): Update to 3.12.0.
[build-system]: Switch to pyproject-build-system.
[native-inputs]: Add python-hatch-vcs, python-hatchling, python-pytest, and
python-pytest-mock.
---
 gnu/packages/python-xyz.scm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 46ceb1583b..f99a4cd6e2 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -2217,15 +2217,19 @@ (define-public python-fb-re2
 (define-public python-filelock
   (package
     (name "python-filelock")
-    (version "3.0.12")
+    (version "3.12.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "filelock" version))
        (sha256
         (base32
-         "0ngzlvb5j8gqs2nxlp2b0jhzii792h66wsn694qm8kqixr225n0q"))))
-    (build-system python-build-system)
+         "0677p0b7jch94kz5g75d38sxna8i3c09g19wm0p3s0cc511sw0zw"))))
+    (build-system pyproject-build-system)
+    (native-inputs (list python-hatch-vcs
+                         python-hatchling
+                         python-pytest
+                         python-pytest-mock))
     (home-page
      "https://github.com/benediktschmitt/py-filelock")
     (synopsis "Platform independent file lock")
--
2.39.2

[bug#63139] [PATCH python 09/20] gnu: python-distlib: Update to 0.3.6.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-distlib): Update to 0.3.6.
---
 gnu/packages/python-xyz.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index f99a4cd6e2..62ed69c437 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -7701,14 +7701,14 @@ (define-public python-autopep8
 (define-public python-distlib
   (package
     (name "python-distlib")
-    (version "0.3.5")
+    (version "0.3.6")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "distlib" version))
        (sha256
         (base32
-         "1zmjraasgqkz0gfv4mc4w4fj4k2fxj62h1pf5dgb5qqbqwvmgxx7"))))
+         "0ihvw6b8h4279zpgin3320hqy9hr5aqk0zy9g893cfjdn3cx5fhl"))))
     (build-system python-build-system)
     (arguments
      (list
--
2.39.2

[bug#63139] [PATCH python 10/20] gnu: python-virtualenv: Update to 20.22.0.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-virtualenv): Update to 20.22.0.
[build-system]: Switch to pyproject-build-system.
[arguments]{test-flags}: Disable failing tests.
[native-inputs]: Remove python-mock and python-setuptools-scm.  Add
python-cython, python-flaky, python-hatch-vcs, python-hatchling, and
python-pytest-mock.
[propagated-inputs]: Remove python-appdirs and python-six.  Add
python-platformdirs.
---
 gnu/packages/python-xyz.scm | 39 ++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 62ed69c437..1f55df0cd9 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -4467,19 +4467,48 @@ (define-public python-vine
 (define-public python-virtualenv
   (package
     (name "python-virtualenv")
-    (version "20.3.1")
+    (version "20.22.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "virtualenv" version))
        (sha256
         (base32
-         "1nbhnpzswcf3lmzn5xabmfdd7ki8r2w2i37y6wml54di6qi1l48c"))))
-    (build-system python-build-system)
+         "1hvcc518l3byvk0hykb5x85fjh70w72s9f3d9vqs1wdfgb2571r7"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list #:test-flags
+           ;; TODO: Some of these tests may be able to be worked around.
+           ;; Currently they fail due to needing network access and/or trying
+           ;; to interact with the build environment in a way that does not
+           ;; work, or for unknown reasons.
+           '(let ((disabled-tests (list "test_acquire"
+                                        "test_activation_support"
+                                        "test_bootstrap_link_via_app_data"
+                                        "test_build_c_ext"
+                                        "test_builtin_discovery_class_preferred"
+                                        "test_can_symlink_when_symlinks_not_enabled"
+                                        "test_cpython3_win"
+                                        "test_creator"
+                                        "test_failed_to_find_implementation"
+                                        "test_periodic_update "
+                                        "test_pip_invoke"
+                                        "test_py_info"
+                                        "test_pypy3"
+                                        "test_value_alias"
+                                        "test_wheel_support_no_python_requires")))
+              (list "-k" (string-append "not "
+                                        (string-join disabled-tests
+                                                     " and not "))))))
     (native-inputs
-     (list python-mock python-pytest python-setuptools-scm))
+     (list python-cython
+           python-flaky
+           python-hatch-vcs
+           python-hatchling
+           python-pytest
+           python-pytest-mock))
     (propagated-inputs
-     (list python-appdirs python-distlib python-filelock python-six))
+     (list python-distlib python-filelock python-platformdirs))
     (home-page "https://virtualenv.pypa.io/")
     (synopsis "Virtual Python environment builder")
     (description
--
2.39.2

[bug#63139] [PATCH python 11/20] gnu: python-pkginfo: Update to 1.9.6.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-pkginfo): Update to 1.9.6.
---
 gnu/packages/python-xyz.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 1f55df0cd9..674420c0c4 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -21912,13 +21912,13 @@ (define-public python-tqdm
 (define-public python-pkginfo
   (package
     (name "python-pkginfo")
-    (version "1.8.3")
+    (version "1.9.6")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "pkginfo" version))
        (sha256
-        (base32 "0z46w559hrl79gf7navgzimj21ma821wka27jh58fvyqilqs8kd8"))))
+        (base32 "0inh57664sx2vlbd3913dsc9nz21ysb9vk591qpkg90qhxp8kmcg"))))
     (build-system python-build-system)
     (native-inputs
      (list python-wheel))
--
2.39.2

[bug#63139] [PATCH python 12/20] gnu: python-jsonschema: Update to 4.17.3.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-jsonschema): Update to 4.17.3.
[native-inputs]: Remove python-setuptools-scm.  Add
python-hatch-fancy-pypi-readme, python-hatch-vcs, and python-hatchling.
---
 gnu/packages/python-xyz.scm | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 674420c0c4..a3ed048f03 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -4081,15 +4081,13 @@ (define-public python-jsondiff
 (define-public python-jsonschema
   (package
     (name "python-jsonschema")
-    ;; XXX: Update to the latest version requires new build system - Hatch
-    ;; https://hatch.pypa.io/
-    (version "4.5.1")
+    (version "4.17.3")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "jsonschema" version))
        (sha256
-        (base32 "1z0x22691jva7lwfcfh377jdmlz68zhiawxzl53k631l34k8hvbw"))))
+        (base32 "03dnxhvzfxmnpn53zsc0598hsslaz7w3wi87cyx7cq4bmcvl91hg"))))
     (build-system pyproject-build-system)
     (arguments
      (list
@@ -4106,7 +4104,10 @@ (define-public python-jsonschema
               (when tests?
                 (setenv "JSON_SCHEMA_TEST_SUITE" "json")
                 (invoke "trial" "jsonschema")))))))
-    (native-inputs (list python-setuptools-scm python-twisted))
+    (native-inputs (list python-hatch-fancy-pypi-readme
+                         python-hatch-vcs
+                         python-hatchling
+                         python-twisted))
     (propagated-inputs
      (list python-attrs
            python-importlib-metadata
--
2.39.2

[bug#63139] [PATCH python 13/20] gnu: python-dulwich: Update to 0.21.3.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-dulwich): Update to 0.21.3.
[arguments]{phases}: In 'check' phase skip a test which fails without a
configuration file.
[native-inputs]: Add python-paramiko and python-requests.
---
 gnu/packages/python-xyz.scm | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index a3ed048f03..8b61cba8ae 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -19932,7 +19932,7 @@ (define-public python-cheetah
 (define-public python-dulwich
   (package
     (name "python-dulwich")
-    (version "0.20.30")
+    (version "0.21.3")
     (source
       (origin
         (method url-fetch)
@@ -19941,23 +19941,27 @@ (define-public python-dulwich
                    (pypi-uri "dulwich" version)))
         (sha256
           (base32
-           "0hafaff30bmkj30b8pwpwsy3fz5h6c1pn98ihqcvl5zndflr1h22"))))
+           "10yg11kcjc8lbk2hvnyz3hs5m1rlr76wzw2qxjrq7sv7sx9v98vw"))))
     (build-system python-build-system)
     (arguments
-     '(#:phases
-       (modify-phases %standard-phases
-         (add-before 'check 'fix-tests
-           (lambda* (#:key inputs #:allow-other-keys)
-             (substitute* '("dulwich/tests/test_repository.py"
-                            "dulwich/tests/test_porcelain.py"
-                            "dulwich/tests/test_hooks.py")
-               (("/bin/sh") (search-input-file inputs "/bin/sh")))
-             (setenv "TEST_RUNNER" "unittest")
-             (setenv "PYTHONHASHSEED" "random"))))))
+     (list #:phases
+           #~(modify-phases %standard-phases
+               (add-before 'check 'fix-tests
+                 (lambda* (#:key inputs #:allow-other-keys)
+                   (substitute* "dulwich/contrib/test_swift_smoke.py"
+                     (("^class SwiftRepoSmokeTest.*" line)
+                      (string-append "@unittest.skipIf(True, \
+'Needs configuration file')\n" line)))
+                   (substitute* '("dulwich/tests/test_repository.py"
+                                  "dulwich/tests/test_porcelain.py"
+                                  "dulwich/tests/test_hooks.py")
+                     (("/bin/sh") (search-input-file inputs "/bin/sh")))
+                   (setenv "TEST_RUNNER" "unittest")
+                   (setenv "PYTHONHASHSEED" "random"))))))
     (propagated-inputs
      (list python-fastimport python-urllib3))
     (native-inputs
-     (list python-mock python-geventhttpclient python-gpg
+     (list python-mock python-paramiko python-geventhttpclient python-gpg python-requests
            git gnupg))
     (home-page "https://www.dulwich.io/")
     (synopsis "Git implementation in Python")
--
2.39.2

[bug#63139] [PATCH python 14/20] gnu: Update python-pypa-build to 1.0.0.

[John Kehayias via Guix-patches via]

* gnu/packages/python-build.scm (python-pypa-build): Rename to...
(python-pypa-build-bootstrap): ... this.
(python-pypa-build): Update to 1.0.0.
---
 gnu/packages/python-build.scm | 39 +++++++++++++++++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 45b9f52b68..090f0d2274 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -502,9 +502,9 @@ (define-public python-packaging-bootstrap

 ;;; The name 'python-pypa-build' is chosen rather than 'python-build' to avoid
 ;;; a name clash with python-build from (guix build-system python).
-(define-public python-pypa-build
+(define-public python-pypa-build-bootstrap
   (package
-    (name "python-pypa-build")
+    (name "python-pypa-build-bootstrap")
     (version "0.7.0")
     (source (origin
               (method url-fetch)
@@ -534,6 +534,41 @@ (define-public python-pypa-build
 order to make bootstrapping easier.")
     (license license:expat)))

+(define-public python-pypa-build
+  (package
+    (inherit python-pypa-build-bootstrap)
+    (name "python-pypa-build")
+    (version "0.10.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "build" version))
+              (sha256
+               (base32
+                "0scj5k586n8rmnlqhay5j43ci1z8ip3sm0j4f3b52nfvmxj15dym"))))
+    (arguments
+     (list
+      #:tests? #f ; avoid needing pytest as an input
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'use-toml-instead-of-tomli
+            ;; Using toml instead of tomli eases bootstrapping.
+            (lambda _
+              (substitute* "pyproject.toml"
+                (("'tomli.*")
+                 "'toml'"))))
+          (replace 'build
+            (lambda _
+              ;(setenv "PYTHONPATH" ".")
+              (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
+          (replace 'install
+            (lambda _
+              (let ((whl (car (find-files "dist" "\\.whl$"))))
+                (invoke "pip" "--no-cache-dir" "--no-input"
+                        "install" "--no-deps" "--prefix" #$output
+                        "--ignore-installed" whl)))))))
+    (native-inputs (list python-flit-core-bootstrap python-pypa-build-bootstrap))
+    (propagated-inputs (list python-pyproject-hooks))))
+
 (define-public python-poetry-core
   (package
     (name "python-poetry-core")
--
2.39.2

[bug#63139] [PATCH python 15/20] gnu: poetry: Fix build.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (poetry)[build-system]: Switch to
pyproject-build-system.
[arguments]{phases}: New phase 'loosen-requirements' to fix build and avoid
circular dependency.
{test-flags}: Disable failing tests.
[propagated-inputs]: Remove python-pip.  Add python-dulwich, python-installer,
python-jsonschema, python-platfromdirs, python-pypa-build,
python-pyproject-hooks, python-tomli, and python-trove-classifiers.
[native-inputs]: Add python-deepdiff, python-httpretty, python-pytest,
python-pytest-mock, and python-pytest-xdist.
---
 gnu/packages/python-xyz.scm | 62 ++++++++++++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 8b61cba8ae..cbe3bb1094 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -18144,37 +18144,75 @@ (define-public poetry
        (sha256
         (base32
          "0g0vczn6qa4b2bdkq4k7fm1g739vyxp2iiblwwsrcmw24jj81m8b"))))
-    (build-system python-build-system)
+    (build-system pyproject-build-system)
     (arguments
-     `(#:tests? #f                      ;PyPI does not have tests
-       ;; #:phases
-       ;; (modify-phases %standard-phases
-       ;;   (add-before 'build 'patch-setup-py
-       ;;     (lambda _
-       ;;       (substitute* "setup.py"
-       ;;         ;; Relax some of the requirements.
-       ;;         (("(keyring>=21.2.0),<22.0.0" _ keyring) keyring)
-       ;;         (("(packaging>=20.4),<21.0" _ packaging) packaging)))))
-       ))
+     (list #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'loosen-requirements
+                 (lambda _
+                   (substitute* "pyproject.toml"
+                     ;; Permit newer versions of platformdirs.
+                     (("platformdirs =.*") "platformdirs = \"^3.0.0\"\n")
+                     ;; Our python-trove-classifiers reports the wrong version.
+                     (("trove-classifiers =.*") "trove-classifiers = \"*\"\n")
+                     ;; Avoid circular dependency by not including the export
+                     ;; plugin requirement.  See
+                     ;; <https://github.com/python-poetry/poetry/pull/5980>
+                     ;; and
+                     ;; <https://github.com/python-poetry/poetry/issues/6441>
+                     ;; for upstream status in resolving this cycle.
+                     (("poetry-plugin-export.*") "")))))
+           #:test-flags
+           '(let ((disabled-tests
+                   ;; The following tests currently fail, requiring network
+                   ;; connectivity and/or trying to write in the build
+                   ;; environment.
+                   ;; TODO: Some of these might be worked around.
+                   (list "test_shell"
+                         (string-append "test_build_backend_errors_are_"
+                                        "reported_correctly_if_caused_by_subprocess")
+                         "test_prepare_sdist"
+                         (string-append "test_executor_should_write_"
+                                        "pep610_url_references_for_non_wheel_files")
+                         "test_uninstall_git_package_nspkg_pth_cleanup"
+                         "test_builder_setup_generation_runs_with_pip_editable"
+                         "test_prepare_directory"
+                         "test_prepare_directory_with_extensions"
+                         "test_prepare_directory_editable")))
+              (list "-k" (string-append "not "
+                                        (string-join disabled-tests
+                                                     " and not "))))))
     (propagated-inputs
      (list python-cachecontrol
            python-cachy
            python-cleo
            python-crashtest
+           python-dulwich
            python-entrypoints
            python-html5lib
+           python-installer
+           python-jsonschema
            python-keyring
            python-msgpack
            python-packaging
            python-pexpect
-           python-pip
            python-pkginfo
+           python-platformdirs
+           python-pypa-build
            python-poetry-core
+           python-pyproject-hooks
            python-requests
            python-requests-toolbelt
            python-shellingham
+           python-tomli
            python-tomlkit
+           python-trove-classifiers
            python-virtualenv))
+    (native-inputs (list python-deepdiff
+                         python-httpretty
+                         python-pytest
+                         python-pytest-mock
+                         python-pytest-xdist))
     (home-page "https://python-poetry.org")
     (synopsis "Python dependency management and packaging made easy")
     (description "Poetry is a tool for dependency management and packaging
--
2.39.2

[bug#63139] [PATCH python 16/20] gnu: Add python-poetry-plugin-export.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-poetry-plugin-export): New variable.
---
 gnu/packages/python-xyz.scm | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index cbe3bb1094..e468844ff6 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -18133,6 +18133,35 @@ (define-public python-deepdiff
 for objects within other objects and DeepHash to hash any object.")
     (license license:expat)))

+(define-public python-poetry-plugin-export
+  (package
+    (name "python-poetry-plugin-export")
+    (version "1.3.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "poetry_plugin_export" version))
+              (sha256
+               (base32
+                "115zx9nqjbzpgj4akz9g5rzpd2pdykxbz584h6sz19d8awkp8jfr"))))
+    (build-system pyproject-build-system)
+    (native-inputs (list poetry
+                         python-poetry-core
+                         python-pytest
+                         python-pytest-mock
+                         python-pytest-xdist))
+    (propagated-inputs (list python-pypa-build python-cachy python-pendulum))
+    (home-page "https://github.com/python-poetry/poetry-plugin-export")
+    (synopsis "Poetry plugin to export the dependencies to various formats")
+    (description "This package is a plugin for Poetry that allows the export
+of locked packages to various formats.
+
+Note: For now, only the @code{constraints.txt} and @code{requirements.txt}
+formats are available.
+
+This plugin provides the same features as the existing export command of
+Poetry which it will eventually replace.")
+    (license license:expat)))
+
 (define-public poetry
   (package
     (name "poetry")
--
2.39.2

[bug#63139] [PATCH python 17/20] gnu: python-pyscard: Update to 2.0.7.

[John Kehayias via Guix-patches via]

* gnu/packages/security-token.scm (python-pyscard): Update to 2.0.7.
---
 gnu/packages/security-token.scm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/security-token.scm b/gnu/packages/security-token.scm
index b51d67427a..5dfd3ea35d 100644
--- a/gnu/packages/security-token.scm
+++ b/gnu/packages/security-token.scm
@@ -14,7 +14,7 @@
 ;;; Copyright © 2021 Sergey Trofimov <sarg@sarg.org.ru>
 ;;; Copyright © 2021 Dhruvin Gandhi <contact@dhruvin.dev>
 ;;; Copyright © 2021 Ahmad Jarara <git@ajarara.io>
-;;; Copyright © 2022 John Kehayias <john.kehayias@protonmail.com>
+;;; Copyright © 2022, 2023 John Kehayias <john.kehayias@protonmail.com>
 ;;; Copyright © 2022 Petr Hodina <phodina@protonmail.com>
 ;;; Copyright © 2022 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
 ;;; Copyright © 2023 Jake Leporte <jakeleporte@outlook.com>
@@ -462,7 +462,7 @@ (define-public yubikey-personalization
 (define-public python-pyscard
   (package
     (name "python-pyscard")
-    (version "1.9.9")
+    (version "2.0.7")
     (source (origin
               (method url-fetch)
               ;; The maintainer publishes releases on various sites, but
@@ -472,7 +472,7 @@ (define-public python-pyscard
                     version "/pyscard-" version ".tar.gz"))
               (sha256
                (base32
-                "082cjkbxadaz2jb4rbhr0mkrirzlqyqhcf3r823qb0q1k50ybgg6"))))
+                "1gy1hmzrhfa7bqs132v89pchm9q3rpnqf3a6225vwpx7bx959017"))))
     (build-system python-build-system)
     (arguments
      `(#:phases
--
2.39.2

[bug#63139] [PATCH python 18/20] gnu: python-fido2: Update to 1.1.1 and enable tests.

[John Kehayias via Guix-patches via]

* gnu/packages/security-token.scm (python-fido2): Update to 1.1.1.
[build-system]: Switch to pyproject-build-system.
[arguments]: Enable tests.
[native-inputs]: Add python-poetry-core and python-pytest.
---
 gnu/packages/security-token.scm | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/security-token.scm b/gnu/packages/security-token.scm
index 5dfd3ea35d..ad772ef292 100644
--- a/gnu/packages/security-token.scm
+++ b/gnu/packages/security-token.scm
@@ -48,6 +48,7 @@ (define-module (gnu packages security-token)
   #:use-module (guix build-system cmake)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system glib-or-gtk)
+  #:use-module (guix build-system pyproject)
   #:use-module (guix build-system python)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
@@ -73,6 +74,7 @@ (define-module (gnu packages security-token)
   #:use-module (gnu packages networking)
   #:use-module (gnu packages cyrus-sasl)
   #:use-module (gnu packages popt)
+  #:use-module (gnu packages python-build)
   #:use-module (gnu packages readline)
   #:use-module (gnu packages qt)
   #:use-module (gnu packages tls)
@@ -684,7 +686,7 @@ (define-public pam-u2f
 (define-public python-fido2
   (package
     (name "python-fido2")
-    (version "0.9.3")
+    (version "1.1.1")
     (source (origin
               (method url-fetch)
               (uri
@@ -693,17 +695,13 @@ (define-public python-fido2
                 version "/fido2-" version ".tar.gz"))
               (sha256
                (base32
-                "1v366h449f8q74jkmy1291ffj2345nm7cdsipgqvgz4w22k8jpml"))
+                "1hwz0xagkmy6hhcyfl66dxf2vfa69lqqqjrv70vw7harik59bi2x"))
               (snippet
                ;; Remove bundled dependency.
                '(delete-file "fido2/public_suffix_list.dat"))))
-    (build-system python-build-system)
+    (build-system pyproject-build-system)
     (arguments
-     `(;; This attempts to access
-       ;; /System/Library/Frameworks/IOKit.framework/IOKit
-       ;; The recommendation is to use tox for testing.
-       #:tests? #false
-       #:phases
+     `(#:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'install-public-suffix-list
            (lambda* (#:key inputs #:allow-other-keys)
@@ -717,7 +715,11 @@ (define-public python-fido2
     (propagated-inputs
      (list python-cryptography python-six))
     (native-inputs
-     (list python-mock python-pyfakefs public-suffix-list))
+     (list python-poetry-core
+           python-mock
+           python-pyfakefs
+           python-pytest
+           public-suffix-list))
     (home-page "https://github.com/Yubico/python-fido2")
     (synopsis "Python library for communicating with FIDO devices over USB")
     (description
--
2.39.2

[bug#63139] [PATCH python 19/20] gnu: Add python-makefun.

[John Kehayias via Guix-patches via]

* gnu/packages/python-xyz.scm (python-makefun): New variable.
---
 gnu/packages/python-xyz.scm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index e468844ff6..94c0e1ab81 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -24905,6 +24905,24 @@ (define-public python-construct
 make your work easier.")
     (license license:expat)))

+(define-public python-makefun
+  (package
+    (name "python-makefun")
+    (version "1.15.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "makefun" version))
+              (sha256
+               (base32
+                "19a8dga8rnmjn5gy1cy1wdi28swbkdkypwbqikbxil6ynqcg3c20"))))
+    (build-system pyproject-build-system)
+    (native-inputs (list python-setuptools-scm python-pytest))
+    (home-page "https://smarie.github.io/python-makefun")
+    (synopsis "Small library to dynamically create Python functions.")
+    (description "A library to dynamically create Python functions with a
+proper signature.")
+    (license license:bsd-3)))
+
 (define-public python-outcome
   (package
     (name "python-outcome")
--
2.39.2

[bug#63139] [PATCH python 20/20] gnu: python-yubikey-manager: Update to 5.1.0 and enable tests.

[John Kehayias via Guix-patches via]

* gnu/packages/security-token.scm (python-yubikey-manager): Update to 5.1.0.
[source]: Update url.
[build-system]: Switch to pyproject-build-system.
[arguments]: Enable tests.
{phases}: Add a 'fix-piv-commands' phase to avoid an infinite loop in tests.
[propagated-inputs]: Add python-keyring.  Reorder alphabetically.
[native-inputs]: Remove python-mock.  Add python-makefun,
python-poetry-core, and python-pytest.  Reorder alphabetically.
---
 gnu/packages/security-token.scm | 35 ++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/gnu/packages/security-token.scm b/gnu/packages/security-token.scm
index ad772ef292..7db9fc9b61 100644
--- a/gnu/packages/security-token.scm
+++ b/gnu/packages/security-token.scm
@@ -741,33 +741,40 @@ (define-public python-fido2
 (define-public python-yubikey-manager
   (package
     (name "python-yubikey-manager")
-    (version "4.0.7")
+    (version "5.1.0")
     (source (origin
               (method url-fetch)
               (uri (string-append
                     "https://developers.yubico.com/yubikey-manager/Releases"
-                    "/yubikey-manager-" version ".tar.gz"))
+                    "/yubikey_manager-" version ".tar.gz"))
               (sha256
                (base32
-                "0kzwal7i4kyywm4f5zh8b823mh0ih2nsh5c0c4dfn4vw3j5dnwlr"))))
-    (build-system python-build-system)
+                "0wxi3rms9d5f7p4zl1na4412dirp1g2gd5wkgi6zs4g5hagzqgnk"))))
+    (build-system pyproject-build-system)
     (arguments
-     '(;; This attempts to access
-       ;; /System/Library/Frameworks/IOKit.framework/IOKit
-       ;; The recommendation is to use tox for testing.
-       #:tests? #false))
+     (list #:phases
+           #~(modify-phases %standard-phases
+               ;; Avoid an infinite loop with Python < 3.11 which causes tests
+               ;; to fail.  See upstream bug report and fix at
+               ;; <https://github.com/Yubico/yubikey-manager/issues/552>.
+               ;; TODO: Remove in next release.
+               (add-after 'unpack 'fix-piv-commands
+                 (lambda* _
+                   (substitute* "yubikit/piv.py"
+                     (("self:02X") "int(self):02X")))))))
     (propagated-inputs
-     (list python-six
-           python-pyscard
-           python-pyusb
-           python-click
+     (list python-click
            python-cryptography
+           python-fido2
+           python-keyring
            python-pyopenssl
-           python-fido2))
+           python-pyscard
+           python-pyusb
+           python-six))
     (inputs
      (list pcsc-lite))
     (native-inputs
-     (list swig python-mock))
+     (list python-makefun python-poetry-core python-pytest swig))
     (home-page "https://developers.yubico.com/yubikey-manager/")
     (synopsis "Command line tool and library for configuring a YubiKey")
     (description
--
2.39.2

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[Lars-Dominik Braun]

Hi,

> +               (replace 'build
> +                 (lambda _
> +                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
> +               (replace 'install
> +                 (lambda _
> +                   (let ((whl (car (find-files "dist" "\\.whl$"))))
> +                     (invoke "pip" "--no-cache-dir" "--no-input"
> +                             "install" "--no-deps" "--prefix" #$output whl)))))))

why is this necessary? As far as I see pyproject-hooks wants to be built
using flit_core, which we have a fully bootstrapped version of. Same
for pypa-build, which – as far as I can see – also builds using flit_core.

Cheers,
Lars

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[John Kehayias via Guix-patches via]

Hi Lars,

On Mon, May 01, 2023 at 08:58 AM, Lars-Dominik Braun wrote:

> Hi,
>
>> +               (replace 'build
>> +                 (lambda _
>> +                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
>> +               (replace 'install
>> +                 (lambda _
>> +                   (let ((whl (car (find-files "dist" "\\.whl$"))))
>> +                     (invoke "pip" "--no-cache-dir" "--no-input"
>> +                             "install" "--no-deps" "--prefix" #$output whl)))))))
>
> why is this necessary? As far as I see pyproject-hooks wants to be built
> using flit_core, which we have a fully bootstrapped version of. Same
> for pypa-build, which – as far as I can see – also builds using flit_core.
>

This gets to what I wasn't sure about, what exactly needs to be
bootstrapped and what depends on what. From what I remember and
thought, the newer python-pypa-build needs python-pyproject-hooks.
Which itself would be built with pyproject-build-system, creating a
circular dependency. But, I don't know anything about flit_core and
that system. So you are saying flit_core is used to build each of
these without a cycle then?

Can you layout what the new structure would be for updating
python-pypa-build with python-pyproject-hooks then? (And then
deprecate pep517.)

Thanks!
John

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[Lars-Dominik Braun]

Hi,

> From what I remember and
> thought, the newer python-pypa-build needs python-pyproject-hooks.
> Which itself would be built with pyproject-build-system, creating a
> circular dependency.

ah, yes, you’re right. For some reason the current
pyproject-build-system differs from my original proposal and bundles
python-pypa-build in python-toolchain (even though it is not used). As
we can see from this example doing so is a very very bad idea (because
it leads to cycles).

> But, I don't know anything about flit_core and
> that system. So you are saying flit_core is used to build each of
> these without a cycle then?
> Can you layout what the new structure would be for updating
> python-pypa-build with python-pyproject-hooks then? (And then
> deprecate pep517.)

In theory that would be possible if python-toolchain did not depend on
pypa-build. Looking at the respective pyproject.toml files flit-core
is entirely self-contained and can be built using itself. Then you can
build pyproject-hooks, packaging and tomli using flit-core. And then
pypa-build can be built using flit-core and all of the previous packages
as dependencies. poetry-core is also self-contained and can be built
without any dependencies (and the full poetry should never be required
to build any project, but obviously packages in Guix use it for that
purpose -.-). But none of this is really necessary, because – again
– pypa-build is not used by pyproject-build-system.

So I propose the following:

- Create a python-team branch.
- Use python-sans-pip-wrapper as default-python in pyproject-build-system,
  like I originally proposed and add python-setuptools/python-wheel
  where necessary.
- Implement backend-path in pyproject-build-system, so we can bootstrap
  flit-core/poetry-core/… properly.
- Expose ensure-no-mtimes-pre-1980 (#63044).
- Finally fix python-yubikey-manager.

Anything else that’s broken right now?

Cheers,
Lars

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[John Kehayias via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 2711 bytes --]

Hi Lars

Apologies for the delayed response (meant to earlier), top posting, and the formatting. The reason for all three is that I'm currently away from a proper computer for a few weeks but I'll be back at the beginning of June and can start hacking away then. Of course, happy to be beat to the job or have anything to build off of.

I'm not sure of any other deep python breakages, but definitely something to look at for the python branch. Although I'm not python expert, especially at packaging, I should probably add myself to the team when I'm back as well.

Thanks for the details and information, that's very helpful and sounds like a roadmap to have a clean and updated toolchain here.

John
-------- Original Message --------
On May 8, 2023, 3:44 PM, Lars-Dominik Braun wrote:

> Hi, > From what I remember and > thought, the newer python-pypa-build needs python-pyproject-hooks. > Which itself would be built with pyproject-build-system, creating a > circular dependency. ah, yes, you’re right. For some reason the current pyproject-build-system differs from my original proposal and bundles python-pypa-build in python-toolchain (even though it is not used). As we can see from this example doing so is a very very bad idea (because it leads to cycles). > But, I don't know anything about flit_core and > that system. So you are saying flit_core is used to build each of > these without a cycle then? > Can you layout what the new structure would be for updating > python-pypa-build with python-pyproject-hooks then? (And then > deprecate pep517.) In theory that would be possible if python-toolchain did not depend on pypa-build. Looking at the respective pyproject.toml files flit-core is entirely self-contained and can be built using itself. Then you can build pyproject-hooks, packaging and tomli using flit-core. And then pypa-build can be built using flit-core and all of the previous packages as dependencies. poetry-core is also self-contained and can be built without any dependencies (and the full poetry should never be required to build any project, but obviously packages in Guix use it for that purpose -.-). But none of this is really necessary, because – again – pypa-build is not used by pyproject-build-system. So I propose the following: - Create a python-team branch. - Use python-sans-pip-wrapper as default-python in pyproject-build-system, like I originally proposed and add python-setuptools/python-wheel where necessary. - Implement backend-path in pyproject-build-system, so we can bootstrap flit-core/poetry-core/… properly. - Expose ensure-no-mtimes-pre-1980 (#63044). - Finally fix python-yubikey-manager. Anything else that’s broken right now? Cheers, Lars

[-- Attachment #2: Type: text/html, Size: 2863 bytes --]

[bug#63139] [PATCH python 02/20] gnu: Add python-pyproject-hooks.

[Lars-Dominik Braun]

Hi John,

> Apologies for the delayed response (meant to earlier), top posting, and the formatting. The reason for all three is that I'm currently away from a proper computer for a few weeks but I'll be back at the beginning of June and can start hacking away then. Of course, happy to be beat to the job or have anything to build off of.

no worries. I also won’t have any time to look into this
for a few weeks. My (rather dirty) python-team branch is here:
https://github.com/PromyLOPh/guix/commits/python-team
Perhaps it helps you to fix this issue.

Cheers,
Lars

[bug#63139] poetry not building

[Reza Housseini]

[-- Attachment #1.1.1: Type: text/plain, Size: 398 bytes --]

Hi

Poetry is not building on ci.guix.gnu.org [1]. There is a pending patch 
[2] on the issue tracker. What is missing to apply this patch and how 
can I help?

Cheers,
Reza

[1] https://ci.guix.gnu.org/search?query=poetry
[2] https://issues.guix.gnu.org/63139

-- 
Reza Housseini

This message is signed with my GnuPG key:

     C0F3 0812 9AF2 80F4 0830 C2C1 C375 C6AF 0512 5C52

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 15557 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[bug#63139] [PATCH python 00/20] Updates for pyproject-build-system, poetry, python-yubikey-manager

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 147 bytes --]

Hello,

Gentle ping on this issue.

Lets check it out on python-team branch?
I can volunteer myself to resolve dependence cycles ;-)

Thanks,
Oleg

[-- Attachment #2: Type: text/html, Size: 350 bytes --]

[bug#63139] [PATCH python 00/20] Updates for pyproject-build-system, poetry, python-yubikey-manager

[John Kehayias via Guix-patches via]

Hi Oleg,

On Fri, Feb 02, 2024 at 08:43 PM, Sharlatan Hellseher wrote:

> Hello,
>
> Gentle ping on this issue.
>

Your message looks like it was just sent to the debbugs number, which
doesn't reach any humans (as far as I know); it is not a mailing list.
Confusing, and who knows how many messages have not been seen because of
that.

Anyway, CC'ing Lars since there was a mention of some WIP patches on a
off guix branch.

> Lets check it out on python-team branch?
> I can volunteer myself to resolve dependence cycles ;-)
>

Thanks for the volunteering, I guess we should see what the status is
with whatever Lars got up to before and see what is next.

John

[bug#63139] [PATCH python 00/20] Updates for pyproject-build-system, poetry, python-yubikey-manager

[Lars-Dominik Braun]

Hi,

> Anyway, CC'ing Lars since there was a mention of some WIP patches on a
> off guix branch.

the patches from #65001 have been merged into the python-team
branch. That should fix the first four steps from my proposal at
https://issues.guix.gnu.org/issue/63139#23 and allow us to properly
bootstrap the required packages and in the end also fix
python-yubikey-manager.

Cheers,
Lars