* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. @ 2022-10-17 20:06 Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-18 9:33 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system zimoun 0 siblings, 2 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:06 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer This is an adaptation of Lars' work in wip-python-pep517 for 'master'. I've implemented it as a separate build system so we can do a slow transition, starting at the leafs. Changes from the original branch: * Renamed to pyproject-build-system. * It inherits some code from python-build-system. * Python 2 and <3.7 support removed. * A (temporary) sanity-check-next.py file is introduced. * A new (hidden) 'python-sans-pip' variable is introduced. * The 'python-toolchain' package includes pip, wheel, setuptools, and pypa-build. The build system logic is unchanged from Lars' branch. I've converted a handful packages that were doing PEP 517 style builds to the new build system with great success. It should help a lot with packaging "modern" Python packages. Please try it for new packages, even those using setup.py, and don't be afraid to report problems! I plan to eventually merge it with python-build-system instead of keeping it separate. Thoughts? Note: documentation is missing. Will work on that next. Lars-Dominik Braun (3): gnu: python-setuptools: Move to python-build. gnu: pypy: Move to separate module. build-system: Add pyproject-build-system. Marius Bakke (11): gnu: python-pip: Move to (gnu packages python-build). gnu: python-autopage: Use pyproject-build-system. gnu: flair: Switch to pyproject-build-system. gnu: python-pydyf: Use pyproject-build-system. gnu: weasyprint: Use pyproject-build-system. gnu: python-glyphslib: Use pyproject-build-system. gnu: python-statmake: Use pyproject-build-system. gnu: python-ufolib2: Use pyproject-build-system. gnu: python-mypy-protobuf: Switch to pyproject-build-system. gnu: python-tempora: Switch to pyproject-build-system. gnu: python-pygmsh: Use pyproject-build-system. Makefile.am | 2 + gnu/local.mk | 2 + .../aux-files/python/sanity-check-next.py | 98 ++++ gnu/packages/bioinformatics.scm | 23 +- gnu/packages/chemistry.scm | 1 + gnu/packages/fontutils.scm | 91 +--- gnu/packages/messaging.scm | 1 + gnu/packages/pdf.scm | 40 +- gnu/packages/protobuf.scm | 15 +- gnu/packages/pypy.scm | 273 +++++++++++ gnu/packages/python-build.scm | 61 +++ gnu/packages/python-commencement.scm | 64 +++ gnu/packages/python-xyz.scm | 110 +---- gnu/packages/python.scm | 180 +------ gnu/packages/sequoia.scm | 2 +- gnu/packages/simulation.scm | 20 +- guix/build-system/pyproject.scm | 148 ++++++ guix/build/pyproject-build-system.scm | 460 ++++++++++++++++++ tests/lint.scm | 2 +- 19 files changed, 1164 insertions(+), 429 deletions(-) create mode 100644 gnu/packages/aux-files/python/sanity-check-next.py create mode 100644 gnu/packages/pypy.scm create mode 100644 gnu/packages/python-commencement.scm create mode 100644 guix/build-system/pyproject.scm create mode 100644 guix/build/pyproject-build-system.scm -- 2.38.0 ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build. 2022-10-17 20:06 [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module Marius Bakke ` (13 more replies) 2022-10-18 9:33 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system zimoun 1 sibling, 14 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> * gnu/packages/python-xyz.scm (python-setuptools): Move… * gnu/packages/python-build.scm: …here. * gnu/packages/chemistry.scm, gnu/packages/messaging.scm, gnu/packages/sequoia.scm, tests/lint.scm: Adjust module imports accordingly. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/packages/chemistry.scm | 1 + gnu/packages/messaging.scm | 1 + gnu/packages/python-build.scm | 40 +++++++++++++++++++++++++++++++++++ gnu/packages/python-xyz.scm | 40 ----------------------------------- gnu/packages/sequoia.scm | 2 +- tests/lint.scm | 2 +- 6 files changed, 44 insertions(+), 42 deletions(-) diff --git a/gnu/packages/chemistry.scm b/gnu/packages/chemistry.scm index d0582c4b6f..6549da3453 100644 --- a/gnu/packages/chemistry.scm +++ b/gnu/packages/chemistry.scm @@ -56,6 +56,7 @@ (define-module (gnu packages chemistry) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-xyz) #:use-module (gnu packages qt) #:use-module (gnu packages serialization) diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm index fa889d24a7..86d9914c90 100644 --- a/gnu/packages/messaging.scm +++ b/gnu/packages/messaging.scm @@ -116,6 +116,7 @@ (define-module (gnu packages messaging) #:use-module (gnu packages pkg-config) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-check) #:use-module (gnu packages python-crypto) #:use-module (gnu packages python-web) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 9d9b07f769..3ff988820f 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,46 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-setuptools + (package + (name "python-setuptools") + (version "64.0.3") + (source + (origin + (method url-fetch) + (uri (pypi-uri "setuptools" version)) + (sha256 + (base32 + "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) + (modules '((guix build utils))) + (snippet + ;; Remove included binaries which are used to build self-extracting + ;; installers for Windows. + ;; TODO: Find some way to build them ourself so we can include them. + '(for-each delete-file (find-files "setuptools" + "^(cli|gui).*\\.exe$"))))) + (build-system python-build-system) + ;; FIXME: Tests require pytest, which itself relies on setuptools. + ;; One could bootstrap with an internal untested setuptools. + (arguments (list #:tests? #f)) + (home-page "https://pypi.org/project/setuptools/") + (synopsis "Library designed to facilitate packaging Python projects") + (description "Setuptools is a fully-featured, stable library designed to +facilitate packaging Python projects, where packaging includes: +@itemize +@item Python package and module definitions +@item distribution package metadata +@item test hooks +@item project installation +@item platform-specific details. +@end itemize") + ;; TODO: setuptools now bundles the following libraries: + ;; packaging, pyparsing, six and appdirs. How to unbundle? + (license (list license:psfl ;setuptools itself + license:expat ;six, appdirs, pyparsing + license:asl2.0 ;packaging is dual ASL2/BSD-2 + license:bsd-2)))) + (define-public python-wheel (package (name "python-wheel") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index ebc56bbc11..3ea29ec3b9 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -2043,46 +2043,6 @@ (define-public python-serpent other machines, such as over the network.") (license license:expat))) -(define-public python-setuptools - (package - (name "python-setuptools") - (version "64.0.3") - (source - (origin - (method url-fetch) - (uri (pypi-uri "setuptools" version)) - (sha256 - (base32 - "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) - (modules '((guix build utils))) - (snippet - ;; Remove included binaries which are used to build self-extracting - ;; installers for Windows. - ;; TODO: Find some way to build them ourself so we can include them. - '(for-each delete-file (find-files "setuptools" - "^(cli|gui).*\\.exe$"))))) - (build-system python-build-system) - ;; FIXME: Tests require pytest, which itself relies on setuptools. - ;; One could bootstrap with an internal untested setuptools. - (arguments (list #:tests? #f)) - (home-page "https://pypi.org/project/setuptools/") - (synopsis "Library designed to facilitate packaging Python projects") - (description "Setuptools is a fully-featured, stable library designed to -facilitate packaging Python projects, where packaging includes: -@itemize -@item Python package and module definitions -@item distribution package metadata -@item test hooks -@item project installation -@item platform-specific details. -@end itemize") - ;; TODO: setuptools now bundles the following libraries: - ;; packaging, pyparsing, six and appdirs. How to unbundle? - (license (list license:psfl ;setuptools itself - license:expat ;six, appdirs, pyparsing - license:asl2.0 ;packaging is dual ASL2/BSD-2 - license:bsd-2)))) - (define-public python-setuptools-declarative-requirements (package (name "python-setuptools-declarative-requirements") diff --git a/gnu/packages/sequoia.scm b/gnu/packages/sequoia.scm index a8d21ac1a4..7002f684b8 100644 --- a/gnu/packages/sequoia.scm +++ b/gnu/packages/sequoia.scm @@ -36,7 +36,7 @@ (define-module (gnu packages sequoia) #:use-module (gnu packages nettle) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) - #:use-module (gnu packages python-xyz) ;; python-setuptools + #:use-module (gnu packages python-build) ;python-setuptools #:use-module (gnu packages rust) #:use-module (gnu packages tls)) diff --git a/tests/lint.scm b/tests/lint.scm index 8be74d2604..641486f89b 100644 --- a/tests/lint.scm +++ b/tests/lint.scm @@ -49,7 +49,7 @@ (define-module (test-lint) #:use-module (gnu packages) #:use-module (gnu packages glib) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) + #:use-module (gnu packages python-build) #:use-module ((gnu packages bash) #:select (bash bash-minimal)) #:use-module (web uri) #:use-module (web server) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-27 13:28 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke ` (12 subsequent siblings) 13 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This removes the need to import (gnu packages python-xyz) in (gnu packages python), avoiding issues with circular imports. * gnu/packages/python.scm (pypy): Move… * gnu/packages/pypy.scm (pypy): …here * gnu/local.mk: Register new file. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/local.mk | 1 + gnu/packages/pypy.scm | 273 ++++++++++++++++++++++++++++++++++++++++ gnu/packages/python.scm | 167 ------------------------ 3 files changed, 274 insertions(+), 167 deletions(-) create mode 100644 gnu/packages/pypy.scm diff --git a/gnu/local.mk b/gnu/local.mk index 8247180bef..bf598cec8b 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -506,6 +506,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/pure.scm \ %D%/packages/purescript.scm \ %D%/packages/pv.scm \ + %D%/packages/pypy.scm \ %D%/packages/python.scm \ %D%/packages/python-build.scm \ %D%/packages/python-check.scm \ diff --git a/gnu/packages/pypy.scm b/gnu/packages/pypy.scm new file mode 100644 index 0000000000..002cfd59cd --- /dev/null +++ b/gnu/packages/pypy.scm @@ -0,0 +1,273 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> +;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2021 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013, 2014, 2015, 2016 Andreas Enge <andreas@enge.fr> +;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org> +;;; Copyright © 2014, 2017, 2019 Eric Bavier <bavier@member.fsf.org> +;;; Copyright © 2014, 2015 Federico Beffa <beffa@fbengineering.ch> +;;; Copyright © 2015 Omar Radwan <toxemicsquire4@gmail.com> +;;; Copyright © 2015 Pierre-Antoine Rault <par@rigelk.eu> +;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net> +;;; Copyright © 2015, 2016 Christine Lemmer-Webber <cwebber@dustycloud.org> +;;; Copyright © 2015 Eric Dvorsak <eric@dvorsak.fr> +;;; Copyright © 2015, 2016 David Thompson <davet@gnu.org> +;;; Copyright © 2015, 2016, 2017, 2021 Leo Famulari <leo@famulari.name> +;;; Copyright © 2015, 2017 Ben Woodcroft <donttrustben@gmail.com> +;;; Copyright © 2015, 2016 Erik Edrosa <erik.edrosa@gmail.com> +;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il> +;;; Copyright © 2015, 2017 Kyle Meyer <kyle@kyleam.com> +;;; Copyright © 2015, 2016 Chris Marusich <cmmarusich@gmail.com> +;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org> +;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox.org> +;;; Copyright © 2016, 2018 Hartmut Goebel <h.goebel@crazy-compilers.com> +;;; Copyright © 2016 Daniel Pimentel <d4n1@d4n1.org> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@gmail.com> +;;; Copyright © 2016, 2017 Troy Sankey <sankeytms@gmail.com> +;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016 David Craven <david@craven.ch> +;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> +;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> +;;; Copyright © 2016–2018, 2021 Tobias Geerinckx-Rice <me@tobias.gr> +;;; Copyright © 2016, 2017, 2018, 2021 Julien Lepiller <julien@lepiller.eu> +;;; Copyright © 2016, 2017 Thomas Danckaert <post@thomasdanckaert.be> +;;; Copyright © 2017 Carlo Zancanaro <carlo@zancanaro.id.au> +;;; Copyright © 2017 Frederick M. Muriithi <fredmanglis@gmail.com> +;;; Copyright © 2017, 2018 Adriano Peluso <catonano@gmail.com> +;;; Copyright © 2017 Ben Sturmfels <ben@sturm.com.au> +;;; Copyright © 2017, 2018, 2019 Mathieu Othacehe <m.othacehe@gmail.com> +;;; Copyright © 2017 José Miguel Sánchez García <jmi2k@openmailbox.org> +;;; Copyright © 2017 Roel Janssen <roel@gnu.org> +;;; Copyright © 2017, 2018 Kei Kebreau <kkebreau@posteo.net> +;;; Copyright © 2017 Rutger Helling <rhelling@mykolab.com> +;;; Copyright © 2017 Muriithi Frederick Muriuki <fredmanglis@gmail.com> +;;; Copyright © 2017 Brendan Tildesley <mail@brendan.scot> +;;; Copyright © 2018 Ethan R. Jones <ethanrjones97@gmail.com +;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com> +;;; Copyright © 2018 Vijayalakshmi Vedantham <vijimay12@gmail.com> +;;; Copyright © 2018 Mathieu Lirzin <mthl@gnu.org> +;;; Copyright © 2018 Adam Massmann <massmannak@gmail.com> +;;; Copyright © 2016, 2018 Tomáš Čech <sleep_walker@gnu.org> +;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr> +;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com> +;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org> +;;; Copyright © 2018, 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> +;;; Copyright © 2018 Luther Thompson <lutheroto@gmail.com> +;;; Copyright © 2018 Vagrant Cascadian <vagrant@debian.org> +;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org> +;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> +;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu packages pypy) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (gnu packages) + #:use-module (gnu packages base) + #:use-module (gnu packages bash) + #:use-module (gnu packages certs) + #:use-module (gnu packages check) + #:use-module (gnu packages compression) + #:use-module (gnu packages dbm) + #:use-module (gnu packages hurd) + #:use-module (gnu packages libffi) + #:use-module (gnu packages ncurses) + #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) + #:use-module (gnu packages readline) + #:use-module (gnu packages shells) + #:use-module (gnu packages sqlite) + #:use-module (gnu packages tcl) + #:use-module (gnu packages tls) + #:use-module (gnu packages xml) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix utils) + #:use-module (guix build-system gnu) + #:use-module (guix build-system trivial) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26)) + +(define-public pypy + (package + (name "pypy") + (version "7.3.5") + (source (origin + (method url-fetch) + (uri (string-append "https://downloads.python.org/pypy/" + "pypy3.7-v" version "-src.tar.bz2")) + (sha256 + (base32 + "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) + (build-system gnu-build-system) + (arguments + (list + #:tests? #f ;FIXME: 43 out of 364 tests are failing + #:modules '((ice-9 ftw) (ice-9 match) + (guix build utils) (guix build gnu-build-system)) + #:disallowed-references (list nss-certs) + #:phases + #~(modify-phases %standard-phases + (delete 'configure) + (add-after 'unpack 'patch-source + (lambda* (#:key inputs #:allow-other-keys) + (substitute* '("rpython/rlib/clibffi.py") + ;; find_library does not work for libc + (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) + (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") + ;; Make reproducible (XXX: unused?) + (("time\\.localtime\\(\\)") "time.gmtime(0)")) + (substitute* '("pypy/module/sys/version.py") + ;; Make reproducible + (("t\\.gmtime\\(\\)") "t.gmtime(0)")) + (substitute* '("lib_pypy/_tkinter/tklib_build.py") + ;; Link to versioned libtcl and libtk + (("linklibs = \\['tcl', 'tk'\\]") + "linklibs = ['tcl8.6', 'tk8.6']") + (("incdirs = \\[\\]") + (string-append + "incdirs = ['" + #$(this-package-input "tcl") "/include', '" + #$(this-package-input "tk") "/include']"))) + (substitute* '("lib_pypy/_curses_build.py") + ;; Find curses + (("/usr/local") + #$(this-package-input "ncurses"))) + (substitute* '("lib_pypy/_dbm.py") + ;; Use gdbm compat library, so we don’t need to pull + ;; in bdb. + (("ctypes.util.find_library\\('db'\\)") + (format #f "~s" (search-input-file + inputs "lib/libgdbm_compat.so")))) + (substitute* '("lib_pypy/_sqlite3_build.py") + ;; Always use search paths + (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") + ;; Find sqlite3 + (("/usr/local") (assoc-ref inputs "sqlite")) + (("libname = 'sqlite3'") + (format #f "libname = ~s" + (search-input-file inputs "lib/libsqlite3.so.0")))) + (substitute* '("lib-python/3/subprocess.py") + ;; Fix shell path + (("/bin/sh") + (search-input-file inputs "/bin/sh"))) + (substitute* '("lib-python/3/distutils/unixccompiler.py") + ;; gcc-toolchain does not provide symlink cc -> gcc + (("\"cc\"") "\"gcc\"")))) + (add-after 'unpack 'set-source-file-times-to-1980 + ;; copied from python package, required by zip testcase + (lambda _ + (let ((circa-1980 (* 10 366 24 60 60))) + (ftw "." (lambda (file stat flag) + (utime file circa-1980 circa-1980) + #t))))) + (replace 'build + (lambda _ + (with-directory-excursion "pypy/goal" + ;; Build with jit optimization. + (invoke "python2" + "../../rpython/bin/rpython" + (string-append "--make-jobs=" + (number->string (parallel-job-count))) + "-Ojit" + "targetpypystandalone" + "--allworkingmodules")) + ;; Build c modules and package everything, so tests work. + (with-directory-excursion "pypy/tool/release" + (invoke "python2" "package.py" + "--archive-name" "pypy-dist" + "--builddir" (getcwd))))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (if tests? + (begin + (setenv "HOME" "/tmp") ; test_with_pip tries to + ; access ~/.cache/pip + ;; Run library tests only (no interpreter unit tests). + ;; This is what Gentoo does. + (invoke "python" "pypy/test_all.py" + "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" + "lib-python")) + (format #t "test suite not run~%")))) + (replace 'install + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) + (shebang-match-python "#!.+/bin/python") + (shebang-pypy3 (string-append "#!" bin-pypy3)) + (dist-dir "pypy/tool/release/pypy-dist")) + (with-directory-excursion dist-dir + ;; Delete test data. + (for-each + (lambda (x) + (delete-file-recursively (string-append + "lib-python/3/" x))) + '("tkinter/test" + "test" + "sqlite3/test" + "lib2to3/tests" + "idlelib/idle_test" + "distutils/tests" + "ctypes/test" + "unittest/test")) + ;; Patch shebang referencing python. + (substitute* '("lib-python/3/cgi.py" + "lib-python/3/encodings/rot_13.py") + ((shebang-match-python) shebang-pypy3)) + (with-fluids ((%default-port-encoding "ISO-8859-1")) + (substitute* '("lib_pypy/_md5.py" + "lib_pypy/_sha1.py") + ((shebang-match-python) shebang-pypy3)))) + (copy-recursively dist-dir #$output))))))) + (native-inputs + (list gzip + nss-certs ; For ssl tests + pkg-config + python-2 + python2-pycparser + tar)) ; Required for package.py + (inputs + (list bzip2 + expat + gdbm + glibc + libffi + ncurses + openssl + sqlite + tcl + tk + xz + zlib)) + (home-page "https://www.pypy.org/") + (synopsis "Python implementation with just-in-time compilation") + (description "PyPy is a faster, alternative implementation of the Python +programming language employing a just-in-time compiler. It supports most +Python code natively, including C extensions.") + (license (list license:expat ; pypy itself; _pytest/ + license:psfl ; python standard library in lib-python/ + license:asl2.0 ; dotviewer/font/ and some of lib-python/ + license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* + license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ + (license:non-copyleft + "http://www.unicode.org/copyright.html"))))) + +(define-public pypy3 + (deprecated-package "pypy3" pypy)) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index e31158dfc0..b54e393eb4 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -89,7 +89,6 @@ (define-module (gnu packages python) #:use-module (gnu packages libffi) #:use-module (gnu packages ncurses) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) #:use-module (gnu packages shells) #:use-module (gnu packages sqlite) @@ -736,169 +735,3 @@ (define-public micropython compatible with normal Python as possible to allow you to transfer code with ease from the desktop to a microcontroller or embedded system.") (license license:expat))) - -(define-public pypy - (package - (name "pypy") - (version "7.3.5") - (source (origin - (method url-fetch) - (uri (string-append "https://downloads.python.org/pypy/" - "pypy3.7-v" version "-src.tar.bz2")) - (sha256 - (base32 - "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) - (build-system gnu-build-system) - (arguments - (list - #:tests? #f ;FIXME: 43 out of 364 tests are failing - #:modules '((ice-9 ftw) (ice-9 match) - (guix build utils) (guix build gnu-build-system)) - #:disallowed-references (list nss-certs) - #:phases - #~(modify-phases %standard-phases - (delete 'configure) - (add-after 'unpack 'patch-source - (lambda* (#:key inputs #:allow-other-keys) - (substitute* '("rpython/rlib/clibffi.py") - ;; find_library does not work for libc - (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) - (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") - ;; Make reproducible (XXX: unused?) - (("time\\.localtime\\(\\)") "time.gmtime(0)")) - (substitute* '("pypy/module/sys/version.py") - ;; Make reproducible - (("t\\.gmtime\\(\\)") "t.gmtime(0)")) - (substitute* '("lib_pypy/_tkinter/tklib_build.py") - ;; Link to versioned libtcl and libtk - (("linklibs = \\['tcl', 'tk'\\]") - "linklibs = ['tcl8.6', 'tk8.6']") - (("incdirs = \\[\\]") - (string-append - "incdirs = ['" - #$(this-package-input "tcl") "/include', '" - #$(this-package-input "tk") "/include']"))) - (substitute* '("lib_pypy/_curses_build.py") - ;; Find curses - (("/usr/local") - #$(this-package-input "ncurses"))) - (substitute* '("lib_pypy/_dbm.py") - ;; Use gdbm compat library, so we don’t need to pull - ;; in bdb. - (("ctypes.util.find_library\\('db'\\)") - (format #f "~s" (search-input-file - inputs "lib/libgdbm_compat.so")))) - (substitute* '("lib_pypy/_sqlite3_build.py") - ;; Always use search paths - (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") - ;; Find sqlite3 - (("/usr/local") (assoc-ref inputs "sqlite")) - (("libname = 'sqlite3'") - (format #f "libname = ~s" - (search-input-file inputs "lib/libsqlite3.so.0")))) - (substitute* '("lib-python/3/subprocess.py") - ;; Fix shell path - (("/bin/sh") - (search-input-file inputs "/bin/sh"))) - (substitute* '("lib-python/3/distutils/unixccompiler.py") - ;; gcc-toolchain does not provide symlink cc -> gcc - (("\"cc\"") "\"gcc\"")))) - (add-after 'unpack 'set-source-file-times-to-1980 - ;; copied from python package, required by zip testcase - (lambda _ - (let ((circa-1980 (* 10 366 24 60 60))) - (ftw "." (lambda (file stat flag) - (utime file circa-1980 circa-1980) - #t))))) - (replace 'build - (lambda _ - (with-directory-excursion "pypy/goal" - ;; Build with jit optimization. - (invoke "python2" - "../../rpython/bin/rpython" - (string-append "--make-jobs=" - (number->string (parallel-job-count))) - "-Ojit" - "targetpypystandalone" - "--allworkingmodules")) - ;; Build c modules and package everything, so tests work. - (with-directory-excursion "pypy/tool/release" - (invoke "python2" "package.py" - "--archive-name" "pypy-dist" - "--builddir" (getcwd))))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (if tests? - (begin - (setenv "HOME" "/tmp") ; test_with_pip tries to - ; access ~/.cache/pip - ;; Run library tests only (no interpreter unit tests). - ;; This is what Gentoo does. - (invoke "python" "pypy/test_all.py" - "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" - "lib-python")) - (format #t "test suite not run~%")))) - (replace 'install - (lambda* (#:key inputs outputs #:allow-other-keys) - (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) - (shebang-match-python "#!.+/bin/python") - (shebang-pypy3 (string-append "#!" bin-pypy3)) - (dist-dir "pypy/tool/release/pypy-dist")) - (with-directory-excursion dist-dir - ;; Delete test data. - (for-each - (lambda (x) - (delete-file-recursively (string-append - "lib-python/3/" x))) - '("tkinter/test" - "test" - "sqlite3/test" - "lib2to3/tests" - "idlelib/idle_test" - "distutils/tests" - "ctypes/test" - "unittest/test")) - ;; Patch shebang referencing python. - (substitute* '("lib-python/3/cgi.py" - "lib-python/3/encodings/rot_13.py") - ((shebang-match-python) shebang-pypy3)) - (with-fluids ((%default-port-encoding "ISO-8859-1")) - (substitute* '("lib_pypy/_md5.py" - "lib_pypy/_sha1.py") - ((shebang-match-python) shebang-pypy3)))) - (copy-recursively dist-dir #$output))))))) - (native-inputs - (list gzip - nss-certs ; For ssl tests - pkg-config - python-2 - python2-pycparser - tar)) ; Required for package.py - (inputs - (list bzip2 - expat - gdbm - glibc - libffi - ncurses - openssl - sqlite - tcl - tk - xz - zlib)) - (home-page "https://www.pypy.org/") - (synopsis "Python implementation with just-in-time compilation") - (description "PyPy is a faster, alternative implementation of the Python -programming language employing a just-in-time compiler. It supports most -Python code natively, including C extensions.") - (license (list license:expat ; pypy itself; _pytest/ - license:psfl ; python standard library in lib-python/ - license:asl2.0 ; dotviewer/font/ and some of lib-python/ - license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* - license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ - (license:non-copyleft - "http://www.unicode.org/copyright.html"))))) - -(define-public pypy3 - (deprecated-package "pypy3" pypy)) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module. 2022-10-17 20:11 ` [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module Marius Bakke @ 2022-10-27 13:28 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 13:28 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587, Lars-Dominik Braun Hi, Marius Bakke <marius@gnu.org> writes: > From: Lars-Dominik Braun <lars@6xq.net> > > This removes the need to import (gnu packages python-xyz) in (gnu packages > python), avoiding issues with circular imports. > > * gnu/packages/python.scm (pypy): Move… > * gnu/packages/pypy.scm (pypy): …here > * gnu/local.mk: Register new file. > > Co-authored-by: Marius Bakke <marius@gnu.org> > --- > gnu/local.mk | 1 + > gnu/packages/pypy.scm | 273 ++++++++++++++++++++++++++++++++++++++++ > gnu/packages/python.scm | 167 ------------------------ > 3 files changed, 274 insertions(+), 167 deletions(-) > create mode 100644 gnu/packages/pypy.scm > > diff --git a/gnu/local.mk b/gnu/local.mk > index 8247180bef..bf598cec8b 100644 > --- a/gnu/local.mk > +++ b/gnu/local.mk > @@ -506,6 +506,7 @@ GNU_SYSTEM_MODULES = \ > %D%/packages/pure.scm \ > %D%/packages/purescript.scm \ > %D%/packages/pv.scm \ > + %D%/packages/pypy.scm \ > %D%/packages/python.scm \ > %D%/packages/python-build.scm \ > %D%/packages/python-check.scm \ > diff --git a/gnu/packages/pypy.scm b/gnu/packages/pypy.scm > new file mode 100644 > index 0000000000..002cfd59cd > --- /dev/null > +++ b/gnu/packages/pypy.scm > @@ -0,0 +1,273 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> > +;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2021 Ludovic Courtès <ludo@gnu.org> > +;;; Copyright © 2013, 2014, 2015, 2016 Andreas Enge <andreas@enge.fr> > +;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org> > +;;; Copyright © 2014, 2017, 2019 Eric Bavier <bavier@member.fsf.org> > +;;; Copyright © 2014, 2015 Federico Beffa <beffa@fbengineering.ch> > +;;; Copyright © 2015 Omar Radwan <toxemicsquire4@gmail.com> > +;;; Copyright © 2015 Pierre-Antoine Rault <par@rigelk.eu> > +;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net> > +;;; Copyright © 2015, 2016 Christine Lemmer-Webber <cwebber@dustycloud.org> > +;;; Copyright © 2015 Eric Dvorsak <eric@dvorsak.fr> > +;;; Copyright © 2015, 2016 David Thompson <davet@gnu.org> > +;;; Copyright © 2015, 2016, 2017, 2021 Leo Famulari <leo@famulari.name> > +;;; Copyright © 2015, 2017 Ben Woodcroft <donttrustben@gmail.com> > +;;; Copyright © 2015, 2016 Erik Edrosa <erik.edrosa@gmail.com> > +;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il> > +;;; Copyright © 2015, 2017 Kyle Meyer <kyle@kyleam.com> > +;;; Copyright © 2015, 2016 Chris Marusich <cmmarusich@gmail.com> > +;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org> > +;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox.org> > +;;; Copyright © 2016, 2018 Hartmut Goebel <h.goebel@crazy-compilers.com> > +;;; Copyright © 2016 Daniel Pimentel <d4n1@d4n1.org> > +;;; Copyright © 2016 Sou Bunnbu <iyzsong@gmail.com> > +;;; Copyright © 2016, 2017 Troy Sankey <sankeytms@gmail.com> > +;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> > +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> > +;;; Copyright © 2016 David Craven <david@craven.ch> > +;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> > +;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> > +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> > +;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> > +;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> > +;;; Copyright © 2016–2018, 2021 Tobias Geerinckx-Rice <me@tobias.gr> > +;;; Copyright © 2016, 2017, 2018, 2021 Julien Lepiller <julien@lepiller.eu> > +;;; Copyright © 2016, 2017 Thomas Danckaert <post@thomasdanckaert.be> > +;;; Copyright © 2017 Carlo Zancanaro <carlo@zancanaro.id.au> > +;;; Copyright © 2017 Frederick M. Muriithi <fredmanglis@gmail.com> > +;;; Copyright © 2017, 2018 Adriano Peluso <catonano@gmail.com> > +;;; Copyright © 2017 Ben Sturmfels <ben@sturm.com.au> > +;;; Copyright © 2017, 2018, 2019 Mathieu Othacehe <m.othacehe@gmail.com> > +;;; Copyright © 2017 José Miguel Sánchez García <jmi2k@openmailbox.org> > +;;; Copyright © 2017 Roel Janssen <roel@gnu.org> > +;;; Copyright © 2017, 2018 Kei Kebreau <kkebreau@posteo.net> > +;;; Copyright © 2017 Rutger Helling <rhelling@mykolab.com> > +;;; Copyright © 2017 Muriithi Frederick Muriuki <fredmanglis@gmail.com> > +;;; Copyright © 2017 Brendan Tildesley <mail@brendan.scot> > +;;; Copyright © 2018 Ethan R. Jones <ethanrjones97@gmail.com > +;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com> > +;;; Copyright © 2018 Vijayalakshmi Vedantham <vijimay12@gmail.com> > +;;; Copyright © 2018 Mathieu Lirzin <mthl@gnu.org> > +;;; Copyright © 2018 Adam Massmann <massmannak@gmail.com> > +;;; Copyright © 2016, 2018 Tomáš Čech <sleep_walker@gnu.org> > +;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr> > +;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com> > +;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org> > +;;; Copyright © 2018, 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> > +;;; Copyright © 2018 Luther Thompson <lutheroto@gmail.com> > +;;; Copyright © 2018 Vagrant Cascadian <vagrant@debian.org> > +;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org> > +;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> > +;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> I really doubt all these individuals have contributed to *pypy*. The rest of this patch LGTM. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build). 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-27 13:29 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system Marius Bakke ` (11 subsequent siblings) 13 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-pip): Move from here ... * gnu/packages/python-build.scm (python-pip): ... to here. --- gnu/packages/python-build.scm | 21 +++++++++++++++++++++ gnu/packages/python-xyz.scm | 21 --------------------- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 3ff988820f..c8f57ea286 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,27 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-pip + (package + (name "python-pip") + (version "22.2.2") + (source + (origin + (method url-fetch) + (uri (pypi-uri "pip" version)) + (sha256 + (base32 + "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) + (build-system python-build-system) + (arguments + '(#:tests? #f)) ; there are no tests in the pypi archive. + (home-page "https://pip.pypa.io/") + (synopsis "Package manager for Python software") + (description + "Pip is a package manager for Python software, that finds packages on the +Python Package Index (PyPI).") + (license license:expat))) + (define-public python-setuptools (package (name "python-setuptools") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 3ea29ec3b9..1ded6f0384 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -12650,27 +12650,6 @@ (define-public python-pretend responses, rather than doing any computation.") (license license:bsd-3))) -(define-public python-pip - (package - (name "python-pip") - (version "22.2.2") - (source - (origin - (method url-fetch) - (uri (pypi-uri "pip" version)) - (sha256 - (base32 - "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) - (build-system python-build-system) - (arguments - '(#:tests? #f)) ; there are no tests in the pypi archive. - (home-page "https://pip.pypa.io/") - (synopsis "Package manager for Python software") - (description - "Pip is a package manager for Python software, that finds packages on the -Python Package Index (PyPI).") - (license license:expat))) - ;;; Variant used to break a dependency cycle with ;;; python-pytest-perf-bootstrap. (define-public python-pip-run-bootstrap -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build). 2022-10-17 20:11 ` [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke @ 2022-10-27 13:29 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 13:29 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587 Hi, Marius Bakke <marius@gnu.org> writes: > * gnu/packages/python-xyz.scm (python-pip): Move from here ... > * gnu/packages/python-build.scm (python-pip): ... to here. LGTM. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-27 14:23 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system Marius Bakke ` (10 subsequent siblings) 13 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This is an experimental build system based on python-build-system that implements PEP 517-compliant builds. * guix/build-system/pyproject.scm, guix/build/pyproject-build-system.scm, gnu/packages/aux-files/python/sanity-check-next.py, gnu/packages/python-commencement.scm: New files. * Makefile.am (MODULES): Register the new build systems. * gnu/local.mk (GNU_SYSTEM_MODULES): Add python-commencement.scm. * gnu/packages/python.scm (python-sans-pip, python-sans-pip-wrapper): New variables. Co-authored-by: Marius Bakke <marius@gnu.org> --- Makefile.am | 2 + gnu/local.mk | 1 + .../aux-files/python/sanity-check-next.py | 98 ++++ gnu/packages/python-commencement.scm | 64 +++ gnu/packages/python.scm | 13 + guix/build-system/pyproject.scm | 148 ++++++ guix/build/pyproject-build-system.scm | 460 ++++++++++++++++++ 7 files changed, 786 insertions(+) create mode 100644 gnu/packages/aux-files/python/sanity-check-next.py create mode 100644 gnu/packages/python-commencement.scm create mode 100644 guix/build-system/pyproject.scm create mode 100644 guix/build/pyproject-build-system.scm diff --git a/Makefile.am b/Makefile.am index 22dcc43f99..6ccb790c11 100644 --- a/Makefile.am +++ b/Makefile.am @@ -166,6 +166,7 @@ MODULES = \ guix/build-system/maven.scm \ guix/build-system/node.scm \ guix/build-system/perl.scm \ + guix/build-system/pyproject.scm \ guix/build-system/python.scm \ guix/build-system/renpy.scm \ guix/build-system/ocaml.scm \ @@ -222,6 +223,7 @@ MODULES = \ guix/build/minetest-build-system.scm \ guix/build/node-build-system.scm \ guix/build/perl-build-system.scm \ + guix/build/pyproject-build-system.scm \ guix/build/python-build-system.scm \ guix/build/ocaml-build-system.scm \ guix/build/qt-build-system.scm \ diff --git a/gnu/local.mk b/gnu/local.mk index bf598cec8b..b6e0d262be 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -511,6 +511,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/python-build.scm \ %D%/packages/python-check.scm \ %D%/packages/python-compression.scm \ + %D%/packages/python-commencement.scm \ %D%/packages/python-crypto.scm \ %D%/packages/python-science.scm \ %D%/packages/python-web.scm \ diff --git a/gnu/packages/aux-files/python/sanity-check-next.py b/gnu/packages/aux-files/python/sanity-check-next.py new file mode 100644 index 0000000000..891606f72b --- /dev/null +++ b/gnu/packages/aux-files/python/sanity-check-next.py @@ -0,0 +1,98 @@ +# -*- coding: utf-8 -*- + +# This version adds a small change to accomodate missing python-setuptools. +# Original patch by Lars-Dominik Braun in wip-python-pep517, commit +# 720dbe22d431262938be29dd9a9ddb78c44a99b3. +# --- sanity-check.py 2022-06-12 14:40:06.814337702 +0200 +# +++ sanity-check.py 2022-10-16 23:21:38.990651568 +0200 +# @@ -19,9 +19,13 @@ +# from __future__ import print_function # Python 2 support. +# import importlib +# -import pkg_resources +# import sys +# import traceback +# +try: +# + import pkg_resources +# +except ImportError: +# + print('Warning: Skipping, because python-setuptools are not available.') +# + sys.exit(0) + +# TODO: Merge with sanity-check.py in the next core-updates cycle. + +from __future__ import print_function # Python 2 support. +import importlib +import sys +import traceback +try: + import pkg_resources +except ImportError: + print('Warning: Skipping, because python-setuptools are not available.') + sys.exit(0) + +try: + from importlib.machinery import PathFinder +except ImportError: + PathFinder = None + +ret = 0 + +# Only check site-packages installed by this package, but not dependencies +# (which pkg_resources.working_set would include). Path supplied via argv. +ws = pkg_resources.find_distributions(sys.argv[1]) + +for dist in ws: + print('validating', repr(dist.project_name), dist.location) + try: + print('...checking requirements: ', end='') + req = str(dist.as_requirement()) + # dist.activate() is not enough to actually check requirements, we + # have to .require() it. + pkg_resources.require(req) + print('OK') + except Exception as e: + print('ERROR:', req, repr(e)) + ret = 1 + continue + + # Try to load top level modules. This should not have any side-effects. + try: + metalines = dist.get_metadata_lines('top_level.txt') + except (KeyError, EnvironmentError): + # distutils (i.e. #:use-setuptools? #f) will not install any metadata. + # This file is also missing for packages built using a PEP 517 builder + # such as poetry. + print('WARNING: cannot determine top-level modules') + continue + for name in metalines: + # Only available on Python 3. + if PathFinder and PathFinder.find_spec(name) is None: + # Ignore unavailable modules, often C modules, which were not + # installed at the top-level. Cannot use ModuleNotFoundError, + # because it is raised by failed imports too. + continue + try: + print('...trying to load module', name, end=': ') + importlib.import_module(name) + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + + # Try to load entry points of console scripts too, making sure they + # work. They should be removed if they don't. Other groups may not be + # safe, as they can depend on optional packages. + for group, v in dist.get_entry_map().items(): + if group not in {'console_scripts', 'gui_scripts'}: + continue + for name, ep in v.items(): + try: + print('...trying to load endpoint', group, name, end=': ') + ep.load() + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + +sys.exit(ret) diff --git a/gnu/packages/python-commencement.scm b/gnu/packages/python-commencement.scm new file mode 100644 index 0000000000..011ba2c38d --- /dev/null +++ b/gnu/packages/python-commencement.scm @@ -0,0 +1,64 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu packages python-commencement) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (guix packages) + #:use-module (guix gexp) + #:use-module (guix build-system trivial) + #:use-module (gnu packages) + #:use-module (gnu packages python) + #:use-module (gnu packages python-build) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26)) + +(define-public python-toolchain + (package + (name "python-toolchain") + (version (package-version python)) + (source #f) + (build-system trivial-build-system) + (arguments + (list #:modules '((guix build union)) + #:builder + #~(begin + (use-modules (ice-9 match) + (srfi srfi-1) + (guix build union)) + (union-build #$output + (filter-map (match-lambda + ((_ . directory) directory)) + %build-inputs))))) + (inputs + (list python-sans-pip-wrapper + python-pypa-build + python-pip + python-setuptools + python-wheel)) + (native-search-paths + (package-native-search-paths python)) + (search-paths + (package-search-paths python)) + (license (package-license python)) + (synopsis "Python toolchain") + (description + "Python toolchain including Python itself, setuptools and pip. Use this +package if you need a minimal Python toolchain instead of just the +interpreter.") + (home-page (package-home-page python)))) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index b54e393eb4..377fd1c282 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -677,6 +677,19 @@ (define* (wrap-python3 python (define-public python-wrapper (wrap-python3 python)) (define-public python-minimal-wrapper (wrap-python3 python-minimal)) +;; The Python used in pyproject-build-system. +(define-public python-sans-pip + (hidden-package + (package/inherit python + (arguments + (substitute-keyword-arguments (package-arguments python) + ((#:configure-flags flags #~()) + #~(append '("--with-ensurepip=no") + (delete "--with-ensurepip=install" #$flags)))))))) + +(define-public python-sans-pip-wrapper + (wrap-python3 python-sans-pip)) + (define-public micropython (package (name "micropython") diff --git a/guix/build-system/pyproject.scm b/guix/build-system/pyproject.scm new file mode 100644 index 0000000000..8e320529cc --- /dev/null +++ b/guix/build-system/pyproject.scm @@ -0,0 +1,148 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build-system pyproject) + #:use-module ((gnu packages) #:select (search-auxiliary-file)) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (guix memoization) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix packages) + #:use-module (guix derivations) + #:use-module (guix search-paths) + #:use-module (guix build-system) + #:use-module (guix build-system gnu) + #:use-module (guix build-system python) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (%pyproject-build-system-modules + default-python + pyproject-build + pyproject-build-system)) + +;; Commentary: +;; +;; Standard build procedure for Python packages using 'pyproject.toml'. +;; This is implemented as an extension of 'python-build-system'. +;; +;; Code: + +(define %pyproject-build-system-modules + ;; Build-side modules imported by default. + `((guix build pyproject-build-system) + (guix build json) + ,@%python-build-system-modules + ,@%gnu-build-system-modules)) + +(define (default-python) + "Return the default Python package." + ;; Lazily resolve the binding to avoid a circular dependency. + (let ((python (resolve-interface '(gnu packages python-commencement)))) + (module-ref python 'python-toolchain))) + +(define sanity-check.py + ;; TODO: Merge with sanity-check.py in the next rebuild cycle. + (search-auxiliary-file "python/sanity-check-next.py")) + +(define* (lower name + #:key source inputs native-inputs outputs system target + (python (default-python)) + #:allow-other-keys + #:rest arguments) + "Return a bag for NAME." + (define private-keywords + '(#:target #:python #:inputs #:native-inputs)) + + (and (not target) ;XXX: no cross-compilation + (bag + (name name) + (system system) + (host-inputs `(,@(if source + `(("source" ,source)) + '()) + ,@inputs + + ;; Keep the standard inputs of 'gnu-build-system'. + ,@(standard-packages))) + (build-inputs `(("python" ,python) + ("sanity-check.py" ,(local-file sanity-check.py)) + ,@native-inputs)) + (outputs (append outputs '(wheel))) + (build pyproject-build) + (arguments (strip-keyword-arguments private-keywords arguments))))) + +(define* (pyproject-build name inputs + #:key source + (tests? #t) + (configure-flags ''()) + (build-backend #f) + (test-backend #f) + (test-flags #f) + (phases '%standard-phases) + (outputs '("out" "wheel")) + (search-paths '()) + (system (%current-system)) + (guile #f) + (imported-modules %pyproject-build-system-modules) + (modules '((guix build pyproject-build-system) + (guix build utils)))) + "Build SOURCE using PYTHON, and with INPUTS." + (define build + (with-imported-modules imported-modules + #~(begin + (use-modules #$@(sexp->gexp modules)) + + #$(with-build-variables inputs outputs + #~(pyproject-build + #:name #$name + #:source #+source + #:configure-flags #$configure-flags + #:system #$system + #:build-backend #$build-backend + #:test-backend #$test-backend + #:test-flags #$test-flags + #:tests? #$tests? + #:phases #$(if (pair? phases) + (sexp->gexp phases) + phases) + #:outputs %outputs + #:search-paths '#$(sexp->gexp + (map search-path-specification->sexp + search-paths)) + #:inputs %build-inputs))))) + + + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system #:graft? #f))) + (gexp->derivation name build + #:system system + #:graft? #f ;consistent with 'gnu-build' + #:target #f + #:guile-for-build guile))) + +(define pyproject-build-system + (build-system + (name 'pyproject) + (description "The PEP517-compliant Python build system") + (lower lower))) + +;;; pyproject.scm ends here diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm new file mode 100644 index 0000000000..141d4d849f --- /dev/null +++ b/guix/build/pyproject-build-system.scm @@ -0,0 +1,460 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2013, 2015, 2016, 2018, 2019 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013 Andreas Enge <andreas@enge.fr> +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> +;;; Copyright © 2015, 2018 Mark H Weaver <mhw@netris.org> +;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com> +;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net> +;;; Copyright © 2018 Arun Isaac <arunisaac@systemreboot.net> +;;; Copyright © 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> +;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> +;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il> +;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be> +;;; +;;; The above copyright is taken from python-build-system. Contributions +;;; made only to this module are listed below: +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build pyproject-build-system) + #:use-module ((guix build gnu-build-system) #:prefix gnu:) + #:use-module (guix build utils) + #:use-module (guix build json) + #:use-module (ice-9 match) + #:use-module (ice-9 ftw) + #:use-module (ice-9 format) + #:use-module (ice-9 rdelim) + #:use-module (ice-9 regex) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:export (%standard-phases + add-installed-pythonpath + site-packages + python-version + pyproject-build)) + +;; Commentary: +;; +;; PEP 517-compatible build system for Python packages. +;; +;; PEP 517 mandates the use of a TOML file called pyproject.toml at the +;; project root, describing build and runtime dependencies, as well as the +;; build system, which can be different from setuptools. This module uses +;; that file to extract the build system used and call its wheel-building +;; entry point build_wheel (see 'build). setuptools’ wheel builder is +;; used as a fallback if either no pyproject.toml exists or it does not +;; declare a build-system. It supports config_settings through the +;; standard #:configure-flags argument. +;; +;; This wheel, which is just a ZIP file with a file structure defined +;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked +;; and its contents are moved to the appropriate locations in 'install. +;; +;; Then entry points, as defined by the PyPa Entry Point Specification +;; (https://packaging.python.org/specifications/entry-points/) are read +;; from a file called entry_points.txt in the package’s site-packages +;; subdirectory and scripts are written to bin/. These are not part of a +;; wheel and expected to be created by the installing utility. +;; +;; Caveats: +;; - There is no support for in-tree build backends. + +;; Base error type. +(define-condition-type &python-build-error &error + python-build-error?) + +;; Raised when 'check cannot find a valid test system in the inputs. +(define-condition-type &test-system-not-found &python-build-error + test-system-not-found?) + +;; Raised when multiple wheels are created by 'build. +(define-condition-type &cannot-extract-multiple-wheels &python-build-error + cannot-extract-multiple-wheels?) + +;; Raised, when no wheel has been built by the build system. +(define-condition-type &no-wheels-built &python-build-error + no-wheels-built?) + +(define* (sanity-check #:key tests? inputs outputs #:allow-other-keys) + "Ensure packages depending on this package via setuptools work properly, +their advertised endpoints work and their top level modules are importable +without errors." + (let ((sanity-check.py (assoc-ref inputs "sanity-check.py"))) + ;; Make sure the working directory is empty (i.e. no Python modules in it) + (with-directory-excursion "/tmp" + (invoke "python" sanity-check.py (site-packages inputs outputs))))) + +(define* (build #:key outputs build-backend configure-flags #:allow-other-keys) + "Build a given Python package." + + (define (pyproject.toml->build-backend file) + "Look up the build backend in a pyproject.toml file." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in 'concat))) + (if (eof-object? line) + #f + (let ((m (string-match "build-backend = [\"'](.+)[\"']" line))) + (if m (match:substring m 1) + (loop (read-line in 'concat))))))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + ;; There is no easy way to get data from Guile into Python via + ;; s-expressions, but we have JSON serialization already, which Python + ;; also supports out-of-the-box. + (config-settings (call-with-output-string (cut write-json configure-flags <>))) + ;; python-setuptools’ default backend supports setup.py *and* + ;; pyproject.toml. Allow overriding this automatic detection via + ;; build-backend. + (auto-build-backend (if (file-exists? "pyproject.toml") + (pyproject.toml->build-backend "pyproject.toml") + #f)) + ;; Use build system detection here and not in importer, because a) we + ;; have alot of legacy packages and b) the importer cannot update arbitrary + ;; fields in case a package switches its build system. + (use-build-backend (or + build-backend + auto-build-backend + "setuptools.build_meta"))) + (format #t "Using '~a' to build wheels, auto-detected '~a', override '~a'.~%" + use-build-backend auto-build-backend build-backend) + (mkdir-p wheel-dir) + ;; Call the PEP 517 build function, which drops a .whl into wheel-dir. + (invoke "python" "-c" "import sys, importlib, json +config_settings = json.loads (sys.argv[3]) +builder = importlib.import_module(sys.argv[1]) +builder.build_wheel(sys.argv[2], config_settings=config_settings)" + use-build-backend wheel-dir config-settings))) + +(define* (check #:key inputs outputs tests? test-backend test-flags #:allow-other-keys) + "Run the test suite of a given Python package." + (if tests? + ;; Unfortunately with PEP 517 there is no common method to specify test + ;; systems. Guess test system based on inputs instead. + (let* ((pytest (which "pytest")) + (nosetests (which "nosetests")) + (nose2 (which "nose2")) + (have-setup-py (file-exists? "setup.py")) + (use-test-backend + (or + test-backend + ;; Prefer pytest + (if pytest 'pytest #f) + (if nosetests 'nose #f) + (if nose2 'nose2 #f) + ;; But fall back to setup.py, which should work for most + ;; packages. XXX: would be nice not to depend on setup.py here? fails + ;; more often than not to find any tests at all. Maybe we can run + ;; `python -m unittest`? + (if have-setup-py 'setup.py #f)))) + (format #t "Using ~a~%" use-test-backend) + (match use-test-backend + ('pytest + (apply invoke (cons pytest (or test-flags '("-vv"))))) + ('nose + (apply invoke (cons nosetests (or test-flags '("-v"))))) + ('nose2 + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) + ('setup.py + (apply invoke (append '("python" "setup.py") (or test-flags '("test" "-v"))))) + ;; The developer should explicitly disable tests in this case. + (else (raise (condition (&test-system-not-found)))))) + (format #t "test suite not run~%"))) + +(define (python-version python) + (let* ((version (last (string-split python #\-))) + (components (string-split version #\.)) + (major+minor (take components 2))) + (string-join major+minor "."))) + +(define (python-output outputs) + "Return the path of the python output, if there is one, or fall-back to out." + (or (assoc-ref outputs "python") + (assoc-ref outputs "out"))) + +(define (site-packages inputs outputs) + "Return the path of the current output's Python site-package." + (let* ((out (python-output outputs)) + (python (assoc-ref inputs "python"))) + (string-append out "/lib/python" (python-version python) "/site-packages"))) + +(define (add-installed-pythonpath inputs outputs) + "Prepend the site-package of OUTPUT to GUIX_PYTHONPATH. This is useful when +running checks after installing the package." + (setenv "GUIX_PYTHONPATH" (string-append (site-packages inputs outputs) ":" + (getenv "GUIX_PYTHONPATH")))) + +(define* (add-install-to-pythonpath #:key inputs outputs #:allow-other-keys) + "A phase that just wraps the 'add-installed-pythonpath' procedure." + (add-installed-pythonpath inputs outputs)) + +(define* (add-install-to-path #:key outputs #:allow-other-keys) + "Adding Python scripts to PATH is also often useful in tests." + (setenv "PATH" (string-append (assoc-ref outputs "out") + "/bin:" + (getenv "PATH")))) + +(define* (install #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Install a wheel file according to PEP 427" + ;; See https://www.python.org/dev/peps/pep-0427/#installing-a-wheel-distribution-1-0-py32-none-any-whl + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (out (assoc-ref outputs "out"))) + (define (extract file) + "Extract wheel (ZIP file) into site-packages directory" + ;; Use Python’s zipfile to avoid extra dependency + (invoke "python" "-m" "zipfile" "-e" file site-dir)) + + (define python-hashbang + (string-append "#!" python "/bin/python")) + + (define* (merge-directories source destination #:optional (post-move #f)) + "Move all files in SOURCE into DESTINATION, merging the two directories." + (format #t "Merging directory ~a into ~a~%" source destination) + (for-each + (lambda (file) + (format #t "~a/~a -> ~a/~a~%" source file destination file) + (mkdir-p destination) + (rename-file + (string-append source "/" file) + (string-append destination "/" file)) + (when post-move + (post-move file))) + (scandir source (negate (cut member <> '("." ".."))))) + (rmdir source)) + + (define (expand-data-directory directory) + "Move files from all .data subdirectories to their respective +destinations." + ;; Python’s distutils.command.install defines this mapping from source to + ;; destination mapping. + (let ((source (string-append directory "/scripts")) + (destination (string-append out "/bin"))) + (when (file-exists? source) + (merge-directories + source + destination + (lambda (f) + (let ((dest-path (string-append destination "/" f))) + (chmod dest-path #o755) + (substitute* dest-path (("#!python") python-hashbang))))))) + ;; data can create arbitrary directory structures. Most commonly + ;; it is used for share/. + (let ((source (string-append directory "/data")) + (destination out)) + (when (file-exists? source) + (merge-directories source destination))) + (let* ((distribution (car (string-split (basename directory) #\-))) + (source (string-append directory "/headers")) + (destination (string-append out "/include/python" (python-version python) "/" distribution))) + (when (file-exists? source) + (merge-directories source destination)))) + + (define (list-directories base predicate) + ;; Cannot use find-files here, because it’s recursive. + (scandir + base + (lambda (name) + (let ((stat (lstat (string-append base "/" name)))) + (and + (not (member name '("." ".."))) + (eq? (stat:type stat) 'directory) + (predicate name stat)))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + (wheels (map (cut string-append wheel-dir "/" <>) + (scandir wheel-dir (cut string-suffix? ".whl" <>))))) + (cond + ((> (length wheels) 1) ; This code does not support multiple wheels + ; yet, because their outputs would have to be + ; merged properly. + (raise (condition (&cannot-extract-multiple-wheels)))) + ((= (length wheels) 0) + (raise (condition (&no-wheels-built))))) + (for-each extract wheels)) + (let ((datadirs (map + (cut string-append site-dir "/" <>) + (list-directories site-dir (file-name-predicate "\\.data$"))))) + (for-each (lambda (directory) + (expand-data-directory directory) + (rmdir directory)) + datadirs)))) + +(define* (compile-bytecode #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Compile installed byte-code in site-packages." + (let ((site-dir (site-packages inputs outputs))) + (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" + site-dir))) + +(define* (create-entrypoints #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Implement Entry Points Specification +(https://packaging.python.org/specifications/entry-points/) by PyPa, +which creates runnable scripts in bin/ from entry point specification +file entry_points.txt. This is necessary, because wheels do not contain +these binaries and installers are expected to create them." + + (define (entry-points.txt->entry-points file) + "Specialized parser for Python configfile-like files, in particular +entry_points.txt. Returns a list of console_script and gui_scripts +entry points." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in)) + (inside #f) + (result '())) + (if (eof-object? line) + result + (let* ((group-match (string-match "^\\[(.+)\\]$" line)) + (group-name (if group-match (match:substring group-match 1) #f)) + (next-inside + (if (not group-name) + inside + (or + (string=? group-name "console_scripts") + (string=? group-name "gui_scripts")))) + (item-match (string-match "^([^ =]+)\\s*=\\s*([^:]+):(.+)$" line))) + (if (and inside item-match) + (loop (read-line in) next-inside (cons (list + (match:substring item-match 1) + (match:substring item-match 2) + (match:substring item-match 3)) + result)) + (loop (read-line in) next-inside result)))))))) + + (define (create-script path name module function) + "Create a Python script from an entry point’s NAME, MODULE and + FUNCTION and return write it to PATH/NAME." + (let ((interpreter (which "python")) + (file-path (string-append path "/" name))) + (format #t "Creating entry point for '~a.~a' at '~a'.~%" module function + file-path) + (call-with-output-file file-path + (lambda (port) + ;; Technically the script could also include search-paths, + ;; but having a generic 'wrap phases also handles manually + ;; written entry point scripts. + (format port "#!~a +# Auto-generated entry point script. +import sys +import ~a as mod +sys.exit (mod.~a ())~%" interpreter module function))) + (chmod file-path #o755))) + + (let* ((site-dir (site-packages inputs outputs)) + (out (assoc-ref outputs "out")) + (bin-dir (string-append out "/bin")) + (entry-point-files (find-files site-dir "^entry_points.txt$"))) + (mkdir-p bin-dir) + (for-each + (lambda (f) + (for-each + (lambda (ep) (apply create-script (cons bin-dir ep))) + (entry-points.txt->entry-points f))) + entry-point-files))) + +(define* (wrap #:key inputs outputs search-paths #:allow-other-keys) + (define (list-of-files dir) + (find-files dir (lambda (file stat) + (and (eq? 'regular (stat:type stat)) + (not (wrapped-program? file)))))) + + (define bindirs + (append-map (match-lambda + ((_ . dir) + (list (string-append dir "/bin") + (string-append dir "/sbin")))) + outputs)) + + ;; Do not require "bash" to be present in the package inputs + ;; even when there is nothing to wrap. + ;; Also, calculate (sh) only once to prevent some I/O. + (define %sh (delay (search-input-file inputs "bin/bash"))) + (define (sh) (force %sh)) + + (let* ((var `("GUIX_PYTHONPATH" prefix + ,(search-path-as-string->list + (or (getenv "GUIX_PYTHONPATH") ""))))) + (for-each (lambda (dir) + (let ((files (list-of-files dir))) + (for-each (cut wrap-program <> #:sh (sh) var) + files))) + bindirs))) + +(define* (set-SOURCE-DATE-EPOCH #:rest _) + "Set the 'SOURCE_DATE_EPOCH' environment variable. This is used by tools +that incorporate timestamps as a way to tell them to use a fixed timestamp. +See https://reproducible-builds.org/specs/source-date-epoch/." + ;; Use a post-1980 timestamp because the Zip format used in wheels do + ;; not support timestamps before 1980. + (setenv "SOURCE_DATE_EPOCH" "315619200")) + +(define* (enable-bytecode-determinism #:rest _) + "Improve determinism of pyc files." + ;; Use deterministic hashes for strings, bytes, and datetime objects. + (setenv "PYTHONHASHSEED" "0") + ;; Prevent Python from creating .pyc files when loading modules (such as + ;; when running a test suite). + (setenv "PYTHONDONTWRITEBYTECODE" "1")) + +(define* (ensure-no-cythonized-files #:rest _) + "Check the source code for @code{.c} files which may have been pre-generated +by Cython." + (for-each + (lambda (file) + (let ((generated-file + (string-append (string-drop-right file 3) "c"))) + (when (file-exists? generated-file) + (format #t "Possible Cythonized file found: ~a~%" generated-file)))) + (find-files "." "\\.pyx$"))) + +(define %standard-phases + ;; The build phase only builds C extensions and copies the Python sources, + ;; while the install phase copies then byte-compiles the sources to the + ;; prefix directory. The check phase is moved after the installation phase + ;; to ease testing the built package. + (modify-phases gnu:%standard-phases + (add-after 'unpack 'enable-bytecode-determinism + enable-bytecode-determinism) + (add-after 'enable-bytecode-determinism 'ensure-no-cythonized-files + ensure-no-cythonized-files) + (replace 'set-SOURCE-DATE-EPOCH set-SOURCE-DATE-EPOCH) + (delete 'bootstrap) + (delete 'configure) ;not needed + (replace 'build build) + (delete 'check) ;moved after the install phase + (replace 'install install) + (add-after 'install 'add-install-to-pythonpath add-install-to-pythonpath) + (add-after 'add-install-to-pythonpath 'add-install-to-path + add-install-to-path) + (add-after 'add-install-to-path 'wrap wrap) + ;; must be before tests, so they can use installed packages’ entry points. + (add-before 'wrap 'create-entrypoints create-entrypoints) + (add-after 'wrap 'check check) + (add-after 'check 'sanity-check sanity-check) + (add-before 'check 'compile-bytecode compile-bytecode))) + +(define* (pyproject-build #:key inputs (phases %standard-phases) + #:allow-other-keys #:rest args) + "Build the given Python package, applying all of PHASES in order." + (apply gnu:gnu-build #:inputs inputs #:phases phases args)) + +;;; pyproject-build-system.scm ends here -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system Marius Bakke @ 2022-10-27 14:23 ` Maxim Cournoyer 2022-10-27 15:17 ` Lars-Dominik Braun 2022-10-27 17:23 ` Marius Bakke 0 siblings, 2 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 14:23 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587, Lars-Dominik Braun Hello, Marius Bakke <marius@gnu.org> writes: > From: Lars-Dominik Braun <lars@6xq.net> > > This is an experimental build system based on python-build-system > that implements PEP 517-compliant builds. > > * guix/build-system/pyproject.scm, > guix/build/pyproject-build-system.scm, > gnu/packages/aux-files/python/sanity-check-next.py, I know this file is temporary, but I'd keep the copyright/license notices header anyway, less it may be forgotten in the future. > gnu/packages/python-commencement.scm: New files. > * Makefile.am (MODULES): Register the new build systems. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add python-commencement.scm. Seems this could be merged into gnu/packages/python-build.scm instead of creating a new module. [...] > diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm > new file mode 100644 > index 0000000000..141d4d849f > --- /dev/null > +++ b/guix/build/pyproject-build-system.scm > @@ -0,0 +1,460 @@ > +;;; GNU Guix --- Functional package management for GNU > +;;; Copyright © 2013, 2015, 2016, 2018, 2019 Ludovic Courtès <ludo@gnu.org> > +;;; Copyright © 2013 Andreas Enge <andreas@enge.fr> > +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> > +;;; Copyright © 2015, 2018 Mark H Weaver <mhw@netris.org> > +;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com> > +;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net> > +;;; Copyright © 2018 Arun Isaac <arunisaac@systemreboot.net> > +;;; Copyright © 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> > +;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> > +;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il> > +;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be> > +;;; > +;;; The above copyright is taken from python-build-system. Contributions > +;;; made only to this module are listed below: > +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> > +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> > +;;; > +;;; This file is part of GNU Guix. > +;;; > +;;; GNU Guix is free software; you can redistribute it and/or modify it > +;;; under the terms of the GNU General Public License as published by > +;;; the Free Software Foundation; either version 3 of the License, or (at > +;;; your option) any later version. > +;;; > +;;; GNU Guix is distributed in the hope that it will be useful, but > +;;; WITHOUT ANY WARRANTY; without even the implied warranty of > +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +;;; GNU General Public License for more details. > +;;; > +;;; You should have received a copy of the GNU General Public License > +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. > + > +(define-module (guix build pyproject-build-system) > + #:use-module ((guix build gnu-build-system) #:prefix gnu:) > + #:use-module (guix build utils) > + #:use-module (guix build json) > + #:use-module (ice-9 match) > + #:use-module (ice-9 ftw) > + #:use-module (ice-9 format) > + #:use-module (ice-9 rdelim) > + #:use-module (ice-9 regex) > + #:use-module (srfi srfi-1) > + #:use-module (srfi srfi-26) > + #:use-module (srfi srfi-34) > + #:use-module (srfi srfi-35) > + #:export (%standard-phases > + add-installed-pythonpath > + site-packages > + python-version > + pyproject-build)) > + > +;; Commentary: > +;; > +;; PEP 517-compatible build system for Python packages. > +;; > +;; PEP 517 mandates the use of a TOML file called pyproject.toml at the > +;; project root, describing build and runtime dependencies, as well as the > +;; build system, which can be different from setuptools. This module uses > +;; that file to extract the build system used and call its wheel-building > +;; entry point build_wheel (see 'build). setuptools’ wheel builder is > +;; used as a fallback if either no pyproject.toml exists or it does not > +;; declare a build-system. It supports config_settings through the > +;; standard #:configure-flags argument. > +;; > +;; This wheel, which is just a ZIP file with a file structure defined > +;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked > +;; and its contents are moved to the appropriate locations in 'install. > +;; > +;; Then entry points, as defined by the PyPa Entry Point Specification > +;; (https://packaging.python.org/specifications/entry-points/) are read > +;; from a file called entry_points.txt in the package’s site-packages > +;; subdirectory and scripts are written to bin/. These are not part of a > +;; wheel and expected to be created by the installing utility. > +;; > +;; Caveats: > +;; - There is no support for in-tree build backends. > + > +;; Base error type. > +(define-condition-type &python-build-error &error > + python-build-error?) > + > +;; Raised when 'check cannot find a valid test system in the inputs. > +(define-condition-type &test-system-not-found &python-build-error > + test-system-not-found?) > + > +;; Raised when multiple wheels are created by 'build. > +(define-condition-type &cannot-extract-multiple-wheels &python-build-error > + cannot-extract-multiple-wheels?) > + > +;; Raised, when no wheel has been built by the build system. > +(define-condition-type &no-wheels-built &python-build-error > + no-wheels-built?) > + > +(define* (sanity-check #:key tests? inputs outputs #:allow-other-keys) > + "Ensure packages depending on this package via setuptools work properly, > +their advertised endpoints work and their top level modules are importable > +without errors." > + (let ((sanity-check.py (assoc-ref inputs "sanity-check.py"))) > + ;; Make sure the working directory is empty (i.e. no Python modules in it) > + (with-directory-excursion "/tmp" > + (invoke "python" sanity-check.py (site-packages inputs outputs))))) > + > +(define* (build #:key outputs build-backend configure-flags #:allow-other-keys) > + "Build a given Python package." > + > + (define (pyproject.toml->build-backend file) > + "Look up the build backend in a pyproject.toml file." > + (call-with-input-file file > + (lambda (in) > + (let loop ((line (read-line in 'concat))) > + (if (eof-object? line) > + #f > + (let ((m (string-match "build-backend = [\"'](.+)[\"']" line))) > + (if m (match:substring m 1) > + (loop (read-line in 'concat))))))))) > + > + (let* ((wheel-output (assoc-ref outputs "wheel")) > + (wheel-dir (if wheel-output wheel-output "dist")) > + ;; There is no easy way to get data from Guile into Python via > + ;; s-expressions, but we have JSON serialization already, which Python > + ;; also supports out-of-the-box. > + (config-settings (call-with-output-string (cut write-json configure-flags <>))) > + ;; python-setuptools’ default backend supports setup.py *and* > + ;; pyproject.toml. Allow overriding this automatic detection via > + ;; build-backend. > + (auto-build-backend (if (file-exists? "pyproject.toml") > + (pyproject.toml->build-backend "pyproject.toml") > + #f)) > + ;; Use build system detection here and not in importer, because a) we > + ;; have alot of legacy packages and b) the importer cannot update arbitrary > + ;; fields in case a package switches its build system. > + (use-build-backend (or > + build-backend > + auto-build-backend > + "setuptools.build_meta"))) Odd indentation; I'd re-indent it as (or build-backend auto-build-backend ...) > + (format #t "Using '~a' to build wheels, auto-detected '~a', override '~a'.~%" > + use-build-backend auto-build-backend build-backend) > + (mkdir-p wheel-dir) > + ;; Call the PEP 517 build function, which drops a .whl into wheel-dir. > + (invoke "python" "-c" "import sys, importlib, json > +config_settings = json.loads (sys.argv[3]) > +builder = importlib.import_module(sys.argv[1]) > +builder.build_wheel(sys.argv[2], config_settings=config_settings)" > + use-build-backend wheel-dir config-settings))) > + > +(define* (check #:key inputs outputs tests? test-backend test-flags #:allow-other-keys) > + "Run the test suite of a given Python package." > + (if tests? s/if/when/ > + ;; Unfortunately with PEP 517 there is no common method to specify test > + ;; systems. Guess test system based on inputs instead. Use two spaces between sentences. > + (let* ((pytest (which "pytest")) > + (nosetests (which "nosetests")) > + (nose2 (which "nose2")) > + (have-setup-py (file-exists? "setup.py")) > + (use-test-backend > + (or > + test-backend Odd indentation, same comment as above. > + ;; Prefer pytest > + (if pytest 'pytest #f) > + (if nosetests 'nose #f) > + (if nose2 'nose2 #f) > + ;; But fall back to setup.py, which should work for most > + ;; packages. XXX: would be nice not to depend on setup.py here? fails > + ;; more often than not to find any tests at all. Maybe we can run > + ;; `python -m unittest`? > + (if have-setup-py 'setup.py #f)))) > + (format #t "Using ~a~%" use-test-backend) > + (match use-test-backend > + ('pytest > + (apply invoke (cons pytest (or test-flags '("-vv"))))) > + ('nose > + (apply invoke (cons nosetests (or test-flags '("-v"))))) > + ('nose2 > + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) I think I'd leave the defaults in and document it that way. > + ('setup.py > + (apply invoke (append '("python" "setup.py") (or test-flags '("test" "-v"))))) > + ;; The developer should explicitly disable tests in this case. > + (else (raise (condition (&test-system-not-found)))))) > + (format #t "test suite not run~%"))) > + > +(define (python-version python) > + (let* ((version (last (string-split python #\-))) > + (components (string-split version #\.)) > + (major+minor (take components 2))) > + (string-join major+minor "."))) > + > +(define (python-output outputs) > + "Return the path of the python output, if there is one, or fall-back to out." > + (or (assoc-ref outputs "python") > + (assoc-ref outputs "out"))) > + > +(define (site-packages inputs outputs) > + "Return the path of the current output's Python site-package." > + (let* ((out (python-output outputs)) > + (python (assoc-ref inputs "python"))) > + (string-append out "/lib/python" (python-version python) "/site-packages"))) > + > +(define (add-installed-pythonpath inputs outputs) > + "Prepend the site-package of OUTPUT to GUIX_PYTHONPATH. This is useful when > +running checks after installing the package." > + (setenv "GUIX_PYTHONPATH" (string-append (site-packages inputs outputs) ":" > + (getenv "GUIX_PYTHONPATH")))) > + > +(define* (add-install-to-pythonpath #:key inputs outputs #:allow-other-keys) > + "A phase that just wraps the 'add-installed-pythonpath' procedure." > + (add-installed-pythonpath inputs outputs)) > + > +(define* (add-install-to-path #:key outputs #:allow-other-keys) > + "Adding Python scripts to PATH is also often useful in tests." > + (setenv "PATH" (string-append (assoc-ref outputs "out") > + "/bin:" > + (getenv "PATH")))) > + > +(define* (install #:key inputs outputs (configure-flags '()) #:allow-other-keys) Unused configure-flags argument. Also do not provide a default, as that's the concern of (guix build-system pyproject). > + "Install a wheel file according to PEP 427" > + ;; See https://www.python.org/dev/peps/pep-0427/#installing-a-wheel-distribution-1-0-py32-none-any-whl > + (let* ((site-dir (site-packages inputs outputs)) > + (python (assoc-ref inputs "python")) > + (out (assoc-ref outputs "out"))) Plain "let" seems enough here. > + (define (extract file) > + "Extract wheel (ZIP file) into site-packages directory" > + ;; Use Python’s zipfile to avoid extra dependency > + (invoke "python" "-m" "zipfile" "-e" file site-dir)) > + > + (define python-hashbang > + (string-append "#!" python "/bin/python")) > + > + (define* (merge-directories source destination #:optional (post-move #f)) > + "Move all files in SOURCE into DESTINATION, merging the two directories." > + (format #t "Merging directory ~a into ~a~%" source destination) > + (for-each > + (lambda (file) > + (format #t "~a/~a -> ~a/~a~%" source file destination file) > + (mkdir-p destination) > + (rename-file > + (string-append source "/" file) > + (string-append destination "/" file)) > + (when post-move > + (post-move file))) > + (scandir source (negate (cut member <> '("." ".."))))) > + (rmdir source)) > + > + (define (expand-data-directory directory) > + "Move files from all .data subdirectories to their respective > +destinations." > + ;; Python’s distutils.command.install defines this mapping from source to > + ;; destination mapping. > + (let ((source (string-append directory "/scripts")) > + (destination (string-append out "/bin"))) > + (when (file-exists? source) > + (merge-directories > + source > + destination > + (lambda (f) > + (let ((dest-path (string-append destination "/" f))) > + (chmod dest-path #o755) > + (substitute* dest-path (("#!python") python-hashbang))))))) Not sure how this #!python shebang comes into existence; as that isn't a valid one. Perhaps double check things here? > + ;; data can create arbitrary directory structures. Most commonly > + ;; it is used for share/. Perhaps ;; Data can be contained in arbitrary ... ? > + (let ((source (string-append directory "/data")) > + (destination out)) > + (when (file-exists? source) > + (merge-directories source destination))) > + (let* ((distribution (car (string-split (basename directory) #\-))) > + (source (string-append directory "/headers")) > + (destination (string-append out "/include/python" (python-version python) "/" distribution))) This line should be broken to fit under 80 chars. > + (when (file-exists? source) > + (merge-directories source destination)))) > + > + (define (list-directories base predicate) > + ;; Cannot use find-files here, because it’s recursive. > + (scandir > + base > + (lambda (name) > + (let ((stat (lstat (string-append base "/" name)))) > + (and > + (not (member name '("." ".."))) > + (eq? (stat:type stat) 'directory) > + (predicate name stat)))))) nitpick: and indentation style (same comment as for 'or' above). > + (let* ((wheel-output (assoc-ref outputs "wheel")) > + (wheel-dir (if wheel-output wheel-output "dist")) > + (wheels (map (cut string-append wheel-dir "/" <>) > + (scandir wheel-dir (cut string-suffix? ".whl" <>))))) > + (cond > + ((> (length wheels) 1) ; This code does not support multiple wheels > + ; yet, because their outputs would have to be > + ; merged properly. > + (raise (condition (&cannot-extract-multiple-wheels)))) > + ((= (length wheels) 0) > + (raise (condition (&no-wheels-built))))) Indentation is off here. > + (for-each extract wheels)) > + (let ((datadirs (map > + (cut string-append site-dir "/" <>) > + (list-directories site-dir (file-name-predicate "\\.data$"))))) Too long line and map indentation style. > + (for-each (lambda (directory) > + (expand-data-directory directory) > + (rmdir directory)) > + datadirs)))) > + > +(define* (compile-bytecode #:key inputs outputs (configure-flags '()) #:allow-other-keys) Unused configure-flags argument. > + "Compile installed byte-code in site-packages." > + (let ((site-dir (site-packages inputs outputs))) > + (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" > + site-dir))) > + > +(define* (create-entrypoints #:key inputs outputs (configure-flags '()) #:allow-other-keys) Unused configure-flags argument. > + "Implement Entry Points Specification > +(https://packaging.python.org/specifications/entry-points/) by PyPa, > +which creates runnable scripts in bin/ from entry point specification > +file entry_points.txt. This is necessary, because wheels do not contain > +these binaries and installers are expected to create them." Two spaces between sentences. > + > + (define (entry-points.txt->entry-points file) > + "Specialized parser for Python configfile-like files, in particular > +entry_points.txt. Returns a list of console_script and gui_scripts > +entry points." > + (call-with-input-file file > + (lambda (in) > + (let loop ((line (read-line in)) > + (inside #f) > + (result '())) > + (if (eof-object? line) > + result > + (let* ((group-match (string-match "^\\[(.+)\\]$" line)) > + (group-name (if group-match (match:substring group-match 1) #f)) > + (next-inside > + (if (not group-name) > + inside > + (or > + (string=? group-name "console_scripts") > + (string=? group-name "gui_scripts")))) > + (item-match (string-match "^([^ =]+)\\s*=\\s*([^:]+):(.+)$" line))) > + (if (and inside item-match) > + (loop (read-line in) next-inside (cons (list > + (match:substring item-match 1) > + (match:substring item-match 2) > + (match:substring item-match 3)) > + result)) > + (loop (read-line in) next-inside result)))))))) Our coding standard is 80 chars width, not 100 :-). > + (define (create-script path name module function) > + "Create a Python script from an entry point’s NAME, MODULE and > + FUNCTION and return write it to PATH/NAME." > + (let ((interpreter (which "python")) > + (file-path (string-append path "/" name))) > + (format #t "Creating entry point for '~a.~a' at '~a'.~%" module function > + file-path) > + (call-with-output-file file-path > + (lambda (port) > + ;; Technically the script could also include search-paths, > + ;; but having a generic 'wrap phases also handles manually > + ;; written entry point scripts. > + (format port "#!~a > +# Auto-generated entry point script. > +import sys > +import ~a as mod > +sys.exit (mod.~a ())~%" interpreter module function))) > + (chmod file-path #o755))) > + > + (let* ((site-dir (site-packages inputs outputs)) > + (out (assoc-ref outputs "out")) > + (bin-dir (string-append out "/bin")) > + (entry-point-files (find-files site-dir "^entry_points.txt$"))) > + (mkdir-p bin-dir) > + (for-each > + (lambda (f) > + (for-each > + (lambda (ep) (apply create-script (cons bin-dir ep))) > + (entry-points.txt->entry-points f))) > + entry-point-files))) Phew, that's a lot of parsing and manual handling of things. I'd rather we leave this to uptream tooling instead of having to maintain that ourselves, but based on previous discussion it remember that bootstrapping pypi was thought too difficult. > +(define* (wrap #:key inputs outputs search-paths #:allow-other-keys) > + (define (list-of-files dir) > + (find-files dir (lambda (file stat) > + (and (eq? 'regular (stat:type stat)) > + (not (wrapped-program? file)))))) > + > + (define bindirs > + (append-map (match-lambda > + ((_ . dir) > + (list (string-append dir "/bin") > + (string-append dir "/sbin")))) > + outputs)) > + > + ;; Do not require "bash" to be present in the package inputs > + ;; even when there is nothing to wrap. > + ;; Also, calculate (sh) only once to prevent some I/O. > + (define %sh (delay (search-input-file inputs "bin/bash"))) > + (define (sh) (force %sh)) > + > + (let* ((var `("GUIX_PYTHONPATH" prefix > + ,(search-path-as-string->list > + (or (getenv "GUIX_PYTHONPATH") ""))))) > + (for-each (lambda (dir) > + (let ((files (list-of-files dir))) > + (for-each (cut wrap-program <> #:sh (sh) var) > + files))) > + bindirs))) > + > +(define* (set-SOURCE-DATE-EPOCH #:rest _) Perhaps name this set-SOURCE-DATE-EPOCH* to hint that it's not the same thing duplicated. The rest of the patch LGTM. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-27 14:23 ` Maxim Cournoyer @ 2022-10-27 15:17 ` Lars-Dominik Braun 2022-10-27 17:12 ` Maxim Cournoyer 2022-10-27 17:23 ` Marius Bakke 1 sibling, 1 reply; 65+ messages in thread From: Lars-Dominik Braun @ 2022-10-27 15:17 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 58587, Marius Bakke Hi Maxim, as the original author, I might be able to answer a few of these questions. > Not sure how this #!python shebang comes into existence; as that isn't a > valid one. Perhaps double check things here? As mentioned by the docstring of INSTALL this conforms to PEP 427. > Phew, that's a lot of parsing and manual handling of things. I'd rather > we leave this to uptream tooling instead of having to maintain that > ourselves, but based on previous discussion it remember that > bootstrapping pypi was thought too difficult. Last time I checked installer (https://github.com/pypa/installer) wasn’t in a good enough shape to use it here. We’d still have to write a few lines of Python code, since it does not come with an executable. Bootstrapping pip is pretty much impossible due to the amount of vendored components. Lars ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-27 15:17 ` Lars-Dominik Braun @ 2022-10-27 17:12 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 17:12 UTC (permalink / raw) To: Lars-Dominik Braun; +Cc: 58587, Marius Bakke Hi Lars-Dominik, Lars-Dominik Braun <lars@6xq.net> writes: > Hi Maxim, > > as the original author, I might be able to answer a few of these > questions. > >> Not sure how this #!python shebang comes into existence; as that isn't a >> valid one. Perhaps double check things here? > As mentioned by the docstring of INSTALL this conforms to PEP 427. OK, thanks for clearing that up for me. >> Phew, that's a lot of parsing and manual handling of things. I'd rather >> we leave this to uptream tooling instead of having to maintain that >> ourselves, but based on previous discussion it remember that >> bootstrapping pypi was thought too difficult. > Last time I checked installer (https://github.com/pypa/installer) > wasn’t in a good enough shape to use it here. We’d still have > to write a few lines of Python code, since it does not come with an > executable. Bootstrapping pip is pretty much impossible due to the amount > of vendored components. This sounds like the kind of tool we'd want to use longer term, to avoid having to go back to the drawing board every time the Python community changes their mind on packaging matters (which is relatively often as of late). That said, I think the current approach can get us started, since it's already done and seems to work fine so far. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-27 14:23 ` Maxim Cournoyer 2022-10-27 15:17 ` Lars-Dominik Braun @ 2022-10-27 17:23 ` Marius Bakke 2022-10-27 19:32 ` bug#58587: " Marius Bakke 1 sibling, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-27 17:23 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 58587, Lars-Dominik Braun [-- Attachment #1: Type: text/plain, Size: 2690 bytes --] Maxim Cournoyer <maxim.cournoyer@gmail.com> skriver: > Hello, > > Marius Bakke <marius@gnu.org> writes: > >> From: Lars-Dominik Braun <lars@6xq.net> >> >> This is an experimental build system based on python-build-system >> that implements PEP 517-compliant builds. >> >> * guix/build-system/pyproject.scm, >> guix/build/pyproject-build-system.scm, >> gnu/packages/aux-files/python/sanity-check-next.py, > > I know this file is temporary, but I'd keep the copyright/license > notices header anyway, less it may be forgotten in the future. Makes sense; done. >> gnu/packages/python-commencement.scm: New files. >> * Makefile.am (MODULES): Register the new build systems. >> * gnu/local.mk (GNU_SYSTEM_MODULES): Add python-commencement.scm. > > Seems this could be merged into gnu/packages/python-build.scm instead of > creating a new module. I think you reviewed an old revision; the latest revision adds python-toolchain straight into python.scm. We may have to bring back python-commencement.scm for bootstrapping setuptools and friends at a later core-updates cycle though. >> + ;; Prefer pytest >> + (if pytest 'pytest #f) >> + (if nosetests 'nose #f) >> + (if nose2 'nose2 #f) >> + ;; But fall back to setup.py, which should work for most >> + ;; packages. XXX: would be nice not to depend on setup.py here? fails >> + ;; more often than not to find any tests at all. Maybe we can run >> + ;; `python -m unittest`? >> + (if have-setup-py 'setup.py #f)))) >> + (format #t "Using ~a~%" use-test-backend) >> + (match use-test-backend >> + ('pytest >> + (apply invoke (cons pytest (or test-flags '("-vv"))))) >> + ('nose >> + (apply invoke (cons nosetests (or test-flags '("-v"))))) >> + ('nose2 >> + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) > > I think I'd leave the defaults in and document it that way. By leave, you mean: (apply invoke "pytest" "-vv" test-flags) ? >> +(define* (set-SOURCE-DATE-EPOCH #:rest _) > > Perhaps name this set-SOURCE-DATE-EPOCH* to hint that it's not the same > thing duplicated. Done; although in the latest revision nothing is duplicated as it inherits a bunch from python-build-system. Plans for the immediate core-updates cycle: * merge sanity-check-next.py into sanity-check.py * set SOURCE_DATE_EPOCH to ~1980 also in python-build-system (and inherit in pyproject-build-system) * move the compile-bytecode phase into python-build-system (currently python-build-system does this during install, hence duplication) [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* bug#58587: [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-27 17:23 ` Marius Bakke @ 2022-10-27 19:32 ` Marius Bakke 2022-10-28 1:12 ` [bug#58587] " Maxim Cournoyer 0 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-27 19:32 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 58587-done, Lars-Dominik Braun [-- Attachment #1: Type: text/plain, Size: 1340 bytes --] Marius Bakke <marius@gnu.org> skriver: > Maxim Cournoyer <maxim.cournoyer@gmail.com> skriver: > >> Hello, >> >> Marius Bakke <marius@gnu.org> writes: >> >>> From: Lars-Dominik Braun <lars@6xq.net> >>> >>> + ;; Prefer pytest >>> + (if pytest 'pytest #f) >>> + (if nosetests 'nose #f) >>> + (if nose2 'nose2 #f) >>> + ;; But fall back to setup.py, which should work for most >>> + ;; packages. XXX: would be nice not to depend on setup.py here? fails >>> + ;; more often than not to find any tests at all. Maybe we can run >>> + ;; `python -m unittest`? >>> + (if have-setup-py 'setup.py #f)))) >>> + (format #t "Using ~a~%" use-test-backend) >>> + (match use-test-backend >>> + ('pytest >>> + (apply invoke (cons pytest (or test-flags '("-vv"))))) >>> + ('nose >>> + (apply invoke (cons nosetests (or test-flags '("-v"))))) >>> + ('nose2 >>> + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) >> >> I think I'd leave the defaults in and document it that way. > > By leave, you mean: > > (apply invoke "pytest" "-vv" test-flags) > > ? I made this change in e944734ef9afa1ac9b46579934482b7d909ed24e. The patches are now merged! [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system. 2022-10-27 19:32 ` bug#58587: " Marius Bakke @ 2022-10-28 1:12 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-28 1:12 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587-done, Lars-Dominik Braun Hi Marius, Marius Bakke <marius@gnu.org> writes: > Marius Bakke <marius@gnu.org> skriver: > >> Maxim Cournoyer <maxim.cournoyer@gmail.com> skriver: >> >>> Hello, >>> >>> Marius Bakke <marius@gnu.org> writes: >>> >>>> From: Lars-Dominik Braun <lars@6xq.net> >>>> >>>> + ;; Prefer pytest >>>> + (if pytest 'pytest #f) >>>> + (if nosetests 'nose #f) >>>> + (if nose2 'nose2 #f) >>>> + ;; But fall back to setup.py, which should work for most >>>> + ;; packages. XXX: would be nice not to depend on setup.py here? fails >>>> + ;; more often than not to find any tests at all. Maybe we can run >>>> + ;; `python -m unittest`? >>>> + (if have-setup-py 'setup.py #f)))) >>>> + (format #t "Using ~a~%" use-test-backend) >>>> + (match use-test-backend >>>> + ('pytest >>>> + (apply invoke (cons pytest (or test-flags '("-vv"))))) >>>> + ('nose >>>> + (apply invoke (cons nosetests (or test-flags '("-v"))))) >>>> + ('nose2 >>>> + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) >>> >>> I think I'd leave the defaults in and document it that way. >> >> By leave, you mean: >> >> (apply invoke "pytest" "-vv" test-flags) >> >> ? > > I made this change in e944734ef9afa1ac9b46579934482b7d909ed24e. > > The patches are now merged! Very nice! Well done! -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (2 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-27 14:24 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 06/14] gnu: flair: Switch to pyproject-build-system Marius Bakke ` (9 subsequent siblings) 13 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-autopage)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. --- gnu/packages/python-xyz.scm | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 1ded6f0384..42935cbaaa 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -259,6 +259,7 @@ (define-module (gnu packages python-xyz) #:use-module (guix utils) #:use-module (guix build-system gnu) #:use-module (guix build-system cmake) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system trivial) #:use-module (srfi srfi-1) @@ -6959,33 +6960,16 @@ (define-public python-autopage (sha256 (base32 "169ixll1ncm2a2pcc86665ikjv2lrzs10p6c1w4yj55p3gk3xgh1")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; Do a manual PEP 517 style build/install procedure until the - ;; python-build-system overhaul is merged. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (let ((circa-1980 (* 10 366 24 60 60))) - (setenv "SOURCE_DATE_EPOCH" (number->string circa-1980)) - (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))) (add-before 'check 'disable-e2e-tests (lambda _ ;; These tests rely on KeyboardInterrupts which do not ;; work in the build container. - (delete-file "autopage/tests/test_end_to_end.py"))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + (delete-file "autopage/tests/test_end_to_end.py")))))) (native-inputs (list python-pypa-build python-setuptools -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system Marius Bakke @ 2022-10-27 14:24 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 14:24 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587 Hi, Marius Bakke <marius@gnu.org> writes: > * gnu/packages/python-xyz.scm (python-autopage)[build-system]: Switch to > PYPROJECT-BUILD-SYSTEM. > [arguments]: Remove redundant phases. Nice! LGTM. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 06/14] gnu: flair: Switch to pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (3 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 07/14] gnu: python-pydyf: Use pyproject-build-system Marius Bakke ` (8 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/bioinformatics.scm (flair)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove #:phases. [native-inputs]: Remove. --- gnu/packages/bioinformatics.scm | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index f2de09aa32..3588f4e3fd 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -56,6 +56,7 @@ (define-module (gnu packages bioinformatics) #:use-module (guix build-system meson) #:use-module (guix build-system ocaml) #:use-module (guix build-system perl) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system qt) #:use-module (guix build-system r) @@ -16928,33 +16929,15 @@ (define-public flair (sha256 (base32 "106swb2q7l20ki58fca1hg95q5f79bgp9gjb0clr2243ycrzyxf8")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - (list - #:tests? #false ;there are none - #:phases - #~(modify-phases %standard-phases - ;; TODO: implement as a feature of python-build-system (PEP-621, - ;; PEP-631, PEP-660) - (replace 'build - (lambda _ - (setenv "SETUPTOOLS_SCM_PRETEND_VERSION" #$version) - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (apply invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output - (find-files "dist" "\\.whl$"))))))) + (list #:tests? #false)) ;there are none (propagated-inputs (list python-mappy python-ncls python-pybedtools python-pysam python-tqdm)) - (native-inputs - (list python-pypa-build python-setuptools)) (home-page "https://flair.readthedocs.io/en/latest/") (synopsis "Full-length alternative isoform analysis of RNA") (description "This package implements FLAIR (Full-Length Alternative -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 07/14] gnu: python-pydyf: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (4 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 06/14] gnu: flair: Switch to pyproject-build-system Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 08/14] gnu: weasyprint: " Marius Bakke ` (7 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/pdf.scm (python-pydyf)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove #:phases. Add #:test-flags. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/pdf.scm | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/gnu/packages/pdf.scm b/gnu/packages/pdf.scm index b49e5264f9..ec75bca6e3 100644 --- a/gnu/packages/pdf.scm +++ b/gnu/packages/pdf.scm @@ -49,6 +49,7 @@ (define-module (gnu packages pdf) #:use-module (guix build-system gnu) #:use-module (guix build-system cmake) #:use-module (guix build-system meson) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system qt) #:use-module (guix build-system trivial) @@ -735,28 +736,14 @@ (define-public python-pydyf (uri (pypi-uri "pydyf" version)) (sha256 (base32 "18q43g5d9455msipcgd5fvnh8m4a2rz189slzfg80yycjw66rshs")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list - #:phases - #~(modify-phases %standard-phases - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "-c" "/dev/null"))))))) + #:test-flags #~'("-vv" "-c" "/dev/null"))) (propagated-inputs (list python-pillow)) (native-inputs (list ghostscript python-flit-core - python-pypa-build python-pytest)) (home-page "https://github.com/CourtBouillon/pydyf") (synopsis "Low-level PDF generator") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 08/14] gnu: weasyprint: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (5 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 07/14] gnu: python-pydyf: Use pyproject-build-system Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 09/14] gnu: python-glyphslib: " Marius Bakke ` (6 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/pdf.scm (weasyprint)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. Add #:test-flags. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/pdf.scm | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/gnu/packages/pdf.scm b/gnu/packages/pdf.scm index ec75bca6e3..8c871e4341 100644 --- a/gnu/packages/pdf.scm +++ b/gnu/packages/pdf.scm @@ -1497,9 +1497,11 @@ (define-public weasyprint (sha256 (base32 "0cn8gpgyic6pmrnhp0540nbgplpsd5aybi7k89anz6m1sshgjzgs")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list + #:test-flags #~(list "-vv" "-c" "/dev/null" + "-n" (number->string (parallel-job-count))) #:phases #~(modify-phases %standard-phases (add-after 'unpack 'patch-library-paths @@ -1520,21 +1522,7 @@ (define-public weasyprint (("'pangoft2-1.0-0'") (format #f "~s" (search-input-file inputs - "lib/libpangoft2-1.0.so")))))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "-c" "/dev/null" - "-n" (number->string (parallel-job-count)))))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + "lib/libpangoft2-1.0.so"))))))))) (inputs (list fontconfig glib harfbuzz pango)) (propagated-inputs (list gdk-pixbuf @@ -1552,7 +1540,6 @@ (define-public weasyprint (list font-dejavu ;tests depend on it ghostscript python-flit-core - python-pypa-build python-pytest python-pytest-xdist)) (home-page "https://weasyprint.org/") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 09/14] gnu: python-glyphslib: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (6 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 08/14] gnu: weasyprint: " Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 10/14] gnu: python-statmake: " Marius Bakke ` (5 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-glyphslib)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Add #:test-flags. Remove #:phases. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/fontutils.scm | 42 ++++++++++---------------------------- 1 file changed, 11 insertions(+), 31 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index 9b7f1ac879..eefde1d893 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -76,8 +76,9 @@ (define-module (gnu packages fontutils) #:use-module (guix build-system copy) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) - #:use-module (guix build-system python) #:use-module (guix build-system meson) + #:use-module (guix build-system pyproject) + #:use-module (guix build-system python) #:use-module (guix utils) #:use-module (srfi srfi-1)) @@ -591,39 +592,18 @@ (define-public python-glyphslib (sha256 (base32 "0mkkwd09g76hvif603ij5aqicxh47zvhgyyd0pjcjmpdy6dr70yw")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list - #:phases - #~(modify-phases %standard-phases - (replace 'build - (lambda _ - ;; The Zip format does not support pre-1980 time stamps. - (let ((circa-1980 (* 10 366 24 60 60))) - (setenv "SOURCE_DATE_EPOCH" (number->string circa-1980)) - (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (setenv "GUIX_PYTHONPATH" - (string-append (getcwd) ":" (getenv "GUIX_PYTHONPATH"))) - (invoke "pytest" "-vv" - ;; These fail because the test data has not yet been - ;; updated for newer FontTools: - ;; https://github.com/googlefonts/glyphsLib/issues/787 - ;; Re-enable for versions > 6.0.7. - "--ignore=tests/builder/designspace_gen_test.py" - "--ignore=tests/builder/interpolation_test.py" - ))))))) + #:test-flags #~'("-vv" + ;; These fail because the test data has not yet been + ;; updated for newer FontTools: + ;; https://github.com/googlefonts/glyphsLib/issues/787 + ;; Re-enable for versions > 6.0.7. + "--ignore=tests/builder/designspace_gen_test.py" + "--ignore=tests/builder/interpolation_test.py"))) (native-inputs - (list python-pypa-build - python-setuptools-scm - python-wheel + (list python-setuptools-scm ;; For tests. python-pytest -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 10/14] gnu: python-statmake: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (7 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 09/14] gnu: python-glyphslib: " Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 11/14] gnu: python-ufolib2: " Marius Bakke ` (4 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-statmake)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/fontutils.scm | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index eefde1d893..5c95f9a427 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -1561,12 +1561,11 @@ (define-public python-statmake (sha256 (base32 "0qavzspxhwnaayj5mxq6ncjjziggabxj157ls04h2rdrpq167706")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. (add-after 'unpack 'adjust-for-older-attrs ;; Our older attrs package is using the 'attr' rather than 'attrs' ;; namespace. @@ -1583,27 +1582,9 @@ (define-public python-statmake (("@attrs") "@attr") (("\\battrs\\.") - "attr.")))) - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "tests" - ;;"-n" (number->string (parallel-job-count)) - ;; This test requires orjson, which needs the maturin - ;; build system and new Rust dependencies. - ;;"--ignore" "tests/test_preconf.py" - ))))))) + "attr."))))))) (native-inputs (list python-poetry-core - python-pypa-build python-pytest python-ufo2ft)) (propagated-inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 11/14] gnu: python-ufolib2: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (8 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 10/14] gnu: python-statmake: " Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 12/14] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke ` (3 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-ufolib2)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/fontutils.scm | 26 ++------------------------ 1 file changed, 2 insertions(+), 24 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index 5c95f9a427..46fbc2370a 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -1612,31 +1612,9 @@ (define-public python-ufolib2 (uri (pypi-uri "ufoLib2" version)) (sha256 (base32 "0yx4i8q5rfyqhr2fj70a7z1bp1jv7bdlr64ww9z4nv9ycbda4x9j")))) - (build-system python-build-system) - (arguments - (list - #:phases - #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + (build-system pyproject-build-system) (native-inputs - (list python-pypa-build - python-pytest - python-setuptools-scm - python-wheel)) + (list python-pytest python-setuptools-scm)) (propagated-inputs (list python-attrs python-fonttools-full)) (home-page "https://github.com/fonttools/ufoLib2") (synopsis "Unified Font Object (UFO) font processing library") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 12/14] gnu: python-mypy-protobuf: Switch to pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (9 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 11/14] gnu: python-ufolib2: " Marius Bakke @ 2022-10-17 20:11 ` Marius Bakke 2022-10-17 20:12 ` [bug#58587] [PATCH 13/14] gnu: python-tempora: " Marius Bakke ` (2 subsequent siblings) 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:11 UTC (permalink / raw) To: 58587 * gnu/packages/protobuf.scm (python-mypy-protobuf)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/protobuf.scm | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/gnu/packages/protobuf.scm b/gnu/packages/protobuf.scm index a61d9d7492..e90e0ad3fa 100644 --- a/gnu/packages/protobuf.scm +++ b/gnu/packages/protobuf.scm @@ -31,6 +31,7 @@ (define-module (gnu packages protobuf) #:use-module (guix git-download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system emacs) #:use-module (guix build-system ruby) @@ -285,22 +286,11 @@ (define-public python-mypy-protobuf (sha256 (base32 "0z03h9k68qvnlyhpk0ndwp01bdx77vrjr6mybxq4ldilkkbksklk")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) (add-before 'check 'generate-protos-for-tests (lambda _ ;; Generate Python sources. @@ -324,7 +314,6 @@ (define-public python-mypy-protobuf (native-inputs (list python-grpc-stubs python-grpcio-tools - python-pypa-build python-pytest python-typing-extensions-next)) (propagated-inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 13/14] gnu: python-tempora: Switch to pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (10 preceding siblings ...) 2022-10-17 20:11 ` [bug#58587] [PATCH 12/14] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke @ 2022-10-17 20:12 ` Marius Bakke 2022-10-17 20:12 ` [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke 2022-10-27 13:25 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Maxim Cournoyer 13 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:12 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-tempora)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Add #:test-flags. Remove #:phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/python-xyz.scm | 27 +++++---------------------- 1 file changed, 5 insertions(+), 22 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 42935cbaaa..f4113ca351 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -21199,30 +21199,13 @@ (define-public python-tempora (uri (pypi-uri "tempora" version)) (sha256 (base32 "09wirlk5vmxlhl9rnxp7g5qz2nsd6b0gnzk5fczbz0s8lsbz386b")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - (list - #:phases - #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - ;; Do not test the myproject.toml build as it tries to pull - ;; dependencies from the Internet. - (invoke "pytest" "-k" "not project"))))))) + ;; Do not test the myproject.toml build as it tries to pull + ;; dependencies from the Internet. + (list #:test-flags #~'("-vv" "-k" "not project"))) (native-inputs - (list python-pypa-build - python-freezegun + (list python-freezegun python-pytest python-pytest-black python-pytest-checkdocs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (11 preceding siblings ...) 2022-10-17 20:12 ` [bug#58587] [PATCH 13/14] gnu: python-tempora: " Marius Bakke @ 2022-10-17 20:12 ` Marius Bakke 2022-10-27 14:26 ` Maxim Cournoyer 2022-10-27 13:25 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Maxim Cournoyer 13 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-17 20:12 UTC (permalink / raw) To: 58587 * gnu/packages/simulation.scm (python-pygmsh)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/simulation.scm | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm index 39b5620822..57a7882765 100644 --- a/gnu/packages/simulation.scm +++ b/gnu/packages/simulation.scm @@ -57,6 +57,7 @@ (define-module (gnu packages simulation) #:use-module (guix svn-download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) @@ -828,7 +829,7 @@ (define-public python-pygmsh (sha256 (base32 "11flp2c4ynk1fhanf4mqyzrpd0gjbnv6afrwwc7xi3mb6ms69lr0")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases @@ -838,21 +839,8 @@ (define-public python-pygmsh ;; Due to lack of metadata, the gmsh Python package is not ;; detected although importable. (substitute* "pyproject.toml" - (("\"gmsh\",") "")))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-v" "tests")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) - (native-inputs (list python-pypa-build python-flit-core python-pytest)) + (("\"gmsh\",") ""))))))) + (native-inputs (list python-flit-core python-pytest)) (propagated-inputs (list gmsh python-meshio python-numpy)) (home-page "https://github.com/nschloe/pygmsh") (synopsis "Python frontend for Gmsh") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system. 2022-10-17 20:12 ` [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke @ 2022-10-27 14:26 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 14:26 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587 Hi, Marius Bakke <marius@gnu.org> writes: > * gnu/packages/simulation.scm (python-pygmsh)[build-system]: Switch to > PYPROJECT-BUILD-SYSTEM. > [arguments]: Remove redundant phases. > [native-inputs]: Remove PYTHON-PYPA-BUILD. LGTM, as well as all the previous patches switching to pyproject-build-system. Well done! :-) -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build. 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke ` (12 preceding siblings ...) 2022-10-17 20:12 ` [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke @ 2022-10-27 13:25 ` Maxim Cournoyer 2022-10-27 14:08 ` Marius Bakke 13 siblings, 1 reply; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 13:25 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587, Lars-Dominik Braun Hi, Marius Bakke <marius@gnu.org> writes: > From: Lars-Dominik Braun <lars@6xq.net> > > * gnu/packages/python-xyz.scm (python-setuptools): Move… > * gnu/packages/python-build.scm: …here. > * gnu/packages/chemistry.scm, > gnu/packages/messaging.scm, > gnu/packages/sequoia.scm, > tests/lint.scm: Adjust module imports accordingly. > > Co-authored-by: Marius Bakke <marius@gnu.org> [...] > +(define-public python-setuptools > + (package > + (name "python-setuptools") > + (version "64.0.3") > + (source > + (origin > + (method url-fetch) > + (uri (pypi-uri "setuptools" version)) > + (sha256 > + (base32 > + "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) > + (modules '((guix build utils))) > + (snippet > + ;; Remove included binaries which are used to build self-extracting > + ;; installers for Windows. > + ;; TODO: Find some way to build them ourself so we can include them. I don't think we need a TODO to fix installers for Windows. > + '(for-each delete-file (find-files "setuptools" > + "^(cli|gui).*\\.exe$"))))) > + (build-system python-build-system) > + ;; FIXME: Tests require pytest, which itself relies on setuptools. > + ;; One could bootstrap with an internal untested setuptools. > + (arguments (list #:tests? #f)) > + (home-page "https://pypi.org/project/setuptools/") > + (synopsis "Library designed to facilitate packaging Python projects") > + (description "Setuptools is a fully-featured, stable library designed to > +facilitate packaging Python projects, where packaging includes: > +@itemize > +@item Python package and module definitions > +@item distribution package metadata > +@item test hooks > +@item project installation > +@item platform-specific details. > +@end itemize") > + ;; TODO: setuptools now bundles the following libraries: > + ;; packaging, pyparsing, six and appdirs. How to unbundle? A better place for this TODO would be next to the snippet in the origin, perhaps? -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build. 2022-10-27 13:25 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Maxim Cournoyer @ 2022-10-27 14:08 ` Marius Bakke 2022-10-27 17:34 ` Maxim Cournoyer 0 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-27 14:08 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: 58587, Lars-Dominik Braun [-- Attachment #1: Type: text/plain, Size: 1926 bytes --] Maxim Cournoyer <maxim.cournoyer@gmail.com> skriver: >> +(define-public python-setuptools >> + (package >> + (name "python-setuptools") >> + (version "64.0.3") >> + (source >> + (origin >> + (method url-fetch) >> + (uri (pypi-uri "setuptools" version)) >> + (sha256 >> + (base32 >> + "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) >> + (modules '((guix build utils))) >> + (snippet >> + ;; Remove included binaries which are used to build self-extracting >> + ;; installers for Windows. >> + ;; TODO: Find some way to build them ourself so we can include them. > > I don't think we need a TODO to fix installers for Windows. I concur. >> + '(for-each delete-file (find-files "setuptools" >> + "^(cli|gui).*\\.exe$"))))) >> + (build-system python-build-system) >> + ;; FIXME: Tests require pytest, which itself relies on setuptools. >> + ;; One could bootstrap with an internal untested setuptools. >> + (arguments (list #:tests? #f)) >> + (home-page "https://pypi.org/project/setuptools/") >> + (synopsis "Library designed to facilitate packaging Python projects") >> + (description "Setuptools is a fully-featured, stable library designed to >> +facilitate packaging Python projects, where packaging includes: >> +@itemize >> +@item Python package and module definitions >> +@item distribution package metadata >> +@item test hooks >> +@item project installation >> +@item platform-specific details. >> +@end itemize") >> + ;; TODO: setuptools now bundles the following libraries: >> + ;; packaging, pyparsing, six and appdirs. How to unbundle? > > A better place for this TODO would be next to the snippet in the origin, > perhaps? The comments were taken verbatim from the current package definition. I'll submit a follow-up commit with these changes. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build. 2022-10-27 14:08 ` Marius Bakke @ 2022-10-27 17:34 ` Maxim Cournoyer 0 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 17:34 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587, Lars-Dominik Braun Hi Marius, Marius Bakke <marius@gnu.org> writes: > Maxim Cournoyer <maxim.cournoyer@gmail.com> skriver: [...] >>> + '(for-each delete-file (find-files "setuptools" >>> + "^(cli|gui).*\\.exe$"))))) >>> + (build-system python-build-system) >>> + ;; FIXME: Tests require pytest, which itself relies on setuptools. >>> + ;; One could bootstrap with an internal untested setuptools. >>> + (arguments (list #:tests? #f)) >>> + (home-page "https://pypi.org/project/setuptools/") >>> + (synopsis "Library designed to facilitate packaging Python projects") >>> + (description "Setuptools is a fully-featured, stable library designed to >>> +facilitate packaging Python projects, where packaging includes: >>> +@itemize >>> +@item Python package and module definitions >>> +@item distribution package metadata >>> +@item test hooks >>> +@item project installation >>> +@item platform-specific details. >>> +@end itemize") >>> + ;; TODO: setuptools now bundles the following libraries: >>> + ;; packaging, pyparsing, six and appdirs. How to unbundle? >> >> A better place for this TODO would be next to the snippet in the origin, >> perhaps? > > The comments were taken verbatim from the current package definition. > > I'll submit a follow-up commit with these changes. I noticed when I got to read the removal part of the diff ;-). Sorry for giving you more work! -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-17 20:06 [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-18 9:33 ` zimoun 2022-10-18 21:51 ` Marius Bakke 1 sibling, 1 reply; 65+ messages in thread From: zimoun @ 2022-10-18 9:33 UTC (permalink / raw) To: Marius Bakke, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer Hi Marius, On Mon, 17 Oct 2022 at 22:06, Marius Bakke <marius@gnu.org> wrote: > This is an adaptation of Lars' work in wip-python-pep517 for 'master'. Cool! > I've implemented it as a separate build system so we can do a slow > transition, starting at the leafs. [...] > I plan to eventually merge it with python-build-system instead of > keeping it separate. Thoughts? I would suggest to have a clear plan about this transition strategy. :-) I mean, if we go to two separated Python build-systems and then merge them, we have to write down the various steps; otherwise it could be quickly a mess. For example, 1. advertise about this new Python build-system 2. recommend only this new 3. only include package using this new 4. convert from old to new 5. remove old / rename old to new or whatever else. My concern is to have two build systems to maintain on the long term. Once something lands to master and the API, it is then hard to modify. Therefore, it appears to me better to draw beforehand the large lines of this transition plan. Cheers, simon ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-18 9:33 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system zimoun @ 2022-10-18 21:51 ` Marius Bakke 2022-10-19 9:49 ` zimoun 0 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-18 21:51 UTC (permalink / raw) To: zimoun, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer [-- Attachment #1.1: Type: text/plain, Size: 1530 bytes --] zimoun <zimon.toutoune@gmail.com> skriver: > Hi Marius, > > On Mon, 17 Oct 2022 at 22:06, Marius Bakke <marius@gnu.org> wrote: > >> I plan to eventually merge it with python-build-system instead of >> keeping it separate. Thoughts? > > I would suggest to have a clear plan about this transition strategy. :-) > I mean, if we go to two separated Python build-systems and then merge > them, we have to write down the various steps; otherwise it could be > quickly a mess. > > For example, > > 1. advertise about this new Python build-system > 2. recommend only this new > 3. only include package using this new > 4. convert from old to new > 5. remove old / rename old to new > > or whatever else. My concern is to have two build systems to maintain > on the long term. > > Once something lands to master and the API, it is then hard to modify. > Therefore, it appears to me better to draw beforehand the large lines of > this transition plan. I tried to address this in the attached documentation patch. The idea is to: * Recommend it for new packages in the 'guix' channel, mostly during patch review. * Once we are confident in the implementation, remove the "experimental" status and recommend it in the documentation. * Merge pieces of it into python-build-system as we go, such as sanity check changes, using python-toolchain, etc. * When we "know" that there won't be any major regressions, swap out python-build-system with pyproject-build-system entirely and mark the latter as deprecated. Thoughts? [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.2: 0001-doc-Document-pyproject-build-system.patch --] [-- Type: text/x-patch, Size: 4043 bytes --] From 628944b6f4267c4eeb5884074298b82c8d17548c Mon Sep 17 00:00:00 2001 From: Marius Bakke <marius@gnu.org> Date: Tue, 18 Oct 2022 23:38:39 +0200 Subject: [PATCH] doc: Document pyproject-build-system. * doc/guix.texi (Build Systems): Add pyproject-build-system section. * doc/contributing.texi (Python Modules): Mention pyproject.toml and the PYTHON-TOOLCHAIN package. --- doc/contributing.texi | 21 +++++++++++++-------- doc/guix.texi | 14 ++++++++++++++ 2 files changed, 27 insertions(+), 8 deletions(-) diff --git a/doc/contributing.texi b/doc/contributing.texi index 4b1eed1cb1..b9260267fa 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -791,7 +791,9 @@ described above. Dependency information for Python packages is usually available in the package source tree, with varying degrees of accuracy: in the -@file{setup.py} file, in @file{requirements.txt}, or in @file{tox.ini}. +@file{pyproject.toml} file, the @file{setup.py} file, in +@file{requirements.txt}, or in @file{tox.ini} (the latter mostly for +test dependencies). Your mission, when writing a recipe for a Python package, is to map these dependencies to the appropriate type of ``input'' (@pxref{package @@ -802,10 +804,12 @@ following check list to determine which dependency goes where. @itemize @item -We currently package Python 2 with @code{setuptools} and @code{pip} -installed like Python 3.4 has per default. Thus you don't need to -specify either of these as an input. @command{guix lint} will warn you -if you do. +We currently package Python with @code{setuptools} and @code{pip} +installed per default. This is about to change, and users are encouraged +to use @code{python-toolchain} if they want a build environment for Python. + +@command{guix lint} will warn if @code{setuptools} or @code{pip} are +added as native-inputs because they are generally not necessary. @item Python dependencies required at run time go into @@ -814,9 +818,10 @@ Python dependencies required at run time go into @file{requirements.txt} file. @item -Python packages required only at build time---e.g., those listed with -the @code{setup_requires} keyword in @file{setup.py}---or only for -testing---e.g., those in @code{tests_require}---go into +Python packages required only at build time---e.g., those listed under +@code{build-system.requires} in @file{pyproject.toml} or with the +@code{setup_requires} keyword in @file{setup.py}---or dependencies only +for testing---e.g., those in @code{tests_require} or @file{tox.ini}---go into @code{native-inputs}. The rationale is that (1) they do not need to be propagated because they are not needed at run time, and (2) in a cross-compilation context, it's the ``native'' input that we'd want. diff --git a/doc/guix.texi b/doc/guix.texi index f0fb383005..145aa16afd 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9311,7 +9311,21 @@ instead of the default @code{"out"} output. This is useful for packages that include a Python package as only a part of the software, and thus want to combine the phases of @code{python-build-system} with another build system. Python bindings are a common usecase. +@end defvr + +@defvr {Scheme Variable} pyproject-build-system +This is a variable exported by @code{guix build-system pyproject}. It +is a reimplementation of @code{python-build-system} designed around +@file{pyproject.toml} and @url{https://peps.python.org/pep-0517/, PEP 517}, +and supports a variety of build backends and test frameworks. + +It is considered ``experimental'' in that the implementation details are +not set in stone yet, however users are encouraged to try it for new +Python projects (even those using @file{setup.py}) and there should not +be any breaking changes. +Eventually this build system will be deprecated and merged back into +@code{python-build-system}, probably some time in 2024. @end defvr @defvr {Scheme Variable} perl-build-system -- 2.38.0 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-18 21:51 ` Marius Bakke @ 2022-10-19 9:49 ` zimoun 2022-10-19 23:11 ` Marius Bakke 0 siblings, 1 reply; 65+ messages in thread From: zimoun @ 2022-10-19 9:49 UTC (permalink / raw) To: Marius Bakke, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer [-- Attachment #1: Type: text/plain, Size: 735 bytes --] Hi Marius, On Tue, 18 Oct 2022 at 23:51, Marius Bakke <marius@gnu.org> wrote: > I tried to address this in the attached documentation patch. Neat! > The idea is to: > > * Recommend it for new packages in the 'guix' channel, mostly during > patch review. > * Once we are confident in the implementation, remove the "experimental" > status and recommend it in the documentation. > * Merge pieces of it into python-build-system as we go, such as sanity > check changes, using python-toolchain, etc. > * When we "know" that there won't be any major regressions, swap out > python-build-system with pyproject-build-system entirely and mark the > latter as deprecated. I would add a note, for instance, something like that: [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: patch --] [-- Type: text/x-diff, Size: 1271 bytes --] diff --git a/doc/contributing.texi b/doc/contributing.texi index 4b1eed1cb1..17dd4ec7a9 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -786,6 +786,24 @@ for instance, the module python-dateutil is packaged under the names starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as described above. +@quotation Note +Instead of the default Python build system @var{python-build-system} +(@pxref{Build Systems}), we are encouraging to try the ``experimental'' +@var{pyproject-build-system} exported by @code{guix build-system +pyproject}. It is a reimplementation of @var{python-build-system} +designed around @file{pyproject.toml} and +@url{https://peps.python.org/pep-0517/, PEP 517}, and it supports a +variety of build backends and test frameworks. + +It is considered ``experimental'' in that the implementation details are +not set in stone yet. However, there should not be any breaking +changes, so please report any failure for new Python packages (even +those using @file{setup.py}). + +Eventually this @var{pyproject-build-system} will be deprecated and +merged back into @var{python-build-system}, probably some time in 2024. +@end quotation + @subsubsection Specifying Dependencies @cindex inputs, for Python packages [-- Attachment #3: Type: text/plain, Size: 1514 bytes --] > From 628944b6f4267c4eeb5884074298b82c8d17548c Mon Sep 17 00:00:00 2001 > From: Marius Bakke <marius@gnu.org> > Date: Tue, 18 Oct 2022 23:38:39 +0200 > Subject: [PATCH] doc: Document pyproject-build-system. > > * doc/guix.texi (Build Systems): Add pyproject-build-system section. > * doc/contributing.texi (Python Modules): Mention pyproject.toml and the > PYTHON-TOOLCHAIN package. LGTM. > --- > doc/contributing.texi | 21 +++++++++++++-------- > doc/guix.texi | 14 ++++++++++++++ > 2 files changed, 27 insertions(+), 8 deletions(-) > > diff --git a/doc/contributing.texi b/doc/contributing.texi > index 4b1eed1cb1..b9260267fa 100644 > --- a/doc/contributing.texi > +++ b/doc/contributing.texi [...] > +@defvr {Scheme Variable} pyproject-build-system > +This is a variable exported by @code{guix build-system pyproject}. It > +is a reimplementation of @code{python-build-system} designed around ^ s/code/var > +@file{pyproject.toml} and @url{https://peps.python.org/pep-0517/, PEP 517}, > +and supports a variety of build backends and test frameworks. > + > +It is considered ``experimental'' in that the implementation details are > +not set in stone yet, however users are encouraged to try it for new > +Python projects (even those using @file{setup.py}) and there should not > +be any breaking changes. > > +Eventually this build system will be deprecated and merged back into > +@code{python-build-system}, probably some time in 2024. Idem Cheers, simon ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-19 9:49 ` zimoun @ 2022-10-19 23:11 ` Marius Bakke 2022-10-19 23:17 ` Marius Bakke 2022-10-20 8:08 ` zimoun 0 siblings, 2 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-19 23:11 UTC (permalink / raw) To: zimoun, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer [-- Attachment #1.1: Type: text/plain, Size: 232 bytes --] zimoun <zimon.toutoune@gmail.com> skriver: > I would add a note, for instance, something like that: [...] Good idea, although I don't like to duplicate so much information from the build system description. Here is my attempt: [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.2: Type: text/x-patch, Size: 1337 bytes --] diff --git a/doc/contributing.texi b/doc/contributing.texi index b9260267fa..d12daa7ec3 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -786,6 +786,21 @@ for instance, the module python-dateutil is packaged under the names starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as described above. +@quotation Note +Currently there are two different build systems for Python packages in Guix: +@var{python-build-system} and @var{pyproject-build-system}. For the +longest time, Python packages were built from an informally specified +@file{setup.py} file. That worked amazingly well, considering Pythons +success, but was difficult to build tooling around. As a result, a host +of alternative build systems emerged and the community eventually settled on a +@url{https://peps.python.org/pep-0517/, formal standard} for specifying build +requirements. @var{pyproject-build-system} is Guix's implementation of this +standard. It is considered ``experimental'' in that it does not yet support +all the various PEP-517 @emph{build backends}, but you are encouraged to try +it for new Python packages and report any problems. It will eventually be +deprecated and merged into @var{python-build-system}. +@end quotation + @subsubsection Specifying Dependencies @cindex inputs, for Python packages [-- Attachment #1.3: Type: text/plain, Size: 721 bytes --] >> +@defvr {Scheme Variable} pyproject-build-system >> +This is a variable exported by @code{guix build-system pyproject}. It >> +is a reimplementation of @code{python-build-system} designed around > ^ > s/code/var Fixed both instances, thanks. :-) I also made some substantial changes to the build system: * Moved 'python-toolchain' into python.scm which seems to work fine. * Inherit from python-build-system where possible. * Hint about deprecating 'add-installed-pythonpath', which has always annoyed me :-) I think a 'with-installed-pythonpath' would be much more ergonomic and plan to submit that later. * Most importantly, use three ;;; for module commentary. Patch below: [-- Attachment #1.4: 0001-fixup-pyproject.patch --] [-- Type: text/x-patch, Size: 18682 bytes --] From e765fbe232ddc57dd8ac2622a30736c5483bbf54 Mon Sep 17 00:00:00 2001 From: Marius Bakke <marius@gnu.org> Date: Wed, 19 Oct 2022 23:10:24 +0200 Subject: [PATCH] fixup pyproject --- gnu/local.mk | 1 - gnu/packages/python-commencement.scm | 64 --------- gnu/packages/python.scm | 39 +++++- guix/build-system/pyproject.scm | 2 +- guix/build/pyproject-build-system.scm | 180 ++++++++------------------ 5 files changed, 95 insertions(+), 191 deletions(-) delete mode 100644 gnu/packages/python-commencement.scm diff --git a/gnu/local.mk b/gnu/local.mk index b6e0d262be..bf598cec8b 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -511,7 +511,6 @@ GNU_SYSTEM_MODULES = \ %D%/packages/python-build.scm \ %D%/packages/python-check.scm \ %D%/packages/python-compression.scm \ - %D%/packages/python-commencement.scm \ %D%/packages/python-crypto.scm \ %D%/packages/python-science.scm \ %D%/packages/python-web.scm \ diff --git a/gnu/packages/python-commencement.scm b/gnu/packages/python-commencement.scm deleted file mode 100644 index 011ba2c38d..0000000000 --- a/gnu/packages/python-commencement.scm +++ /dev/null @@ -1,64 +0,0 @@ -;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> -;;; Copyright © 2022 Marius Bakke <marius@gnu.org> -;;; -;;; This file is part of GNU Guix. -;;; -;;; GNU Guix is free software; you can redistribute it and/or modify it -;;; under the terms of the GNU General Public License as published by -;;; the Free Software Foundation; either version 3 of the License, or (at -;;; your option) any later version. -;;; -;;; GNU Guix is distributed in the hope that it will be useful, but -;;; WITHOUT ANY WARRANTY; without even the implied warranty of -;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -;;; GNU General Public License for more details. -;;; -;;; You should have received a copy of the GNU General Public License -;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. - -(define-module (gnu packages python-commencement) - #:use-module ((guix licenses) #:prefix license:) - #:use-module (guix packages) - #:use-module (guix gexp) - #:use-module (guix build-system trivial) - #:use-module (gnu packages) - #:use-module (gnu packages python) - #:use-module (gnu packages python-build) - #:use-module (srfi srfi-1) - #:use-module (srfi srfi-26)) - -(define-public python-toolchain - (package - (name "python-toolchain") - (version (package-version python)) - (source #f) - (build-system trivial-build-system) - (arguments - (list #:modules '((guix build union)) - #:builder - #~(begin - (use-modules (ice-9 match) - (srfi srfi-1) - (guix build union)) - (union-build #$output - (filter-map (match-lambda - ((_ . directory) directory)) - %build-inputs))))) - (inputs - (list python-sans-pip-wrapper - python-pypa-build - python-pip - python-setuptools - python-wheel)) - (native-search-paths - (package-native-search-paths python)) - (search-paths - (package-search-paths python)) - (license (package-license python)) - (synopsis "Python toolchain") - (description - "Python toolchain including Python itself, setuptools and pip. Use this -package if you need a minimal Python toolchain instead of just the -interpreter.") - (home-page (package-home-page python)))) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 953044f55f..f55e5a16da 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -26,7 +26,7 @@ ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> ;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> ;;; Copyright © 2016 David Craven <david@craven.ch> -;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016-2022 Marius Bakke <marius@gnu.org> ;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> ;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> ;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> @@ -60,6 +60,7 @@ ;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> ;;; Copyright © 2022 Philip McGrath <philip@philipmcgrath.com> ;;; Copyright © 2022 jgart <jgart@dismail.de> +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> ;;; ;;; This file is part of GNU Guix. ;;; @@ -87,6 +88,7 @@ (define-module (gnu packages python) #:use-module (gnu packages hurd) #:use-module (gnu packages libffi) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python-build) #:use-module (gnu packages readline) #:use-module (gnu packages sqlite) #:use-module (gnu packages tcl) @@ -687,6 +689,41 @@ (define-public python-sans-pip (define-public python-sans-pip-wrapper (wrap-python3 python-sans-pip)) +(define-public python-toolchain + (package + (name "python-toolchain") + (version (package-version python)) + (source #f) + (build-system trivial-build-system) + (arguments + (list #:modules '((guix build union)) + #:builder + #~(begin + (use-modules (ice-9 match) + (srfi srfi-1) + (guix build union)) + (union-build #$output + (filter-map (match-lambda + ((_ . directory) directory)) + %build-inputs))))) + (inputs + (list python-sans-pip-wrapper + python-pip + python-pypa-build + python-setuptools + python-wheel)) + (native-search-paths + (package-native-search-paths python)) + (search-paths + (package-search-paths python)) + (license (package-license python)) + (synopsis "Python toolchain") + (description + "Python toolchain including Python itself, setuptools and pip. Use this +package if you need a minimal Python toolchain instead of just the +interpreter.") + (home-page (package-home-page python)))) + (define-public micropython (package (name "micropython") diff --git a/guix/build-system/pyproject.scm b/guix/build-system/pyproject.scm index 8e320529cc..d6b727ab5d 100644 --- a/guix/build-system/pyproject.scm +++ b/guix/build-system/pyproject.scm @@ -56,7 +56,7 @@ (define %pyproject-build-system-modules (define (default-python) "Return the default Python package." ;; Lazily resolve the binding to avoid a circular dependency. - (let ((python (resolve-interface '(gnu packages python-commencement)))) + (let ((python (resolve-interface '(gnu packages python)))) (module-ref python 'python-toolchain))) (define sanity-check.py diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm index 141d4d849f..db317e10e5 100644 --- a/guix/build/pyproject-build-system.scm +++ b/guix/build/pyproject-build-system.scm @@ -32,7 +32,7 @@ ;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. (define-module (guix build pyproject-build-system) - #:use-module ((guix build gnu-build-system) #:prefix gnu:) + #:use-module ((guix build python-build-system) #:prefix python:) #:use-module (guix build utils) #:use-module (guix build json) #:use-module (ice-9 match) @@ -50,31 +50,42 @@ (define-module (guix build pyproject-build-system) python-version pyproject-build)) -;; Commentary: -;; -;; PEP 517-compatible build system for Python packages. -;; -;; PEP 517 mandates the use of a TOML file called pyproject.toml at the -;; project root, describing build and runtime dependencies, as well as the -;; build system, which can be different from setuptools. This module uses -;; that file to extract the build system used and call its wheel-building -;; entry point build_wheel (see 'build). setuptools’ wheel builder is -;; used as a fallback if either no pyproject.toml exists or it does not -;; declare a build-system. It supports config_settings through the -;; standard #:configure-flags argument. -;; -;; This wheel, which is just a ZIP file with a file structure defined -;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked -;; and its contents are moved to the appropriate locations in 'install. -;; -;; Then entry points, as defined by the PyPa Entry Point Specification -;; (https://packaging.python.org/specifications/entry-points/) are read -;; from a file called entry_points.txt in the package’s site-packages -;; subdirectory and scripts are written to bin/. These are not part of a -;; wheel and expected to be created by the installing utility. -;; -;; Caveats: -;; - There is no support for in-tree build backends. +;;; Commentary: +;;; +;;; PEP 517-compatible build system for Python packages. +;;; +;;; PEP 517 mandates the use of a TOML file called pyproject.toml at the +;;; project root, describing build and runtime dependencies, as well as the +;;; build system, which can be different from setuptools. This module uses +;;; that file to extract the build system used and call its wheel-building +;;; entry point build_wheel (see 'build). setuptools’ wheel builder is +;;; used as a fallback if either no pyproject.toml exists or it does not +;;; declare a build-system. It supports config_settings through the +;;; standard #:configure-flags argument. +;;; +;;; This wheel, which is just a ZIP file with a file structure defined +;;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked +;;; and its contents are moved to the appropriate locations in 'install. +;;; +;;; Then entry points, as defined by the PyPa Entry Point Specification +;;; (https://packaging.python.org/specifications/entry-points/) are read +;;; from a file called entry_points.txt in the package’s site-packages +;;; subdirectory and scripts are written to bin/. These are not part of a +;;; wheel and expected to be created by the installing utility. +;;; +;;; Caveats: +;;; - There is no support for in-tree build backends. +;;; +;;; Code: +;;; + +;; Re-export these variables from python-build-system as many packages +;; rely on these. +(define python-version python:python-version) +(define site-packages python:site-packages) +;; XXX: This procedure is not very functional, now might be a good time +;; to deprecate it. +(define add-installed-pythonpath python:add-installed-pythonpath) ;; Base error type. (define-condition-type &python-build-error &error @@ -179,39 +190,6 @@ (define* (check #:key inputs outputs tests? test-backend test-flags #:allow-othe (else (raise (condition (&test-system-not-found)))))) (format #t "test suite not run~%"))) -(define (python-version python) - (let* ((version (last (string-split python #\-))) - (components (string-split version #\.)) - (major+minor (take components 2))) - (string-join major+minor "."))) - -(define (python-output outputs) - "Return the path of the python output, if there is one, or fall-back to out." - (or (assoc-ref outputs "python") - (assoc-ref outputs "out"))) - -(define (site-packages inputs outputs) - "Return the path of the current output's Python site-package." - (let* ((out (python-output outputs)) - (python (assoc-ref inputs "python"))) - (string-append out "/lib/python" (python-version python) "/site-packages"))) - -(define (add-installed-pythonpath inputs outputs) - "Prepend the site-package of OUTPUT to GUIX_PYTHONPATH. This is useful when -running checks after installing the package." - (setenv "GUIX_PYTHONPATH" (string-append (site-packages inputs outputs) ":" - (getenv "GUIX_PYTHONPATH")))) - -(define* (add-install-to-pythonpath #:key inputs outputs #:allow-other-keys) - "A phase that just wraps the 'add-installed-pythonpath' procedure." - (add-installed-pythonpath inputs outputs)) - -(define* (add-install-to-path #:key outputs #:allow-other-keys) - "Adding Python scripts to PATH is also often useful in tests." - (setenv "PATH" (string-append (assoc-ref outputs "out") - "/bin:" - (getenv "PATH")))) - (define* (install #:key inputs outputs (configure-flags '()) #:allow-other-keys) "Install a wheel file according to PEP 427" ;; See https://www.python.org/dev/peps/pep-0427/#installing-a-wheel-distribution-1-0-py32-none-any-whl @@ -301,9 +279,20 @@ (define (list-directories base predicate) (define* (compile-bytecode #:key inputs outputs (configure-flags '()) #:allow-other-keys) "Compile installed byte-code in site-packages." - (let ((site-dir (site-packages inputs outputs))) - (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" - site-dir))) + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (major-minor (map string->number + (take (string-split (python-version python) #\.) 2))) + (<3.7? (match major-minor + ((major minor) + (or (< major 3) (and (= major 3) (< minor 7))))))) + (if <3.7? + ;; These versions don’t have the hash invalidation modes and do + ;; not produce reproducible bytecode files. + (format #t "Skipping bytecode compilation for Python version ~a < 3.7~%" + (python-version python)) + (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" + site-dir)))) (define* (create-entrypoints #:key inputs outputs (configure-flags '()) #:allow-other-keys) "Implement Entry Points Specification @@ -371,34 +360,6 @@ (define (create-script path name module function) (entry-points.txt->entry-points f))) entry-point-files))) -(define* (wrap #:key inputs outputs search-paths #:allow-other-keys) - (define (list-of-files dir) - (find-files dir (lambda (file stat) - (and (eq? 'regular (stat:type stat)) - (not (wrapped-program? file)))))) - - (define bindirs - (append-map (match-lambda - ((_ . dir) - (list (string-append dir "/bin") - (string-append dir "/sbin")))) - outputs)) - - ;; Do not require "bash" to be present in the package inputs - ;; even when there is nothing to wrap. - ;; Also, calculate (sh) only once to prevent some I/O. - (define %sh (delay (search-input-file inputs "bin/bash"))) - (define (sh) (force %sh)) - - (let* ((var `("GUIX_PYTHONPATH" prefix - ,(search-path-as-string->list - (or (getenv "GUIX_PYTHONPATH") ""))))) - (for-each (lambda (dir) - (let ((files (list-of-files dir))) - (for-each (cut wrap-program <> #:sh (sh) var) - files))) - bindirs))) - (define* (set-SOURCE-DATE-EPOCH #:rest _) "Set the 'SOURCE_DATE_EPOCH' environment variable. This is used by tools that incorporate timestamps as a way to tell them to use a fixed timestamp. @@ -407,54 +368,25 @@ (define* (set-SOURCE-DATE-EPOCH #:rest _) ;; not support timestamps before 1980. (setenv "SOURCE_DATE_EPOCH" "315619200")) -(define* (enable-bytecode-determinism #:rest _) - "Improve determinism of pyc files." - ;; Use deterministic hashes for strings, bytes, and datetime objects. - (setenv "PYTHONHASHSEED" "0") - ;; Prevent Python from creating .pyc files when loading modules (such as - ;; when running a test suite). - (setenv "PYTHONDONTWRITEBYTECODE" "1")) - -(define* (ensure-no-cythonized-files #:rest _) - "Check the source code for @code{.c} files which may have been pre-generated -by Cython." - (for-each - (lambda (file) - (let ((generated-file - (string-append (string-drop-right file 3) "c"))) - (when (file-exists? generated-file) - (format #t "Possible Cythonized file found: ~a~%" generated-file)))) - (find-files "." "\\.pyx$"))) - (define %standard-phases ;; The build phase only builds C extensions and copies the Python sources, ;; while the install phase copies then byte-compiles the sources to the ;; prefix directory. The check phase is moved after the installation phase ;; to ease testing the built package. - (modify-phases gnu:%standard-phases - (add-after 'unpack 'enable-bytecode-determinism - enable-bytecode-determinism) - (add-after 'enable-bytecode-determinism 'ensure-no-cythonized-files - ensure-no-cythonized-files) + (modify-phases python:%standard-phases (replace 'set-SOURCE-DATE-EPOCH set-SOURCE-DATE-EPOCH) - (delete 'bootstrap) - (delete 'configure) ;not needed (replace 'build build) - (delete 'check) ;moved after the install phase (replace 'install install) - (add-after 'install 'add-install-to-pythonpath add-install-to-pythonpath) - (add-after 'add-install-to-pythonpath 'add-install-to-path - add-install-to-path) - (add-after 'add-install-to-path 'wrap wrap) - ;; must be before tests, so they can use installed packages’ entry points. + (delete 'check) + ;; Must be before tests, so they can use installed packages’ entry points. (add-before 'wrap 'create-entrypoints create-entrypoints) (add-after 'wrap 'check check) - (add-after 'check 'sanity-check sanity-check) + (replace 'sanity-check sanity-check) (add-before 'check 'compile-bytecode compile-bytecode))) (define* (pyproject-build #:key inputs (phases %standard-phases) #:allow-other-keys #:rest args) "Build the given Python package, applying all of PHASES in order." - (apply gnu:gnu-build #:inputs inputs #:phases phases args)) + (apply python:python-build #:inputs inputs #:phases phases args)) ;;; pyproject-build-system.scm ends here -- 2.38.0 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-19 23:11 ` Marius Bakke @ 2022-10-19 23:17 ` Marius Bakke 2022-10-20 8:08 ` zimoun 1 sibling, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-19 23:17 UTC (permalink / raw) To: zimoun, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer [-- Attachment #1: Type: text/plain, Size: 708 bytes --] Marius Bakke <marius@gnu.org> skriver: > I also made some substantial changes to the build system: > > * Moved 'python-toolchain' into python.scm which seems to work fine. > * Inherit from python-build-system where possible. > * Hint about deprecating 'add-installed-pythonpath', which has always > annoyed me :-) I think a 'with-installed-pythonpath' would be much > more ergonomic and plan to submit that later. > * Most importantly, use three ;;; for module commentary. Also reinstated support for Python < 3.7, which I realized will be necessary when merging back into python-build-system. I'll squash the last patch before merging the series if that was not obvious from the commit message. :-) [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-19 23:11 ` Marius Bakke 2022-10-19 23:17 ` Marius Bakke @ 2022-10-20 8:08 ` zimoun 2022-10-22 19:06 ` Marius Bakke 1 sibling, 1 reply; 65+ messages in thread From: zimoun @ 2022-10-20 8:08 UTC (permalink / raw) To: Marius Bakke, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer Hi Marius, On jeu., 20 oct. 2022 at 01:11, Marius Bakke <marius@gnu.org> wrote: > Good idea, although I don't like to duplicate so much information from > the build system description. Here is my attempt: Cool! > diff --git a/doc/contributing.texi b/doc/contributing.texi > index b9260267fa..d12daa7ec3 100644 > --- a/doc/contributing.texi > +++ b/doc/contributing.texi > @@ -786,6 +786,21 @@ for instance, the module python-dateutil is packaged under the names > starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as > described above. > > +@quotation Note > +Currently there are two different build systems for Python packages in Guix: > +@var{python-build-system} and @var{pyproject-build-system}. For the > +longest time, Python packages were built from an informally specified > +@file{setup.py} file. That worked amazingly well, considering Pythons ^ typo? > +success, but was difficult to build tooling around. As a result, a host > +of alternative build systems emerged and the community eventually settled on a > +@url{https://peps.python.org/pep-0517/, formal standard} for specifying build > +requirements. @var{pyproject-build-system} is Guix's implementation of this > +standard. It is considered ``experimental'' in that it does not yet support > +all the various PEP-517 @emph{build backends}, but you are encouraged to try > +it for new Python packages and report any problems. It will eventually be > +deprecated and merged into @var{python-build-system}. > +@end quotation LTGM. For the rest of the series, I have not yet given a look. Cheers, simon ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-20 8:08 ` zimoun @ 2022-10-22 19:06 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (2 more replies) 0 siblings, 3 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:06 UTC (permalink / raw) To: zimoun, 58587; +Cc: Lars-Dominik Braun, Maxim Cournoyer [-- Attachment #1: Type: text/plain, Size: 2581 bytes --] zimoun <zimon.toutoune@gmail.com> skriver: > Hi Marius, > > On jeu., 20 oct. 2022 at 01:11, Marius Bakke <marius@gnu.org> wrote: > >> diff --git a/doc/contributing.texi b/doc/contributing.texi >> index b9260267fa..d12daa7ec3 100644 >> --- a/doc/contributing.texi >> +++ b/doc/contributing.texi >> @@ -786,6 +786,21 @@ for instance, the module python-dateutil is packaged under the names >> starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as >> described above. >> >> +@quotation Note >> +Currently there are two different build systems for Python packages in Guix: >> +@var{python-build-system} and @var{pyproject-build-system}. For the >> +longest time, Python packages were built from an informally specified >> +@file{setup.py} file. That worked amazingly well, considering Pythons > ^ > typo? Right, I've adjusted it! >> +success, but was difficult to build tooling around. As a result, a host >> +of alternative build systems emerged and the community eventually settled on a >> +@url{https://peps.python.org/pep-0517/, formal standard} for specifying build >> +requirements. @var{pyproject-build-system} is Guix's implementation of this >> +standard. It is considered ``experimental'' in that it does not yet support >> +all the various PEP-517 @emph{build backends}, but you are encouraged to try >> +it for new Python packages and report any problems. It will eventually be >> +deprecated and merged into @var{python-build-system}. >> +@end quotation > > LTGM. Thanks for checking. I've now added a section in the manual about API changes and stability, and removed more duplicate code from pyproject-build-system. I also added a news entry. I think this is now ready for merge, but have some concerns: * The python-toolchain package is a union that includes setuptools, wheel and others. Packages that pick up a reference to the build-time Python will needlessly include those other packages. It could be worked around by propagating instead of using a union. * Overriding #:test-flags means having to add back the implicit defaults for verbose output. We could add them even when #:test-flags is present so users don't need to add "-vv" manually. Thoughts? I will merge this in a few days if there are no further comments. It can still be tweaked directly on 'master' if necessary. Updated patches follow shortly. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build. 2022-10-22 19:06 ` Marius Bakke @ 2022-10-22 19:09 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 02/22] gnu: pypy: Move to separate module Marius Bakke ` (3 more replies) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-27 13:10 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Maxim Cournoyer 2 siblings, 4 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:09 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> * gnu/packages/python-xyz.scm (python-setuptools): Move… * gnu/packages/python-build.scm: …here. * gnu/packages/chemistry.scm, gnu/packages/messaging.scm, gnu/packages/sequoia.scm, tests/lint.scm: Adjust module imports accordingly. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/packages/chemistry.scm | 1 + gnu/packages/messaging.scm | 1 + gnu/packages/python-build.scm | 40 +++++++++++++++++++++++++++++++++++ gnu/packages/python-xyz.scm | 40 ----------------------------------- gnu/packages/sequoia.scm | 2 +- tests/lint.scm | 2 +- 6 files changed, 44 insertions(+), 42 deletions(-) diff --git a/gnu/packages/chemistry.scm b/gnu/packages/chemistry.scm index d0582c4b6f..6549da3453 100644 --- a/gnu/packages/chemistry.scm +++ b/gnu/packages/chemistry.scm @@ -56,6 +56,7 @@ (define-module (gnu packages chemistry) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-xyz) #:use-module (gnu packages qt) #:use-module (gnu packages serialization) diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm index fa889d24a7..86d9914c90 100644 --- a/gnu/packages/messaging.scm +++ b/gnu/packages/messaging.scm @@ -116,6 +116,7 @@ (define-module (gnu packages messaging) #:use-module (gnu packages pkg-config) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-check) #:use-module (gnu packages python-crypto) #:use-module (gnu packages python-web) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 9d9b07f769..3ff988820f 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,46 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-setuptools + (package + (name "python-setuptools") + (version "64.0.3") + (source + (origin + (method url-fetch) + (uri (pypi-uri "setuptools" version)) + (sha256 + (base32 + "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) + (modules '((guix build utils))) + (snippet + ;; Remove included binaries which are used to build self-extracting + ;; installers for Windows. + ;; TODO: Find some way to build them ourself so we can include them. + '(for-each delete-file (find-files "setuptools" + "^(cli|gui).*\\.exe$"))))) + (build-system python-build-system) + ;; FIXME: Tests require pytest, which itself relies on setuptools. + ;; One could bootstrap with an internal untested setuptools. + (arguments (list #:tests? #f)) + (home-page "https://pypi.org/project/setuptools/") + (synopsis "Library designed to facilitate packaging Python projects") + (description "Setuptools is a fully-featured, stable library designed to +facilitate packaging Python projects, where packaging includes: +@itemize +@item Python package and module definitions +@item distribution package metadata +@item test hooks +@item project installation +@item platform-specific details. +@end itemize") + ;; TODO: setuptools now bundles the following libraries: + ;; packaging, pyparsing, six and appdirs. How to unbundle? + (license (list license:psfl ;setuptools itself + license:expat ;six, appdirs, pyparsing + license:asl2.0 ;packaging is dual ASL2/BSD-2 + license:bsd-2)))) + (define-public python-wheel (package (name "python-wheel") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index ebc56bbc11..3ea29ec3b9 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -2043,46 +2043,6 @@ (define-public python-serpent other machines, such as over the network.") (license license:expat))) -(define-public python-setuptools - (package - (name "python-setuptools") - (version "64.0.3") - (source - (origin - (method url-fetch) - (uri (pypi-uri "setuptools" version)) - (sha256 - (base32 - "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) - (modules '((guix build utils))) - (snippet - ;; Remove included binaries which are used to build self-extracting - ;; installers for Windows. - ;; TODO: Find some way to build them ourself so we can include them. - '(for-each delete-file (find-files "setuptools" - "^(cli|gui).*\\.exe$"))))) - (build-system python-build-system) - ;; FIXME: Tests require pytest, which itself relies on setuptools. - ;; One could bootstrap with an internal untested setuptools. - (arguments (list #:tests? #f)) - (home-page "https://pypi.org/project/setuptools/") - (synopsis "Library designed to facilitate packaging Python projects") - (description "Setuptools is a fully-featured, stable library designed to -facilitate packaging Python projects, where packaging includes: -@itemize -@item Python package and module definitions -@item distribution package metadata -@item test hooks -@item project installation -@item platform-specific details. -@end itemize") - ;; TODO: setuptools now bundles the following libraries: - ;; packaging, pyparsing, six and appdirs. How to unbundle? - (license (list license:psfl ;setuptools itself - license:expat ;six, appdirs, pyparsing - license:asl2.0 ;packaging is dual ASL2/BSD-2 - license:bsd-2)))) - (define-public python-setuptools-declarative-requirements (package (name "python-setuptools-declarative-requirements") diff --git a/gnu/packages/sequoia.scm b/gnu/packages/sequoia.scm index a8d21ac1a4..7002f684b8 100644 --- a/gnu/packages/sequoia.scm +++ b/gnu/packages/sequoia.scm @@ -36,7 +36,7 @@ (define-module (gnu packages sequoia) #:use-module (gnu packages nettle) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) - #:use-module (gnu packages python-xyz) ;; python-setuptools + #:use-module (gnu packages python-build) ;python-setuptools #:use-module (gnu packages rust) #:use-module (gnu packages tls)) diff --git a/tests/lint.scm b/tests/lint.scm index 8be74d2604..641486f89b 100644 --- a/tests/lint.scm +++ b/tests/lint.scm @@ -49,7 +49,7 @@ (define-module (test-lint) #:use-module (gnu packages) #:use-module (gnu packages glib) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) + #:use-module (gnu packages python-build) #:use-module ((gnu packages bash) #:select (bash bash-minimal)) #:use-module (web uri) #:use-module (web server) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v2 02/22] gnu: pypy: Move to separate module. 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-22 19:09 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke ` (2 subsequent siblings) 3 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:09 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This removes the need to import (gnu packages python-xyz) in (gnu packages python), avoiding issues with circular imports. * gnu/packages/python.scm (pypy): Move… * gnu/packages/pypy.scm (pypy): …here * gnu/local.mk: Register new file. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/local.mk | 1 + gnu/packages/pypy.scm | 273 ++++++++++++++++++++++++++++++++++++++++ gnu/packages/python.scm | 170 ------------------------- 3 files changed, 274 insertions(+), 170 deletions(-) create mode 100644 gnu/packages/pypy.scm diff --git a/gnu/local.mk b/gnu/local.mk index 8247180bef..bf598cec8b 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -506,6 +506,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/pure.scm \ %D%/packages/purescript.scm \ %D%/packages/pv.scm \ + %D%/packages/pypy.scm \ %D%/packages/python.scm \ %D%/packages/python-build.scm \ %D%/packages/python-check.scm \ diff --git a/gnu/packages/pypy.scm b/gnu/packages/pypy.scm new file mode 100644 index 0000000000..002cfd59cd --- /dev/null +++ b/gnu/packages/pypy.scm @@ -0,0 +1,273 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> +;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2021 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013, 2014, 2015, 2016 Andreas Enge <andreas@enge.fr> +;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org> +;;; Copyright © 2014, 2017, 2019 Eric Bavier <bavier@member.fsf.org> +;;; Copyright © 2014, 2015 Federico Beffa <beffa@fbengineering.ch> +;;; Copyright © 2015 Omar Radwan <toxemicsquire4@gmail.com> +;;; Copyright © 2015 Pierre-Antoine Rault <par@rigelk.eu> +;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net> +;;; Copyright © 2015, 2016 Christine Lemmer-Webber <cwebber@dustycloud.org> +;;; Copyright © 2015 Eric Dvorsak <eric@dvorsak.fr> +;;; Copyright © 2015, 2016 David Thompson <davet@gnu.org> +;;; Copyright © 2015, 2016, 2017, 2021 Leo Famulari <leo@famulari.name> +;;; Copyright © 2015, 2017 Ben Woodcroft <donttrustben@gmail.com> +;;; Copyright © 2015, 2016 Erik Edrosa <erik.edrosa@gmail.com> +;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il> +;;; Copyright © 2015, 2017 Kyle Meyer <kyle@kyleam.com> +;;; Copyright © 2015, 2016 Chris Marusich <cmmarusich@gmail.com> +;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org> +;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox.org> +;;; Copyright © 2016, 2018 Hartmut Goebel <h.goebel@crazy-compilers.com> +;;; Copyright © 2016 Daniel Pimentel <d4n1@d4n1.org> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@gmail.com> +;;; Copyright © 2016, 2017 Troy Sankey <sankeytms@gmail.com> +;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016 David Craven <david@craven.ch> +;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> +;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> +;;; Copyright © 2016–2018, 2021 Tobias Geerinckx-Rice <me@tobias.gr> +;;; Copyright © 2016, 2017, 2018, 2021 Julien Lepiller <julien@lepiller.eu> +;;; Copyright © 2016, 2017 Thomas Danckaert <post@thomasdanckaert.be> +;;; Copyright © 2017 Carlo Zancanaro <carlo@zancanaro.id.au> +;;; Copyright © 2017 Frederick M. Muriithi <fredmanglis@gmail.com> +;;; Copyright © 2017, 2018 Adriano Peluso <catonano@gmail.com> +;;; Copyright © 2017 Ben Sturmfels <ben@sturm.com.au> +;;; Copyright © 2017, 2018, 2019 Mathieu Othacehe <m.othacehe@gmail.com> +;;; Copyright © 2017 José Miguel Sánchez García <jmi2k@openmailbox.org> +;;; Copyright © 2017 Roel Janssen <roel@gnu.org> +;;; Copyright © 2017, 2018 Kei Kebreau <kkebreau@posteo.net> +;;; Copyright © 2017 Rutger Helling <rhelling@mykolab.com> +;;; Copyright © 2017 Muriithi Frederick Muriuki <fredmanglis@gmail.com> +;;; Copyright © 2017 Brendan Tildesley <mail@brendan.scot> +;;; Copyright © 2018 Ethan R. Jones <ethanrjones97@gmail.com +;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com> +;;; Copyright © 2018 Vijayalakshmi Vedantham <vijimay12@gmail.com> +;;; Copyright © 2018 Mathieu Lirzin <mthl@gnu.org> +;;; Copyright © 2018 Adam Massmann <massmannak@gmail.com> +;;; Copyright © 2016, 2018 Tomáš Čech <sleep_walker@gnu.org> +;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr> +;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com> +;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org> +;;; Copyright © 2018, 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> +;;; Copyright © 2018 Luther Thompson <lutheroto@gmail.com> +;;; Copyright © 2018 Vagrant Cascadian <vagrant@debian.org> +;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org> +;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> +;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu packages pypy) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (gnu packages) + #:use-module (gnu packages base) + #:use-module (gnu packages bash) + #:use-module (gnu packages certs) + #:use-module (gnu packages check) + #:use-module (gnu packages compression) + #:use-module (gnu packages dbm) + #:use-module (gnu packages hurd) + #:use-module (gnu packages libffi) + #:use-module (gnu packages ncurses) + #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) + #:use-module (gnu packages readline) + #:use-module (gnu packages shells) + #:use-module (gnu packages sqlite) + #:use-module (gnu packages tcl) + #:use-module (gnu packages tls) + #:use-module (gnu packages xml) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix utils) + #:use-module (guix build-system gnu) + #:use-module (guix build-system trivial) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26)) + +(define-public pypy + (package + (name "pypy") + (version "7.3.5") + (source (origin + (method url-fetch) + (uri (string-append "https://downloads.python.org/pypy/" + "pypy3.7-v" version "-src.tar.bz2")) + (sha256 + (base32 + "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) + (build-system gnu-build-system) + (arguments + (list + #:tests? #f ;FIXME: 43 out of 364 tests are failing + #:modules '((ice-9 ftw) (ice-9 match) + (guix build utils) (guix build gnu-build-system)) + #:disallowed-references (list nss-certs) + #:phases + #~(modify-phases %standard-phases + (delete 'configure) + (add-after 'unpack 'patch-source + (lambda* (#:key inputs #:allow-other-keys) + (substitute* '("rpython/rlib/clibffi.py") + ;; find_library does not work for libc + (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) + (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") + ;; Make reproducible (XXX: unused?) + (("time\\.localtime\\(\\)") "time.gmtime(0)")) + (substitute* '("pypy/module/sys/version.py") + ;; Make reproducible + (("t\\.gmtime\\(\\)") "t.gmtime(0)")) + (substitute* '("lib_pypy/_tkinter/tklib_build.py") + ;; Link to versioned libtcl and libtk + (("linklibs = \\['tcl', 'tk'\\]") + "linklibs = ['tcl8.6', 'tk8.6']") + (("incdirs = \\[\\]") + (string-append + "incdirs = ['" + #$(this-package-input "tcl") "/include', '" + #$(this-package-input "tk") "/include']"))) + (substitute* '("lib_pypy/_curses_build.py") + ;; Find curses + (("/usr/local") + #$(this-package-input "ncurses"))) + (substitute* '("lib_pypy/_dbm.py") + ;; Use gdbm compat library, so we don’t need to pull + ;; in bdb. + (("ctypes.util.find_library\\('db'\\)") + (format #f "~s" (search-input-file + inputs "lib/libgdbm_compat.so")))) + (substitute* '("lib_pypy/_sqlite3_build.py") + ;; Always use search paths + (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") + ;; Find sqlite3 + (("/usr/local") (assoc-ref inputs "sqlite")) + (("libname = 'sqlite3'") + (format #f "libname = ~s" + (search-input-file inputs "lib/libsqlite3.so.0")))) + (substitute* '("lib-python/3/subprocess.py") + ;; Fix shell path + (("/bin/sh") + (search-input-file inputs "/bin/sh"))) + (substitute* '("lib-python/3/distutils/unixccompiler.py") + ;; gcc-toolchain does not provide symlink cc -> gcc + (("\"cc\"") "\"gcc\"")))) + (add-after 'unpack 'set-source-file-times-to-1980 + ;; copied from python package, required by zip testcase + (lambda _ + (let ((circa-1980 (* 10 366 24 60 60))) + (ftw "." (lambda (file stat flag) + (utime file circa-1980 circa-1980) + #t))))) + (replace 'build + (lambda _ + (with-directory-excursion "pypy/goal" + ;; Build with jit optimization. + (invoke "python2" + "../../rpython/bin/rpython" + (string-append "--make-jobs=" + (number->string (parallel-job-count))) + "-Ojit" + "targetpypystandalone" + "--allworkingmodules")) + ;; Build c modules and package everything, so tests work. + (with-directory-excursion "pypy/tool/release" + (invoke "python2" "package.py" + "--archive-name" "pypy-dist" + "--builddir" (getcwd))))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (if tests? + (begin + (setenv "HOME" "/tmp") ; test_with_pip tries to + ; access ~/.cache/pip + ;; Run library tests only (no interpreter unit tests). + ;; This is what Gentoo does. + (invoke "python" "pypy/test_all.py" + "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" + "lib-python")) + (format #t "test suite not run~%")))) + (replace 'install + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) + (shebang-match-python "#!.+/bin/python") + (shebang-pypy3 (string-append "#!" bin-pypy3)) + (dist-dir "pypy/tool/release/pypy-dist")) + (with-directory-excursion dist-dir + ;; Delete test data. + (for-each + (lambda (x) + (delete-file-recursively (string-append + "lib-python/3/" x))) + '("tkinter/test" + "test" + "sqlite3/test" + "lib2to3/tests" + "idlelib/idle_test" + "distutils/tests" + "ctypes/test" + "unittest/test")) + ;; Patch shebang referencing python. + (substitute* '("lib-python/3/cgi.py" + "lib-python/3/encodings/rot_13.py") + ((shebang-match-python) shebang-pypy3)) + (with-fluids ((%default-port-encoding "ISO-8859-1")) + (substitute* '("lib_pypy/_md5.py" + "lib_pypy/_sha1.py") + ((shebang-match-python) shebang-pypy3)))) + (copy-recursively dist-dir #$output))))))) + (native-inputs + (list gzip + nss-certs ; For ssl tests + pkg-config + python-2 + python2-pycparser + tar)) ; Required for package.py + (inputs + (list bzip2 + expat + gdbm + glibc + libffi + ncurses + openssl + sqlite + tcl + tk + xz + zlib)) + (home-page "https://www.pypy.org/") + (synopsis "Python implementation with just-in-time compilation") + (description "PyPy is a faster, alternative implementation of the Python +programming language employing a just-in-time compiler. It supports most +Python code natively, including C extensions.") + (license (list license:expat ; pypy itself; _pytest/ + license:psfl ; python standard library in lib-python/ + license:asl2.0 ; dotviewer/font/ and some of lib-python/ + license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* + license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ + (license:non-copyleft + "http://www.unicode.org/copyright.html"))))) + +(define-public pypy3 + (deprecated-package "pypy3" pypy)) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index e31158dfc0..00c3eb7774 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -81,17 +81,13 @@ (define-module (gnu packages python) #:use-module (gnu packages) #:use-module (gnu packages base) #:use-module (gnu packages bash) - #:use-module (gnu packages certs) #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages dbm) #:use-module (gnu packages hurd) #:use-module (gnu packages libffi) - #:use-module (gnu packages ncurses) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) - #:use-module (gnu packages shells) #:use-module (gnu packages sqlite) #:use-module (gnu packages tcl) #:use-module (gnu packages tls) @@ -736,169 +732,3 @@ (define-public micropython compatible with normal Python as possible to allow you to transfer code with ease from the desktop to a microcontroller or embedded system.") (license license:expat))) - -(define-public pypy - (package - (name "pypy") - (version "7.3.5") - (source (origin - (method url-fetch) - (uri (string-append "https://downloads.python.org/pypy/" - "pypy3.7-v" version "-src.tar.bz2")) - (sha256 - (base32 - "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) - (build-system gnu-build-system) - (arguments - (list - #:tests? #f ;FIXME: 43 out of 364 tests are failing - #:modules '((ice-9 ftw) (ice-9 match) - (guix build utils) (guix build gnu-build-system)) - #:disallowed-references (list nss-certs) - #:phases - #~(modify-phases %standard-phases - (delete 'configure) - (add-after 'unpack 'patch-source - (lambda* (#:key inputs #:allow-other-keys) - (substitute* '("rpython/rlib/clibffi.py") - ;; find_library does not work for libc - (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) - (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") - ;; Make reproducible (XXX: unused?) - (("time\\.localtime\\(\\)") "time.gmtime(0)")) - (substitute* '("pypy/module/sys/version.py") - ;; Make reproducible - (("t\\.gmtime\\(\\)") "t.gmtime(0)")) - (substitute* '("lib_pypy/_tkinter/tklib_build.py") - ;; Link to versioned libtcl and libtk - (("linklibs = \\['tcl', 'tk'\\]") - "linklibs = ['tcl8.6', 'tk8.6']") - (("incdirs = \\[\\]") - (string-append - "incdirs = ['" - #$(this-package-input "tcl") "/include', '" - #$(this-package-input "tk") "/include']"))) - (substitute* '("lib_pypy/_curses_build.py") - ;; Find curses - (("/usr/local") - #$(this-package-input "ncurses"))) - (substitute* '("lib_pypy/_dbm.py") - ;; Use gdbm compat library, so we don’t need to pull - ;; in bdb. - (("ctypes.util.find_library\\('db'\\)") - (format #f "~s" (search-input-file - inputs "lib/libgdbm_compat.so")))) - (substitute* '("lib_pypy/_sqlite3_build.py") - ;; Always use search paths - (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") - ;; Find sqlite3 - (("/usr/local") (assoc-ref inputs "sqlite")) - (("libname = 'sqlite3'") - (format #f "libname = ~s" - (search-input-file inputs "lib/libsqlite3.so.0")))) - (substitute* '("lib-python/3/subprocess.py") - ;; Fix shell path - (("/bin/sh") - (search-input-file inputs "/bin/sh"))) - (substitute* '("lib-python/3/distutils/unixccompiler.py") - ;; gcc-toolchain does not provide symlink cc -> gcc - (("\"cc\"") "\"gcc\"")))) - (add-after 'unpack 'set-source-file-times-to-1980 - ;; copied from python package, required by zip testcase - (lambda _ - (let ((circa-1980 (* 10 366 24 60 60))) - (ftw "." (lambda (file stat flag) - (utime file circa-1980 circa-1980) - #t))))) - (replace 'build - (lambda _ - (with-directory-excursion "pypy/goal" - ;; Build with jit optimization. - (invoke "python2" - "../../rpython/bin/rpython" - (string-append "--make-jobs=" - (number->string (parallel-job-count))) - "-Ojit" - "targetpypystandalone" - "--allworkingmodules")) - ;; Build c modules and package everything, so tests work. - (with-directory-excursion "pypy/tool/release" - (invoke "python2" "package.py" - "--archive-name" "pypy-dist" - "--builddir" (getcwd))))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (if tests? - (begin - (setenv "HOME" "/tmp") ; test_with_pip tries to - ; access ~/.cache/pip - ;; Run library tests only (no interpreter unit tests). - ;; This is what Gentoo does. - (invoke "python" "pypy/test_all.py" - "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" - "lib-python")) - (format #t "test suite not run~%")))) - (replace 'install - (lambda* (#:key inputs outputs #:allow-other-keys) - (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) - (shebang-match-python "#!.+/bin/python") - (shebang-pypy3 (string-append "#!" bin-pypy3)) - (dist-dir "pypy/tool/release/pypy-dist")) - (with-directory-excursion dist-dir - ;; Delete test data. - (for-each - (lambda (x) - (delete-file-recursively (string-append - "lib-python/3/" x))) - '("tkinter/test" - "test" - "sqlite3/test" - "lib2to3/tests" - "idlelib/idle_test" - "distutils/tests" - "ctypes/test" - "unittest/test")) - ;; Patch shebang referencing python. - (substitute* '("lib-python/3/cgi.py" - "lib-python/3/encodings/rot_13.py") - ((shebang-match-python) shebang-pypy3)) - (with-fluids ((%default-port-encoding "ISO-8859-1")) - (substitute* '("lib_pypy/_md5.py" - "lib_pypy/_sha1.py") - ((shebang-match-python) shebang-pypy3)))) - (copy-recursively dist-dir #$output))))))) - (native-inputs - (list gzip - nss-certs ; For ssl tests - pkg-config - python-2 - python2-pycparser - tar)) ; Required for package.py - (inputs - (list bzip2 - expat - gdbm - glibc - libffi - ncurses - openssl - sqlite - tcl - tk - xz - zlib)) - (home-page "https://www.pypy.org/") - (synopsis "Python implementation with just-in-time compilation") - (description "PyPy is a faster, alternative implementation of the Python -programming language employing a just-in-time compiler. It supports most -Python code natively, including C extensions.") - (license (list license:expat ; pypy itself; _pytest/ - license:psfl ; python standard library in lib-python/ - license:asl2.0 ; dotviewer/font/ and some of lib-python/ - license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* - license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ - (license:non-copyleft - "http://www.unicode.org/copyright.html"))))) - -(define-public pypy3 - (deprecated-package "pypy3" pypy)) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v2 03/22] gnu: python-pip: Move to (gnu packages python-build). 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 02/22] gnu: pypy: Move to separate module Marius Bakke @ 2022-10-22 19:09 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 04/22] build-system: Add pyproject-build-system Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke 3 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:09 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-pip): Move from here ... * gnu/packages/python-build.scm (python-pip): ... to here. --- gnu/packages/python-build.scm | 21 +++++++++++++++++++++ gnu/packages/python-xyz.scm | 21 --------------------- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 3ff988820f..c8f57ea286 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,27 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-pip + (package + (name "python-pip") + (version "22.2.2") + (source + (origin + (method url-fetch) + (uri (pypi-uri "pip" version)) + (sha256 + (base32 + "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) + (build-system python-build-system) + (arguments + '(#:tests? #f)) ; there are no tests in the pypi archive. + (home-page "https://pip.pypa.io/") + (synopsis "Package manager for Python software") + (description + "Pip is a package manager for Python software, that finds packages on the +Python Package Index (PyPI).") + (license license:expat))) + (define-public python-setuptools (package (name "python-setuptools") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 3ea29ec3b9..1ded6f0384 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -12650,27 +12650,6 @@ (define-public python-pretend responses, rather than doing any computation.") (license license:bsd-3))) -(define-public python-pip - (package - (name "python-pip") - (version "22.2.2") - (source - (origin - (method url-fetch) - (uri (pypi-uri "pip" version)) - (sha256 - (base32 - "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) - (build-system python-build-system) - (arguments - '(#:tests? #f)) ; there are no tests in the pypi archive. - (home-page "https://pip.pypa.io/") - (synopsis "Package manager for Python software") - (description - "Pip is a package manager for Python software, that finds packages on the -Python Package Index (PyPI).") - (license license:expat))) - ;;; Variant used to break a dependency cycle with ;;; python-pytest-perf-bootstrap. (define-public python-pip-run-bootstrap -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v2 04/22] build-system: Add pyproject-build-system. 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 02/22] gnu: pypy: Move to separate module Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke @ 2022-10-22 19:09 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke 3 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:09 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This is an experimental build system based on python-build-system that implements PEP 517-compliant builds. * doc/guix.texi (Build Systems): Add pyproject-build-system section. * doc/contributing.texi (Python Modules): Mention pyproject.toml and the PYTHON-TOOLCHAIN package, as well as differences to python-build-system. * guix/build-system/pyproject.scm, guix/build/pyproject-build-system.scm, gnu/packages/aux-files/python/sanity-check-next.py, gnu/packages/python-commencement.scm: New files. * Makefile.am (MODULES): Register the new build systems. * gnu/local.mk (GNU_SYSTEM_MODULES): Add python-commencement.scm. * gnu/packages/python.scm (python-sans-pip, python-sans-pip-wrapper): New variables. Co-authored-by: Marius Bakke <marius@gnu.org> --- Makefile.am | 2 + doc/contributing.texi | 36 +- doc/guix.texi | 29 ++ .../aux-files/python/sanity-check-next.py | 98 +++++ gnu/packages/python.scm | 52 ++- guix/build-system/pyproject.scm | 148 +++++++ guix/build/pyproject-build-system.scm | 367 ++++++++++++++++++ 7 files changed, 723 insertions(+), 9 deletions(-) create mode 100644 gnu/packages/aux-files/python/sanity-check-next.py create mode 100644 guix/build-system/pyproject.scm create mode 100644 guix/build/pyproject-build-system.scm diff --git a/Makefile.am b/Makefile.am index 22dcc43f99..6ccb790c11 100644 --- a/Makefile.am +++ b/Makefile.am @@ -166,6 +166,7 @@ MODULES = \ guix/build-system/maven.scm \ guix/build-system/node.scm \ guix/build-system/perl.scm \ + guix/build-system/pyproject.scm \ guix/build-system/python.scm \ guix/build-system/renpy.scm \ guix/build-system/ocaml.scm \ @@ -222,6 +223,7 @@ MODULES = \ guix/build/minetest-build-system.scm \ guix/build/node-build-system.scm \ guix/build/perl-build-system.scm \ + guix/build/pyproject-build-system.scm \ guix/build/python-build-system.scm \ guix/build/ocaml-build-system.scm \ guix/build/qt-build-system.scm \ diff --git a/doc/contributing.texi b/doc/contributing.texi index 4b1eed1cb1..c3221d23e4 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -786,12 +786,29 @@ for instance, the module python-dateutil is packaged under the names starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as described above. +@quotation Note +Currently there are two different build systems for Python packages in Guix: +@var{python-build-system} and @var{pyproject-build-system}. For the +longest time, Python packages were built from an informally specified +@file{setup.py} file. That worked amazingly well, considering Python's +success, but was difficult to build tooling around. As a result, a host +of alternative build systems emerged and the community eventually settled on a +@url{https://peps.python.org/pep-0517/, formal standard} for specifying build +requirements. @var{pyproject-build-system} is Guix's implementation of this +standard. It is considered ``experimental'' in that it does not yet support +all the various PEP-517 @emph{build backends}, but you are encouraged to try +it for new Python packages and report any problems. It will eventually be +deprecated and merged into @var{python-build-system}. +@end quotation + @subsubsection Specifying Dependencies @cindex inputs, for Python packages Dependency information for Python packages is usually available in the package source tree, with varying degrees of accuracy: in the -@file{setup.py} file, in @file{requirements.txt}, or in @file{tox.ini}. +@file{pyproject.toml} file, the @file{setup.py} file, in +@file{requirements.txt}, or in @file{tox.ini} (the latter mostly for +test dependencies). Your mission, when writing a recipe for a Python package, is to map these dependencies to the appropriate type of ``input'' (@pxref{package @@ -802,10 +819,12 @@ following check list to determine which dependency goes where. @itemize @item -We currently package Python 2 with @code{setuptools} and @code{pip} -installed like Python 3.4 has per default. Thus you don't need to -specify either of these as an input. @command{guix lint} will warn you -if you do. +We currently package Python with @code{setuptools} and @code{pip} +installed per default. This is about to change, and users are encouraged +to use @code{python-toolchain} if they want a build environment for Python. + +@command{guix lint} will warn if @code{setuptools} or @code{pip} are +added as native-inputs because they are generally not necessary. @item Python dependencies required at run time go into @@ -814,9 +833,10 @@ Python dependencies required at run time go into @file{requirements.txt} file. @item -Python packages required only at build time---e.g., those listed with -the @code{setup_requires} keyword in @file{setup.py}---or only for -testing---e.g., those in @code{tests_require}---go into +Python packages required only at build time---e.g., those listed under +@code{build-system.requires} in @file{pyproject.toml} or with the +@code{setup_requires} keyword in @file{setup.py}---or dependencies only +for testing---e.g., those in @code{tests_require} or @file{tox.ini}---go into @code{native-inputs}. The rationale is that (1) they do not need to be propagated because they are not needed at run time, and (2) in a cross-compilation context, it's the ``native'' input that we'd want. diff --git a/doc/guix.texi b/doc/guix.texi index f0fb383005..621f9923b4 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9311,7 +9311,36 @@ instead of the default @code{"out"} output. This is useful for packages that include a Python package as only a part of the software, and thus want to combine the phases of @code{python-build-system} with another build system. Python bindings are a common usecase. +@end defvr + +@defvr {Scheme Variable} pyproject-build-system +This is a variable exported by @code{guix build-system pyproject}. It +is based on of @var{python-build-system}, and adds support for +@file{pyproject.toml} and @url{https://peps.python.org/pep-0517/, PEP 517}. +It also supports a variety of build backends and test frameworks. + +The API is slightly different from @var{python-build-system}: +@enumerate +@item +@code{#:use-setuptools?} and @code{#:test-target} is removed. +@item +@code{#:build-backend} is added. It defaults to @code{#false} and will try +to guess the appropriate backend based on @file{pyproject.toml}. +@item +@code{#:test-backend} is added. It defaults to @code{#false} and will guess +an appropriate test backend based on what is available in package inputs. +@item +@code{#:test-flags} is added. The default is @code{#false}, and varies based +on the detected @code{#:test-backend}. +@end enumerate + +It is considered ``experimental'' in that the implementation details are +not set in stone yet, however users are encouraged to try it for new +Python projects (even those using @file{setup.py}). The API is subject to +change, but any breaking changes in the Guix channel will be dealt with. +Eventually this build system will be deprecated and merged back into +@var{python-build-system}, probably some time in 2024. @end defvr @defvr {Scheme Variable} perl-build-system diff --git a/gnu/packages/aux-files/python/sanity-check-next.py b/gnu/packages/aux-files/python/sanity-check-next.py new file mode 100644 index 0000000000..891606f72b --- /dev/null +++ b/gnu/packages/aux-files/python/sanity-check-next.py @@ -0,0 +1,98 @@ +# -*- coding: utf-8 -*- + +# This version adds a small change to accomodate missing python-setuptools. +# Original patch by Lars-Dominik Braun in wip-python-pep517, commit +# 720dbe22d431262938be29dd9a9ddb78c44a99b3. +# --- sanity-check.py 2022-06-12 14:40:06.814337702 +0200 +# +++ sanity-check.py 2022-10-16 23:21:38.990651568 +0200 +# @@ -19,9 +19,13 @@ +# from __future__ import print_function # Python 2 support. +# import importlib +# -import pkg_resources +# import sys +# import traceback +# +try: +# + import pkg_resources +# +except ImportError: +# + print('Warning: Skipping, because python-setuptools are not available.') +# + sys.exit(0) + +# TODO: Merge with sanity-check.py in the next core-updates cycle. + +from __future__ import print_function # Python 2 support. +import importlib +import sys +import traceback +try: + import pkg_resources +except ImportError: + print('Warning: Skipping, because python-setuptools are not available.') + sys.exit(0) + +try: + from importlib.machinery import PathFinder +except ImportError: + PathFinder = None + +ret = 0 + +# Only check site-packages installed by this package, but not dependencies +# (which pkg_resources.working_set would include). Path supplied via argv. +ws = pkg_resources.find_distributions(sys.argv[1]) + +for dist in ws: + print('validating', repr(dist.project_name), dist.location) + try: + print('...checking requirements: ', end='') + req = str(dist.as_requirement()) + # dist.activate() is not enough to actually check requirements, we + # have to .require() it. + pkg_resources.require(req) + print('OK') + except Exception as e: + print('ERROR:', req, repr(e)) + ret = 1 + continue + + # Try to load top level modules. This should not have any side-effects. + try: + metalines = dist.get_metadata_lines('top_level.txt') + except (KeyError, EnvironmentError): + # distutils (i.e. #:use-setuptools? #f) will not install any metadata. + # This file is also missing for packages built using a PEP 517 builder + # such as poetry. + print('WARNING: cannot determine top-level modules') + continue + for name in metalines: + # Only available on Python 3. + if PathFinder and PathFinder.find_spec(name) is None: + # Ignore unavailable modules, often C modules, which were not + # installed at the top-level. Cannot use ModuleNotFoundError, + # because it is raised by failed imports too. + continue + try: + print('...trying to load module', name, end=': ') + importlib.import_module(name) + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + + # Try to load entry points of console scripts too, making sure they + # work. They should be removed if they don't. Other groups may not be + # safe, as they can depend on optional packages. + for group, v in dist.get_entry_map().items(): + if group not in {'console_scripts', 'gui_scripts'}: + continue + for name, ep in v.items(): + try: + print('...trying to load endpoint', group, name, end=': ') + ep.load() + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + +sys.exit(ret) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 00c3eb7774..f55e5a16da 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -26,7 +26,7 @@ ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> ;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> ;;; Copyright © 2016 David Craven <david@craven.ch> -;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016-2022 Marius Bakke <marius@gnu.org> ;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> ;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> ;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> @@ -60,6 +60,7 @@ ;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> ;;; Copyright © 2022 Philip McGrath <philip@philipmcgrath.com> ;;; Copyright © 2022 jgart <jgart@dismail.de> +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> ;;; ;;; This file is part of GNU Guix. ;;; @@ -87,6 +88,7 @@ (define-module (gnu packages python) #:use-module (gnu packages hurd) #:use-module (gnu packages libffi) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python-build) #:use-module (gnu packages readline) #:use-module (gnu packages sqlite) #:use-module (gnu packages tcl) @@ -674,6 +676,54 @@ (define* (wrap-python3 python (define-public python-wrapper (wrap-python3 python)) (define-public python-minimal-wrapper (wrap-python3 python-minimal)) +;; The Python used in pyproject-build-system. +(define-public python-sans-pip + (hidden-package + (package/inherit python + (arguments + (substitute-keyword-arguments (package-arguments python) + ((#:configure-flags flags #~()) + #~(append '("--with-ensurepip=no") + (delete "--with-ensurepip=install" #$flags)))))))) + +(define-public python-sans-pip-wrapper + (wrap-python3 python-sans-pip)) + +(define-public python-toolchain + (package + (name "python-toolchain") + (version (package-version python)) + (source #f) + (build-system trivial-build-system) + (arguments + (list #:modules '((guix build union)) + #:builder + #~(begin + (use-modules (ice-9 match) + (srfi srfi-1) + (guix build union)) + (union-build #$output + (filter-map (match-lambda + ((_ . directory) directory)) + %build-inputs))))) + (inputs + (list python-sans-pip-wrapper + python-pip + python-pypa-build + python-setuptools + python-wheel)) + (native-search-paths + (package-native-search-paths python)) + (search-paths + (package-search-paths python)) + (license (package-license python)) + (synopsis "Python toolchain") + (description + "Python toolchain including Python itself, setuptools and pip. Use this +package if you need a minimal Python toolchain instead of just the +interpreter.") + (home-page (package-home-page python)))) + (define-public micropython (package (name "micropython") diff --git a/guix/build-system/pyproject.scm b/guix/build-system/pyproject.scm new file mode 100644 index 0000000000..d6b727ab5d --- /dev/null +++ b/guix/build-system/pyproject.scm @@ -0,0 +1,148 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build-system pyproject) + #:use-module ((gnu packages) #:select (search-auxiliary-file)) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (guix memoization) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix packages) + #:use-module (guix derivations) + #:use-module (guix search-paths) + #:use-module (guix build-system) + #:use-module (guix build-system gnu) + #:use-module (guix build-system python) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (%pyproject-build-system-modules + default-python + pyproject-build + pyproject-build-system)) + +;; Commentary: +;; +;; Standard build procedure for Python packages using 'pyproject.toml'. +;; This is implemented as an extension of 'python-build-system'. +;; +;; Code: + +(define %pyproject-build-system-modules + ;; Build-side modules imported by default. + `((guix build pyproject-build-system) + (guix build json) + ,@%python-build-system-modules + ,@%gnu-build-system-modules)) + +(define (default-python) + "Return the default Python package." + ;; Lazily resolve the binding to avoid a circular dependency. + (let ((python (resolve-interface '(gnu packages python)))) + (module-ref python 'python-toolchain))) + +(define sanity-check.py + ;; TODO: Merge with sanity-check.py in the next rebuild cycle. + (search-auxiliary-file "python/sanity-check-next.py")) + +(define* (lower name + #:key source inputs native-inputs outputs system target + (python (default-python)) + #:allow-other-keys + #:rest arguments) + "Return a bag for NAME." + (define private-keywords + '(#:target #:python #:inputs #:native-inputs)) + + (and (not target) ;XXX: no cross-compilation + (bag + (name name) + (system system) + (host-inputs `(,@(if source + `(("source" ,source)) + '()) + ,@inputs + + ;; Keep the standard inputs of 'gnu-build-system'. + ,@(standard-packages))) + (build-inputs `(("python" ,python) + ("sanity-check.py" ,(local-file sanity-check.py)) + ,@native-inputs)) + (outputs (append outputs '(wheel))) + (build pyproject-build) + (arguments (strip-keyword-arguments private-keywords arguments))))) + +(define* (pyproject-build name inputs + #:key source + (tests? #t) + (configure-flags ''()) + (build-backend #f) + (test-backend #f) + (test-flags #f) + (phases '%standard-phases) + (outputs '("out" "wheel")) + (search-paths '()) + (system (%current-system)) + (guile #f) + (imported-modules %pyproject-build-system-modules) + (modules '((guix build pyproject-build-system) + (guix build utils)))) + "Build SOURCE using PYTHON, and with INPUTS." + (define build + (with-imported-modules imported-modules + #~(begin + (use-modules #$@(sexp->gexp modules)) + + #$(with-build-variables inputs outputs + #~(pyproject-build + #:name #$name + #:source #+source + #:configure-flags #$configure-flags + #:system #$system + #:build-backend #$build-backend + #:test-backend #$test-backend + #:test-flags #$test-flags + #:tests? #$tests? + #:phases #$(if (pair? phases) + (sexp->gexp phases) + phases) + #:outputs %outputs + #:search-paths '#$(sexp->gexp + (map search-path-specification->sexp + search-paths)) + #:inputs %build-inputs))))) + + + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system #:graft? #f))) + (gexp->derivation name build + #:system system + #:graft? #f ;consistent with 'gnu-build' + #:target #f + #:guile-for-build guile))) + +(define pyproject-build-system + (build-system + (name 'pyproject) + (description "The PEP517-compliant Python build system") + (lower lower))) + +;;; pyproject.scm ends here diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm new file mode 100644 index 0000000000..16a6db9537 --- /dev/null +++ b/guix/build/pyproject-build-system.scm @@ -0,0 +1,367 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build pyproject-build-system) + #:use-module ((guix build python-build-system) #:prefix python:) + #:use-module (guix build utils) + #:use-module (guix build json) + #:use-module (ice-9 match) + #:use-module (ice-9 ftw) + #:use-module (ice-9 format) + #:use-module (ice-9 rdelim) + #:use-module (ice-9 regex) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:export (%standard-phases + add-installed-pythonpath + site-packages + python-version + pyproject-build)) + +;;; Commentary: +;;; +;;; PEP 517-compatible build system for Python packages. +;;; +;;; PEP 517 mandates the use of a TOML file called pyproject.toml at the +;;; project root, describing build and runtime dependencies, as well as the +;;; build system, which can be different from setuptools. This module uses +;;; that file to extract the build system used and call its wheel-building +;;; entry point build_wheel (see 'build). setuptools’ wheel builder is +;;; used as a fallback if either no pyproject.toml exists or it does not +;;; declare a build-system. It supports config_settings through the +;;; standard #:configure-flags argument. +;;; +;;; This wheel, which is just a ZIP file with a file structure defined +;;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked +;;; and its contents are moved to the appropriate locations in 'install. +;;; +;;; Then entry points, as defined by the PyPa Entry Point Specification +;;; (https://packaging.python.org/specifications/entry-points/) are read +;;; from a file called entry_points.txt in the package’s site-packages +;;; subdirectory and scripts are written to bin/. These are not part of a +;;; wheel and expected to be created by the installing utility. +;;; TODO: Add support for PEP-621 entry points. +;;; +;;; Caveats: +;;; - There is no support for in-tree build backends. +;;; +;;; Code: +;;; + +;; Re-export these variables from python-build-system as many packages +;; rely on these. +(define python-version python:python-version) +(define site-packages python:site-packages) +(define add-installed-pythonpath python:add-installed-pythonpath) + +;; Base error type. +(define-condition-type &python-build-error &error + python-build-error?) + +;; Raised when 'check cannot find a valid test system in the inputs. +(define-condition-type &test-system-not-found &python-build-error + test-system-not-found?) + +;; Raised when multiple wheels are created by 'build. +(define-condition-type &cannot-extract-multiple-wheels &python-build-error + cannot-extract-multiple-wheels?) + +;; Raised, when no wheel has been built by the build system. +(define-condition-type &no-wheels-built &python-build-error + no-wheels-built?) + +(define* (build #:key outputs build-backend configure-flags #:allow-other-keys) + "Build a given Python package." + + (define (pyproject.toml->build-backend file) + "Look up the build backend in a pyproject.toml file." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in 'concat))) + (if (eof-object? line) + #f + (let ((m (string-match "build-backend = [\"'](.+)[\"']" line))) + (if m (match:substring m 1) + (loop (read-line in 'concat))))))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + ;; There is no easy way to get data from Guile into Python via + ;; s-expressions, but we have JSON serialization already, which Python + ;; also supports out-of-the-box. + (config-settings (call-with-output-string (cut write-json configure-flags <>))) + ;; python-setuptools’ default backend supports setup.py *and* + ;; pyproject.toml. Allow overriding this automatic detection via + ;; build-backend. + (auto-build-backend (if (file-exists? "pyproject.toml") + (pyproject.toml->build-backend "pyproject.toml") + #f)) + ;; Use build system detection here and not in importer, because a) we + ;; have alot of legacy packages and b) the importer cannot update arbitrary + ;; fields in case a package switches its build system. + (use-build-backend (or + build-backend + auto-build-backend + "setuptools.build_meta"))) + (format #t "Using '~a' to build wheels, auto-detected '~a', override '~a'.~%" + use-build-backend auto-build-backend build-backend) + (mkdir-p wheel-dir) + ;; Call the PEP 517 build function, which drops a .whl into wheel-dir. + (invoke "python" "-c" "import sys, importlib, json +config_settings = json.loads (sys.argv[3]) +builder = importlib.import_module(sys.argv[1]) +builder.build_wheel(sys.argv[2], config_settings=config_settings)" + use-build-backend wheel-dir config-settings))) + +(define* (check #:key inputs outputs tests? test-backend test-flags #:allow-other-keys) + "Run the test suite of a given Python package." + (if tests? + ;; Unfortunately with PEP 517 there is no common method to specify test + ;; systems. Guess test system based on inputs instead. + (let* ((pytest (which "pytest")) + (nosetests (which "nosetests")) + (nose2 (which "nose2")) + (have-setup-py (file-exists? "setup.py")) + (use-test-backend + (or + test-backend + ;; Prefer pytest + (if pytest 'pytest #f) + (if nosetests 'nose #f) + (if nose2 'nose2 #f) + ;; But fall back to setup.py, which should work for most + ;; packages. XXX: would be nice not to depend on setup.py here? fails + ;; more often than not to find any tests at all. Maybe we can run + ;; `python -m unittest`? + (if have-setup-py 'setup.py #f)))) + (format #t "Using ~a~%" use-test-backend) + (match use-test-backend + ('pytest + (apply invoke (cons pytest (or test-flags '("-vv"))))) + ('nose + (apply invoke (cons nosetests (or test-flags '("-v"))))) + ('nose2 + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) + ('setup.py + (apply invoke (append '("python" "setup.py") (or test-flags '("test" "-v"))))) + ;; The developer should explicitly disable tests in this case. + (else (raise (condition (&test-system-not-found)))))) + (format #t "test suite not run~%"))) + +(define* (install #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Install a wheel file according to PEP 427" + ;; See https://www.python.org/dev/peps/pep-0427/#installing-a-wheel-distribution-1-0-py32-none-any-whl + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (out (assoc-ref outputs "out"))) + (define (extract file) + "Extract wheel (ZIP file) into site-packages directory" + ;; Use Python’s zipfile to avoid extra dependency + (invoke "python" "-m" "zipfile" "-e" file site-dir)) + + (define python-hashbang + (string-append "#!" python "/bin/python")) + + (define* (merge-directories source destination #:optional (post-move #f)) + "Move all files in SOURCE into DESTINATION, merging the two directories." + (format #t "Merging directory ~a into ~a~%" source destination) + (for-each + (lambda (file) + (format #t "~a/~a -> ~a/~a~%" source file destination file) + (mkdir-p destination) + (rename-file + (string-append source "/" file) + (string-append destination "/" file)) + (when post-move + (post-move file))) + (scandir source (negate (cut member <> '("." ".."))))) + (rmdir source)) + + (define (expand-data-directory directory) + "Move files from all .data subdirectories to their respective +destinations." + ;; Python’s distutils.command.install defines this mapping from source to + ;; destination mapping. + (let ((source (string-append directory "/scripts")) + (destination (string-append out "/bin"))) + (when (file-exists? source) + (merge-directories + source + destination + (lambda (f) + (let ((dest-path (string-append destination "/" f))) + (chmod dest-path #o755) + (substitute* dest-path (("#!python") python-hashbang))))))) + ;; data can create arbitrary directory structures. Most commonly + ;; it is used for share/. + (let ((source (string-append directory "/data")) + (destination out)) + (when (file-exists? source) + (merge-directories source destination))) + (let* ((distribution (car (string-split (basename directory) #\-))) + (source (string-append directory "/headers")) + (destination (string-append out "/include/python" (python-version python) "/" distribution))) + (when (file-exists? source) + (merge-directories source destination)))) + + (define (list-directories base predicate) + ;; Cannot use find-files here, because it’s recursive. + (scandir + base + (lambda (name) + (let ((stat (lstat (string-append base "/" name)))) + (and + (not (member name '("." ".."))) + (eq? (stat:type stat) 'directory) + (predicate name stat)))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + (wheels (map (cut string-append wheel-dir "/" <>) + (scandir wheel-dir (cut string-suffix? ".whl" <>))))) + (cond + ((> (length wheels) 1) ; This code does not support multiple wheels + ; yet, because their outputs would have to be + ; merged properly. + (raise (condition (&cannot-extract-multiple-wheels)))) + ((= (length wheels) 0) + (raise (condition (&no-wheels-built))))) + (for-each extract wheels)) + (let ((datadirs (map + (cut string-append site-dir "/" <>) + (list-directories site-dir (file-name-predicate "\\.data$"))))) + (for-each (lambda (directory) + (expand-data-directory directory) + (rmdir directory)) + datadirs)))) + +(define* (compile-bytecode #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Compile installed byte-code in site-packages." + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (major-minor (map string->number + (take (string-split (python-version python) #\.) 2))) + (<3.7? (match major-minor + ((major minor) + (or (< major 3) (and (= major 3) (< minor 7))))))) + (if <3.7? + ;; These versions don’t have the hash invalidation modes and do + ;; not produce reproducible bytecode files. + (format #t "Skipping bytecode compilation for Python version ~a < 3.7~%" + (python-version python)) + (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" + site-dir)))) + +(define* (create-entrypoints #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Implement Entry Points Specification +(https://packaging.python.org/specifications/entry-points/) by PyPa, +which creates runnable scripts in bin/ from entry point specification +file entry_points.txt. This is necessary, because wheels do not contain +these binaries and installers are expected to create them." + + (define (entry-points.txt->entry-points file) + "Specialized parser for Python configfile-like files, in particular +entry_points.txt. Returns a list of console_script and gui_scripts +entry points." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in)) + (inside #f) + (result '())) + (if (eof-object? line) + result + (let* ((group-match (string-match "^\\[(.+)\\]$" line)) + (group-name (if group-match (match:substring group-match 1) #f)) + (next-inside + (if (not group-name) + inside + (or + (string=? group-name "console_scripts") + (string=? group-name "gui_scripts")))) + (item-match (string-match "^([^ =]+)\\s*=\\s*([^:]+):(.+)$" line))) + (if (and inside item-match) + (loop (read-line in) next-inside (cons (list + (match:substring item-match 1) + (match:substring item-match 2) + (match:substring item-match 3)) + result)) + (loop (read-line in) next-inside result)))))))) + + (define (create-script path name module function) + "Create a Python script from an entry point’s NAME, MODULE and + FUNCTION and return write it to PATH/NAME." + (let ((interpreter (which "python")) + (file-path (string-append path "/" name))) + (format #t "Creating entry point for '~a.~a' at '~a'.~%" module function + file-path) + (call-with-output-file file-path + (lambda (port) + ;; Technically the script could also include search-paths, + ;; but having a generic 'wrap phases also handles manually + ;; written entry point scripts. + (format port "#!~a +# Auto-generated entry point script. +import sys +import ~a as mod +sys.exit (mod.~a ())~%" interpreter module function))) + (chmod file-path #o755))) + + (let* ((site-dir (site-packages inputs outputs)) + (out (assoc-ref outputs "out")) + (bin-dir (string-append out "/bin")) + (entry-point-files (find-files site-dir "^entry_points.txt$"))) + (mkdir-p bin-dir) + (for-each + (lambda (f) + (for-each + (lambda (ep) (apply create-script (cons bin-dir ep))) + (entry-points.txt->entry-points f))) + entry-point-files))) + +(define* (set-SOURCE-DATE-EPOCH #:rest _) + "Set the 'SOURCE_DATE_EPOCH' environment variable. This is used by tools +that incorporate timestamps as a way to tell them to use a fixed timestamp. +See https://reproducible-builds.org/specs/source-date-epoch/." + ;; Use a post-1980 timestamp because the Zip format used in wheels do + ;; not support timestamps before 1980. + (setenv "SOURCE_DATE_EPOCH" "315619200")) + +(define %standard-phases + ;; The build phase only builds C extensions and copies the Python sources, + ;; while the install phase copies then byte-compiles the sources to the + ;; prefix directory. The check phase is moved after the installation phase + ;; to ease testing the built package. + (modify-phases python:%standard-phases + (replace 'set-SOURCE-DATE-EPOCH set-SOURCE-DATE-EPOCH) + (replace 'build build) + (replace 'install install) + (delete 'check) + ;; Must be before tests, so they can use installed packages’ entry points. + (add-before 'wrap 'create-entrypoints create-entrypoints) + (add-after 'wrap 'check check) + (add-before 'check 'compile-bytecode compile-bytecode))) + +(define* (pyproject-build #:key inputs (phases %standard-phases) + #:allow-other-keys #:rest args) + "Build the given Python package, applying all of PHASES in order." + (apply python:python-build #:inputs inputs #:phases phases args)) + +;;; pyproject-build-system.scm ends here -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v2 05/22] news: Add entry for 'pyproject-build-system'. 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (2 preceding siblings ...) 2022-10-22 19:09 ` [bug#58587] [PATCH v2 04/22] build-system: Add pyproject-build-system Marius Bakke @ 2022-10-22 19:09 ` Marius Bakke 3 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 19:09 UTC (permalink / raw) To: 58587 * etc/news.scm: Add entry. --- etc/news.scm | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/etc/news.scm b/etc/news.scm index d9ffa036a2..a19eb9aeac 100644 --- a/etc/news.scm +++ b/etc/news.scm @@ -6,7 +6,7 @@ ;; Copyright © 2019, 2020 Konrad Hinsen <konrad.hinsen@fastmail.net> ;; Copyright © 2019, 2020, 2021 Julien Lepiller <julien@lepiller.eu> ;; Copyright © 2019–2022 Florian Pelz <pelzflorian@pelzflorian.de> -;; Copyright © 2020 Marius Bakke <mbakke@fastmail.com> +;; Copyright © 2020, 2022 Marius Bakke <marius@gnu.org> ;; Copyright © 2020, 2021 Mathieu Othacehe <m.othacehe@gmail.com> ;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;; Copyright © 2020, 2021, 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com> @@ -26,6 +26,26 @@ (channel-news (version 0) + (entry (commit "80ad0214389ef682a8037833711397be94b33b97") + (title + (en "New build system for Python packages")) + (body + (en "A new @var{pyproject-build-system} has been added. This +is a redesign of @var{python-build-system} with support for @dfn{PEP 517} +and @file{pyproject.toml} files. It also has built-in support for various +test frameworks such as @command{pytest} and @code{nosetests}. + +There is a complementary @code{python-toolchain} package that comes with +updated versions of @command{pip}, @command{setuptools} and others. + +The build system will eventually be merged into @var{python-build-system} +but you are encouraged to use it for packages in the @code{guix} channel. +Third party channels may want to wait until the API is stable (see the +Guix manual for caveats). + +Despite the name, @var{pyproject-build-system} also works with the +``legacy'' @file{setup.py} format."))) + (entry (commit "c7ba5f38b80433b040d3946b8fc0b1e8621ba30a") (title (en "New @option{--emulate-fhs} option for @command{guix shell}") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build. 2022-10-22 19:06 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 02/22] gnu: pypy: Move to separate module Marius Bakke ` (20 more replies) 2022-10-27 13:10 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Maxim Cournoyer 2 siblings, 21 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> * gnu/packages/python-xyz.scm (python-setuptools): Move… * gnu/packages/python-build.scm: …here. * gnu/packages/chemistry.scm, gnu/packages/messaging.scm, gnu/packages/sequoia.scm, tests/lint.scm: Adjust module imports accordingly. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/packages/chemistry.scm | 1 + gnu/packages/messaging.scm | 1 + gnu/packages/python-build.scm | 40 +++++++++++++++++++++++++++++++++++ gnu/packages/python-xyz.scm | 40 ----------------------------------- gnu/packages/sequoia.scm | 2 +- tests/lint.scm | 2 +- 6 files changed, 44 insertions(+), 42 deletions(-) diff --git a/gnu/packages/chemistry.scm b/gnu/packages/chemistry.scm index d0582c4b6f..6549da3453 100644 --- a/gnu/packages/chemistry.scm +++ b/gnu/packages/chemistry.scm @@ -56,6 +56,7 @@ (define-module (gnu packages chemistry) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-xyz) #:use-module (gnu packages qt) #:use-module (gnu packages serialization) diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm index fa889d24a7..86d9914c90 100644 --- a/gnu/packages/messaging.scm +++ b/gnu/packages/messaging.scm @@ -116,6 +116,7 @@ (define-module (gnu packages messaging) #:use-module (gnu packages pkg-config) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-check) #:use-module (gnu packages python-crypto) #:use-module (gnu packages python-web) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 9d9b07f769..3ff988820f 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,46 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-setuptools + (package + (name "python-setuptools") + (version "64.0.3") + (source + (origin + (method url-fetch) + (uri (pypi-uri "setuptools" version)) + (sha256 + (base32 + "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) + (modules '((guix build utils))) + (snippet + ;; Remove included binaries which are used to build self-extracting + ;; installers for Windows. + ;; TODO: Find some way to build them ourself so we can include them. + '(for-each delete-file (find-files "setuptools" + "^(cli|gui).*\\.exe$"))))) + (build-system python-build-system) + ;; FIXME: Tests require pytest, which itself relies on setuptools. + ;; One could bootstrap with an internal untested setuptools. + (arguments (list #:tests? #f)) + (home-page "https://pypi.org/project/setuptools/") + (synopsis "Library designed to facilitate packaging Python projects") + (description "Setuptools is a fully-featured, stable library designed to +facilitate packaging Python projects, where packaging includes: +@itemize +@item Python package and module definitions +@item distribution package metadata +@item test hooks +@item project installation +@item platform-specific details. +@end itemize") + ;; TODO: setuptools now bundles the following libraries: + ;; packaging, pyparsing, six and appdirs. How to unbundle? + (license (list license:psfl ;setuptools itself + license:expat ;six, appdirs, pyparsing + license:asl2.0 ;packaging is dual ASL2/BSD-2 + license:bsd-2)))) + (define-public python-wheel (package (name "python-wheel") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index ebc56bbc11..3ea29ec3b9 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -2043,46 +2043,6 @@ (define-public python-serpent other machines, such as over the network.") (license license:expat))) -(define-public python-setuptools - (package - (name "python-setuptools") - (version "64.0.3") - (source - (origin - (method url-fetch) - (uri (pypi-uri "setuptools" version)) - (sha256 - (base32 - "1sllqf0bhsl2yilf1w0xnlz0r4yaksmwaj0ap91zdc6kgbigdjiv")) - (modules '((guix build utils))) - (snippet - ;; Remove included binaries which are used to build self-extracting - ;; installers for Windows. - ;; TODO: Find some way to build them ourself so we can include them. - '(for-each delete-file (find-files "setuptools" - "^(cli|gui).*\\.exe$"))))) - (build-system python-build-system) - ;; FIXME: Tests require pytest, which itself relies on setuptools. - ;; One could bootstrap with an internal untested setuptools. - (arguments (list #:tests? #f)) - (home-page "https://pypi.org/project/setuptools/") - (synopsis "Library designed to facilitate packaging Python projects") - (description "Setuptools is a fully-featured, stable library designed to -facilitate packaging Python projects, where packaging includes: -@itemize -@item Python package and module definitions -@item distribution package metadata -@item test hooks -@item project installation -@item platform-specific details. -@end itemize") - ;; TODO: setuptools now bundles the following libraries: - ;; packaging, pyparsing, six and appdirs. How to unbundle? - (license (list license:psfl ;setuptools itself - license:expat ;six, appdirs, pyparsing - license:asl2.0 ;packaging is dual ASL2/BSD-2 - license:bsd-2)))) - (define-public python-setuptools-declarative-requirements (package (name "python-setuptools-declarative-requirements") diff --git a/gnu/packages/sequoia.scm b/gnu/packages/sequoia.scm index a8d21ac1a4..7002f684b8 100644 --- a/gnu/packages/sequoia.scm +++ b/gnu/packages/sequoia.scm @@ -36,7 +36,7 @@ (define-module (gnu packages sequoia) #:use-module (gnu packages nettle) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) - #:use-module (gnu packages python-xyz) ;; python-setuptools + #:use-module (gnu packages python-build) ;python-setuptools #:use-module (gnu packages rust) #:use-module (gnu packages tls)) diff --git a/tests/lint.scm b/tests/lint.scm index 8be74d2604..641486f89b 100644 --- a/tests/lint.scm +++ b/tests/lint.scm @@ -49,7 +49,7 @@ (define-module (test-lint) #:use-module (gnu packages) #:use-module (gnu packages glib) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) + #:use-module (gnu packages python-build) #:use-module ((gnu packages bash) #:select (bash bash-minimal)) #:use-module (web uri) #:use-module (web server) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 02/22] gnu: pypy: Move to separate module. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke ` (19 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This removes the need to import (gnu packages python-xyz) in (gnu packages python), avoiding issues with circular imports. * gnu/packages/python.scm (pypy): Move… * gnu/packages/pypy.scm (pypy): …here * gnu/local.mk: Register new file. Co-authored-by: Marius Bakke <marius@gnu.org> --- gnu/local.mk | 1 + gnu/packages/pypy.scm | 273 ++++++++++++++++++++++++++++++++++++++++ gnu/packages/python.scm | 170 ------------------------- 3 files changed, 274 insertions(+), 170 deletions(-) create mode 100644 gnu/packages/pypy.scm diff --git a/gnu/local.mk b/gnu/local.mk index 8247180bef..bf598cec8b 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -506,6 +506,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/pure.scm \ %D%/packages/purescript.scm \ %D%/packages/pv.scm \ + %D%/packages/pypy.scm \ %D%/packages/python.scm \ %D%/packages/python-build.scm \ %D%/packages/python-check.scm \ diff --git a/gnu/packages/pypy.scm b/gnu/packages/pypy.scm new file mode 100644 index 0000000000..002cfd59cd --- /dev/null +++ b/gnu/packages/pypy.scm @@ -0,0 +1,273 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org> +;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2021 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013, 2014, 2015, 2016 Andreas Enge <andreas@enge.fr> +;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org> +;;; Copyright © 2014, 2017, 2019 Eric Bavier <bavier@member.fsf.org> +;;; Copyright © 2014, 2015 Federico Beffa <beffa@fbengineering.ch> +;;; Copyright © 2015 Omar Radwan <toxemicsquire4@gmail.com> +;;; Copyright © 2015 Pierre-Antoine Rault <par@rigelk.eu> +;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net> +;;; Copyright © 2015, 2016 Christine Lemmer-Webber <cwebber@dustycloud.org> +;;; Copyright © 2015 Eric Dvorsak <eric@dvorsak.fr> +;;; Copyright © 2015, 2016 David Thompson <davet@gnu.org> +;;; Copyright © 2015, 2016, 2017, 2021 Leo Famulari <leo@famulari.name> +;;; Copyright © 2015, 2017 Ben Woodcroft <donttrustben@gmail.com> +;;; Copyright © 2015, 2016 Erik Edrosa <erik.edrosa@gmail.com> +;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il> +;;; Copyright © 2015, 2017 Kyle Meyer <kyle@kyleam.com> +;;; Copyright © 2015, 2016 Chris Marusich <cmmarusich@gmail.com> +;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org> +;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox.org> +;;; Copyright © 2016, 2018 Hartmut Goebel <h.goebel@crazy-compilers.com> +;;; Copyright © 2016 Daniel Pimentel <d4n1@d4n1.org> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@gmail.com> +;;; Copyright © 2016, 2017 Troy Sankey <sankeytms@gmail.com> +;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016 David Craven <david@craven.ch> +;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> +;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> +;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> +;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> +;;; Copyright © 2016–2018, 2021 Tobias Geerinckx-Rice <me@tobias.gr> +;;; Copyright © 2016, 2017, 2018, 2021 Julien Lepiller <julien@lepiller.eu> +;;; Copyright © 2016, 2017 Thomas Danckaert <post@thomasdanckaert.be> +;;; Copyright © 2017 Carlo Zancanaro <carlo@zancanaro.id.au> +;;; Copyright © 2017 Frederick M. Muriithi <fredmanglis@gmail.com> +;;; Copyright © 2017, 2018 Adriano Peluso <catonano@gmail.com> +;;; Copyright © 2017 Ben Sturmfels <ben@sturm.com.au> +;;; Copyright © 2017, 2018, 2019 Mathieu Othacehe <m.othacehe@gmail.com> +;;; Copyright © 2017 José Miguel Sánchez García <jmi2k@openmailbox.org> +;;; Copyright © 2017 Roel Janssen <roel@gnu.org> +;;; Copyright © 2017, 2018 Kei Kebreau <kkebreau@posteo.net> +;;; Copyright © 2017 Rutger Helling <rhelling@mykolab.com> +;;; Copyright © 2017 Muriithi Frederick Muriuki <fredmanglis@gmail.com> +;;; Copyright © 2017 Brendan Tildesley <mail@brendan.scot> +;;; Copyright © 2018 Ethan R. Jones <ethanrjones97@gmail.com +;;; Copyright © 2018 Fis Trivial <ybbs.daans@hotmail.com> +;;; Copyright © 2018 Vijayalakshmi Vedantham <vijimay12@gmail.com> +;;; Copyright © 2018 Mathieu Lirzin <mthl@gnu.org> +;;; Copyright © 2018 Adam Massmann <massmannak@gmail.com> +;;; Copyright © 2016, 2018 Tomáš Čech <sleep_walker@gnu.org> +;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr> +;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com> +;;; Copyright © 2018 Clément Lassieur <clement@lassieur.org> +;;; Copyright © 2018, 2019, 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> +;;; Copyright © 2018 Luther Thompson <lutheroto@gmail.com> +;;; Copyright © 2018 Vagrant Cascadian <vagrant@debian.org> +;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org> +;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> +;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu packages pypy) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (gnu packages) + #:use-module (gnu packages base) + #:use-module (gnu packages bash) + #:use-module (gnu packages certs) + #:use-module (gnu packages check) + #:use-module (gnu packages compression) + #:use-module (gnu packages dbm) + #:use-module (gnu packages hurd) + #:use-module (gnu packages libffi) + #:use-module (gnu packages ncurses) + #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) + #:use-module (gnu packages readline) + #:use-module (gnu packages shells) + #:use-module (gnu packages sqlite) + #:use-module (gnu packages tcl) + #:use-module (gnu packages tls) + #:use-module (gnu packages xml) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix download) + #:use-module (guix utils) + #:use-module (guix build-system gnu) + #:use-module (guix build-system trivial) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26)) + +(define-public pypy + (package + (name "pypy") + (version "7.3.5") + (source (origin + (method url-fetch) + (uri (string-append "https://downloads.python.org/pypy/" + "pypy3.7-v" version "-src.tar.bz2")) + (sha256 + (base32 + "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) + (build-system gnu-build-system) + (arguments + (list + #:tests? #f ;FIXME: 43 out of 364 tests are failing + #:modules '((ice-9 ftw) (ice-9 match) + (guix build utils) (guix build gnu-build-system)) + #:disallowed-references (list nss-certs) + #:phases + #~(modify-phases %standard-phases + (delete 'configure) + (add-after 'unpack 'patch-source + (lambda* (#:key inputs #:allow-other-keys) + (substitute* '("rpython/rlib/clibffi.py") + ;; find_library does not work for libc + (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) + (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") + ;; Make reproducible (XXX: unused?) + (("time\\.localtime\\(\\)") "time.gmtime(0)")) + (substitute* '("pypy/module/sys/version.py") + ;; Make reproducible + (("t\\.gmtime\\(\\)") "t.gmtime(0)")) + (substitute* '("lib_pypy/_tkinter/tklib_build.py") + ;; Link to versioned libtcl and libtk + (("linklibs = \\['tcl', 'tk'\\]") + "linklibs = ['tcl8.6', 'tk8.6']") + (("incdirs = \\[\\]") + (string-append + "incdirs = ['" + #$(this-package-input "tcl") "/include', '" + #$(this-package-input "tk") "/include']"))) + (substitute* '("lib_pypy/_curses_build.py") + ;; Find curses + (("/usr/local") + #$(this-package-input "ncurses"))) + (substitute* '("lib_pypy/_dbm.py") + ;; Use gdbm compat library, so we don’t need to pull + ;; in bdb. + (("ctypes.util.find_library\\('db'\\)") + (format #f "~s" (search-input-file + inputs "lib/libgdbm_compat.so")))) + (substitute* '("lib_pypy/_sqlite3_build.py") + ;; Always use search paths + (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") + ;; Find sqlite3 + (("/usr/local") (assoc-ref inputs "sqlite")) + (("libname = 'sqlite3'") + (format #f "libname = ~s" + (search-input-file inputs "lib/libsqlite3.so.0")))) + (substitute* '("lib-python/3/subprocess.py") + ;; Fix shell path + (("/bin/sh") + (search-input-file inputs "/bin/sh"))) + (substitute* '("lib-python/3/distutils/unixccompiler.py") + ;; gcc-toolchain does not provide symlink cc -> gcc + (("\"cc\"") "\"gcc\"")))) + (add-after 'unpack 'set-source-file-times-to-1980 + ;; copied from python package, required by zip testcase + (lambda _ + (let ((circa-1980 (* 10 366 24 60 60))) + (ftw "." (lambda (file stat flag) + (utime file circa-1980 circa-1980) + #t))))) + (replace 'build + (lambda _ + (with-directory-excursion "pypy/goal" + ;; Build with jit optimization. + (invoke "python2" + "../../rpython/bin/rpython" + (string-append "--make-jobs=" + (number->string (parallel-job-count))) + "-Ojit" + "targetpypystandalone" + "--allworkingmodules")) + ;; Build c modules and package everything, so tests work. + (with-directory-excursion "pypy/tool/release" + (invoke "python2" "package.py" + "--archive-name" "pypy-dist" + "--builddir" (getcwd))))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (if tests? + (begin + (setenv "HOME" "/tmp") ; test_with_pip tries to + ; access ~/.cache/pip + ;; Run library tests only (no interpreter unit tests). + ;; This is what Gentoo does. + (invoke "python" "pypy/test_all.py" + "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" + "lib-python")) + (format #t "test suite not run~%")))) + (replace 'install + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) + (shebang-match-python "#!.+/bin/python") + (shebang-pypy3 (string-append "#!" bin-pypy3)) + (dist-dir "pypy/tool/release/pypy-dist")) + (with-directory-excursion dist-dir + ;; Delete test data. + (for-each + (lambda (x) + (delete-file-recursively (string-append + "lib-python/3/" x))) + '("tkinter/test" + "test" + "sqlite3/test" + "lib2to3/tests" + "idlelib/idle_test" + "distutils/tests" + "ctypes/test" + "unittest/test")) + ;; Patch shebang referencing python. + (substitute* '("lib-python/3/cgi.py" + "lib-python/3/encodings/rot_13.py") + ((shebang-match-python) shebang-pypy3)) + (with-fluids ((%default-port-encoding "ISO-8859-1")) + (substitute* '("lib_pypy/_md5.py" + "lib_pypy/_sha1.py") + ((shebang-match-python) shebang-pypy3)))) + (copy-recursively dist-dir #$output))))))) + (native-inputs + (list gzip + nss-certs ; For ssl tests + pkg-config + python-2 + python2-pycparser + tar)) ; Required for package.py + (inputs + (list bzip2 + expat + gdbm + glibc + libffi + ncurses + openssl + sqlite + tcl + tk + xz + zlib)) + (home-page "https://www.pypy.org/") + (synopsis "Python implementation with just-in-time compilation") + (description "PyPy is a faster, alternative implementation of the Python +programming language employing a just-in-time compiler. It supports most +Python code natively, including C extensions.") + (license (list license:expat ; pypy itself; _pytest/ + license:psfl ; python standard library in lib-python/ + license:asl2.0 ; dotviewer/font/ and some of lib-python/ + license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* + license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ + (license:non-copyleft + "http://www.unicode.org/copyright.html"))))) + +(define-public pypy3 + (deprecated-package "pypy3" pypy)) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index e31158dfc0..00c3eb7774 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -81,17 +81,13 @@ (define-module (gnu packages python) #:use-module (gnu packages) #:use-module (gnu packages base) #:use-module (gnu packages bash) - #:use-module (gnu packages certs) #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages dbm) #:use-module (gnu packages hurd) #:use-module (gnu packages libffi) - #:use-module (gnu packages ncurses) #:use-module (gnu packages pkg-config) - #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) - #:use-module (gnu packages shells) #:use-module (gnu packages sqlite) #:use-module (gnu packages tcl) #:use-module (gnu packages tls) @@ -736,169 +732,3 @@ (define-public micropython compatible with normal Python as possible to allow you to transfer code with ease from the desktop to a microcontroller or embedded system.") (license license:expat))) - -(define-public pypy - (package - (name "pypy") - (version "7.3.5") - (source (origin - (method url-fetch) - (uri (string-append "https://downloads.python.org/pypy/" - "pypy3.7-v" version "-src.tar.bz2")) - (sha256 - (base32 - "18lrdmpcczlbk3cfarkgwqdmilrybz56i1dafk8dkjlyk90gw86r")))) - (build-system gnu-build-system) - (arguments - (list - #:tests? #f ;FIXME: 43 out of 364 tests are failing - #:modules '((ice-9 ftw) (ice-9 match) - (guix build utils) (guix build gnu-build-system)) - #:disallowed-references (list nss-certs) - #:phases - #~(modify-phases %standard-phases - (delete 'configure) - (add-after 'unpack 'patch-source - (lambda* (#:key inputs #:allow-other-keys) - (substitute* '("rpython/rlib/clibffi.py") - ;; find_library does not work for libc - (("ctypes\\.util\\.find_library\\('c'\\)") "'libc.so'")) - (substitute* '("lib_pypy/cffi/_pycparser/ply/cpp.py") - ;; Make reproducible (XXX: unused?) - (("time\\.localtime\\(\\)") "time.gmtime(0)")) - (substitute* '("pypy/module/sys/version.py") - ;; Make reproducible - (("t\\.gmtime\\(\\)") "t.gmtime(0)")) - (substitute* '("lib_pypy/_tkinter/tklib_build.py") - ;; Link to versioned libtcl and libtk - (("linklibs = \\['tcl', 'tk'\\]") - "linklibs = ['tcl8.6', 'tk8.6']") - (("incdirs = \\[\\]") - (string-append - "incdirs = ['" - #$(this-package-input "tcl") "/include', '" - #$(this-package-input "tk") "/include']"))) - (substitute* '("lib_pypy/_curses_build.py") - ;; Find curses - (("/usr/local") - #$(this-package-input "ncurses"))) - (substitute* '("lib_pypy/_dbm.py") - ;; Use gdbm compat library, so we don’t need to pull - ;; in bdb. - (("ctypes.util.find_library\\('db'\\)") - (format #f "~s" (search-input-file - inputs "lib/libgdbm_compat.so")))) - (substitute* '("lib_pypy/_sqlite3_build.py") - ;; Always use search paths - (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") - ;; Find sqlite3 - (("/usr/local") (assoc-ref inputs "sqlite")) - (("libname = 'sqlite3'") - (format #f "libname = ~s" - (search-input-file inputs "lib/libsqlite3.so.0")))) - (substitute* '("lib-python/3/subprocess.py") - ;; Fix shell path - (("/bin/sh") - (search-input-file inputs "/bin/sh"))) - (substitute* '("lib-python/3/distutils/unixccompiler.py") - ;; gcc-toolchain does not provide symlink cc -> gcc - (("\"cc\"") "\"gcc\"")))) - (add-after 'unpack 'set-source-file-times-to-1980 - ;; copied from python package, required by zip testcase - (lambda _ - (let ((circa-1980 (* 10 366 24 60 60))) - (ftw "." (lambda (file stat flag) - (utime file circa-1980 circa-1980) - #t))))) - (replace 'build - (lambda _ - (with-directory-excursion "pypy/goal" - ;; Build with jit optimization. - (invoke "python2" - "../../rpython/bin/rpython" - (string-append "--make-jobs=" - (number->string (parallel-job-count))) - "-Ojit" - "targetpypystandalone" - "--allworkingmodules")) - ;; Build c modules and package everything, so tests work. - (with-directory-excursion "pypy/tool/release" - (invoke "python2" "package.py" - "--archive-name" "pypy-dist" - "--builddir" (getcwd))))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (if tests? - (begin - (setenv "HOME" "/tmp") ; test_with_pip tries to - ; access ~/.cache/pip - ;; Run library tests only (no interpreter unit tests). - ;; This is what Gentoo does. - (invoke "python" "pypy/test_all.py" - "--pypy=pypy/tool/release/pypy-dist/bin/pypy3" - "lib-python")) - (format #t "test suite not run~%")))) - (replace 'install - (lambda* (#:key inputs outputs #:allow-other-keys) - (let* ((bin-pypy3 (string-append #$output "/bin/pypy3")) - (shebang-match-python "#!.+/bin/python") - (shebang-pypy3 (string-append "#!" bin-pypy3)) - (dist-dir "pypy/tool/release/pypy-dist")) - (with-directory-excursion dist-dir - ;; Delete test data. - (for-each - (lambda (x) - (delete-file-recursively (string-append - "lib-python/3/" x))) - '("tkinter/test" - "test" - "sqlite3/test" - "lib2to3/tests" - "idlelib/idle_test" - "distutils/tests" - "ctypes/test" - "unittest/test")) - ;; Patch shebang referencing python. - (substitute* '("lib-python/3/cgi.py" - "lib-python/3/encodings/rot_13.py") - ((shebang-match-python) shebang-pypy3)) - (with-fluids ((%default-port-encoding "ISO-8859-1")) - (substitute* '("lib_pypy/_md5.py" - "lib_pypy/_sha1.py") - ((shebang-match-python) shebang-pypy3)))) - (copy-recursively dist-dir #$output))))))) - (native-inputs - (list gzip - nss-certs ; For ssl tests - pkg-config - python-2 - python2-pycparser - tar)) ; Required for package.py - (inputs - (list bzip2 - expat - gdbm - glibc - libffi - ncurses - openssl - sqlite - tcl - tk - xz - zlib)) - (home-page "https://www.pypy.org/") - (synopsis "Python implementation with just-in-time compilation") - (description "PyPy is a faster, alternative implementation of the Python -programming language employing a just-in-time compiler. It supports most -Python code natively, including C extensions.") - (license (list license:expat ; pypy itself; _pytest/ - license:psfl ; python standard library in lib-python/ - license:asl2.0 ; dotviewer/font/ and some of lib-python/ - license:gpl3+ ; ./rpython/rlib/rvmprof/src/shared/libbacktrace/dwarf2.* - license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ - (license:non-copyleft - "http://www.unicode.org/copyright.html"))))) - -(define-public pypy3 - (deprecated-package "pypy3" pypy)) -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 03/22] gnu: python-pip: Move to (gnu packages python-build). 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 02/22] gnu: pypy: Move to separate module Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 04/22] build-system: Add pyproject-build-system Marius Bakke ` (18 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-pip): Move from here ... * gnu/packages/python-build.scm (python-pip): ... to here. --- gnu/packages/python-build.scm | 21 +++++++++++++++++++++ gnu/packages/python-xyz.scm | 21 --------------------- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 3ff988820f..c8f57ea286 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -41,6 +41,27 @@ (define-module (gnu packages python-build) ;;; ;;; Code: +(define-public python-pip + (package + (name "python-pip") + (version "22.2.2") + (source + (origin + (method url-fetch) + (uri (pypi-uri "pip" version)) + (sha256 + (base32 + "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) + (build-system python-build-system) + (arguments + '(#:tests? #f)) ; there are no tests in the pypi archive. + (home-page "https://pip.pypa.io/") + (synopsis "Package manager for Python software") + (description + "Pip is a package manager for Python software, that finds packages on the +Python Package Index (PyPI).") + (license license:expat))) + (define-public python-setuptools (package (name "python-setuptools") diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 3ea29ec3b9..1ded6f0384 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -12650,27 +12650,6 @@ (define-public python-pretend responses, rather than doing any computation.") (license license:bsd-3))) -(define-public python-pip - (package - (name "python-pip") - (version "22.2.2") - (source - (origin - (method url-fetch) - (uri (pypi-uri "pip" version)) - (sha256 - (base32 - "0jwac0bhfp48w4fqibf1ysrs2grksdv92hwqm7bmdw2jn2fr5l9z")))) - (build-system python-build-system) - (arguments - '(#:tests? #f)) ; there are no tests in the pypi archive. - (home-page "https://pip.pypa.io/") - (synopsis "Package manager for Python software") - (description - "Pip is a package manager for Python software, that finds packages on the -Python Package Index (PyPI).") - (license license:expat))) - ;;; Variant used to break a dependency cycle with ;;; python-pytest-perf-bootstrap. (define-public python-pip-run-bootstrap -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 04/22] build-system: Add pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 02/22] gnu: pypy: Move to separate module Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke ` (17 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587; +Cc: Lars-Dominik Braun From: Lars-Dominik Braun <lars@6xq.net> This is an experimental build system based on python-build-system that implements PEP 517-compliant builds. * doc/guix.texi (Build Systems): Add pyproject-build-system section. * doc/contributing.texi (Python Modules): Mention pyproject.toml and the PYTHON-TOOLCHAIN package, as well as differences to python-build-system. * guix/build-system/pyproject.scm, guix/build/pyproject-build-system.scm, gnu/packages/aux-files/python/sanity-check-next.py, gnu/packages/python-commencement.scm: New files. * Makefile.am (MODULES): Register the new build systems. * gnu/local.mk (GNU_SYSTEM_MODULES): Add python-commencement.scm. * gnu/packages/python.scm (python-sans-pip, python-sans-pip-wrapper): New variables. Co-authored-by: Marius Bakke <marius@gnu.org> --- Makefile.am | 2 + doc/contributing.texi | 36 +- doc/guix.texi | 29 ++ .../aux-files/python/sanity-check-next.py | 98 +++++ gnu/packages/python.scm | 52 ++- guix/build-system/pyproject.scm | 147 +++++++ guix/build/pyproject-build-system.scm | 367 ++++++++++++++++++ 7 files changed, 722 insertions(+), 9 deletions(-) create mode 100644 gnu/packages/aux-files/python/sanity-check-next.py create mode 100644 guix/build-system/pyproject.scm create mode 100644 guix/build/pyproject-build-system.scm diff --git a/Makefile.am b/Makefile.am index 22dcc43f99..6ccb790c11 100644 --- a/Makefile.am +++ b/Makefile.am @@ -166,6 +166,7 @@ MODULES = \ guix/build-system/maven.scm \ guix/build-system/node.scm \ guix/build-system/perl.scm \ + guix/build-system/pyproject.scm \ guix/build-system/python.scm \ guix/build-system/renpy.scm \ guix/build-system/ocaml.scm \ @@ -222,6 +223,7 @@ MODULES = \ guix/build/minetest-build-system.scm \ guix/build/node-build-system.scm \ guix/build/perl-build-system.scm \ + guix/build/pyproject-build-system.scm \ guix/build/python-build-system.scm \ guix/build/ocaml-build-system.scm \ guix/build/qt-build-system.scm \ diff --git a/doc/contributing.texi b/doc/contributing.texi index 4b1eed1cb1..c3221d23e4 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -786,12 +786,29 @@ for instance, the module python-dateutil is packaged under the names starts with @code{py} (e.g.@: @code{pytz}), we keep it and prefix it as described above. +@quotation Note +Currently there are two different build systems for Python packages in Guix: +@var{python-build-system} and @var{pyproject-build-system}. For the +longest time, Python packages were built from an informally specified +@file{setup.py} file. That worked amazingly well, considering Python's +success, but was difficult to build tooling around. As a result, a host +of alternative build systems emerged and the community eventually settled on a +@url{https://peps.python.org/pep-0517/, formal standard} for specifying build +requirements. @var{pyproject-build-system} is Guix's implementation of this +standard. It is considered ``experimental'' in that it does not yet support +all the various PEP-517 @emph{build backends}, but you are encouraged to try +it for new Python packages and report any problems. It will eventually be +deprecated and merged into @var{python-build-system}. +@end quotation + @subsubsection Specifying Dependencies @cindex inputs, for Python packages Dependency information for Python packages is usually available in the package source tree, with varying degrees of accuracy: in the -@file{setup.py} file, in @file{requirements.txt}, or in @file{tox.ini}. +@file{pyproject.toml} file, the @file{setup.py} file, in +@file{requirements.txt}, or in @file{tox.ini} (the latter mostly for +test dependencies). Your mission, when writing a recipe for a Python package, is to map these dependencies to the appropriate type of ``input'' (@pxref{package @@ -802,10 +819,12 @@ following check list to determine which dependency goes where. @itemize @item -We currently package Python 2 with @code{setuptools} and @code{pip} -installed like Python 3.4 has per default. Thus you don't need to -specify either of these as an input. @command{guix lint} will warn you -if you do. +We currently package Python with @code{setuptools} and @code{pip} +installed per default. This is about to change, and users are encouraged +to use @code{python-toolchain} if they want a build environment for Python. + +@command{guix lint} will warn if @code{setuptools} or @code{pip} are +added as native-inputs because they are generally not necessary. @item Python dependencies required at run time go into @@ -814,9 +833,10 @@ Python dependencies required at run time go into @file{requirements.txt} file. @item -Python packages required only at build time---e.g., those listed with -the @code{setup_requires} keyword in @file{setup.py}---or only for -testing---e.g., those in @code{tests_require}---go into +Python packages required only at build time---e.g., those listed under +@code{build-system.requires} in @file{pyproject.toml} or with the +@code{setup_requires} keyword in @file{setup.py}---or dependencies only +for testing---e.g., those in @code{tests_require} or @file{tox.ini}---go into @code{native-inputs}. The rationale is that (1) they do not need to be propagated because they are not needed at run time, and (2) in a cross-compilation context, it's the ``native'' input that we'd want. diff --git a/doc/guix.texi b/doc/guix.texi index f0fb383005..621f9923b4 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9311,7 +9311,36 @@ instead of the default @code{"out"} output. This is useful for packages that include a Python package as only a part of the software, and thus want to combine the phases of @code{python-build-system} with another build system. Python bindings are a common usecase. +@end defvr + +@defvr {Scheme Variable} pyproject-build-system +This is a variable exported by @code{guix build-system pyproject}. It +is based on of @var{python-build-system}, and adds support for +@file{pyproject.toml} and @url{https://peps.python.org/pep-0517/, PEP 517}. +It also supports a variety of build backends and test frameworks. + +The API is slightly different from @var{python-build-system}: +@enumerate +@item +@code{#:use-setuptools?} and @code{#:test-target} is removed. +@item +@code{#:build-backend} is added. It defaults to @code{#false} and will try +to guess the appropriate backend based on @file{pyproject.toml}. +@item +@code{#:test-backend} is added. It defaults to @code{#false} and will guess +an appropriate test backend based on what is available in package inputs. +@item +@code{#:test-flags} is added. The default is @code{#false}, and varies based +on the detected @code{#:test-backend}. +@end enumerate + +It is considered ``experimental'' in that the implementation details are +not set in stone yet, however users are encouraged to try it for new +Python projects (even those using @file{setup.py}). The API is subject to +change, but any breaking changes in the Guix channel will be dealt with. +Eventually this build system will be deprecated and merged back into +@var{python-build-system}, probably some time in 2024. @end defvr @defvr {Scheme Variable} perl-build-system diff --git a/gnu/packages/aux-files/python/sanity-check-next.py b/gnu/packages/aux-files/python/sanity-check-next.py new file mode 100644 index 0000000000..891606f72b --- /dev/null +++ b/gnu/packages/aux-files/python/sanity-check-next.py @@ -0,0 +1,98 @@ +# -*- coding: utf-8 -*- + +# This version adds a small change to accomodate missing python-setuptools. +# Original patch by Lars-Dominik Braun in wip-python-pep517, commit +# 720dbe22d431262938be29dd9a9ddb78c44a99b3. +# --- sanity-check.py 2022-06-12 14:40:06.814337702 +0200 +# +++ sanity-check.py 2022-10-16 23:21:38.990651568 +0200 +# @@ -19,9 +19,13 @@ +# from __future__ import print_function # Python 2 support. +# import importlib +# -import pkg_resources +# import sys +# import traceback +# +try: +# + import pkg_resources +# +except ImportError: +# + print('Warning: Skipping, because python-setuptools are not available.') +# + sys.exit(0) + +# TODO: Merge with sanity-check.py in the next core-updates cycle. + +from __future__ import print_function # Python 2 support. +import importlib +import sys +import traceback +try: + import pkg_resources +except ImportError: + print('Warning: Skipping, because python-setuptools are not available.') + sys.exit(0) + +try: + from importlib.machinery import PathFinder +except ImportError: + PathFinder = None + +ret = 0 + +# Only check site-packages installed by this package, but not dependencies +# (which pkg_resources.working_set would include). Path supplied via argv. +ws = pkg_resources.find_distributions(sys.argv[1]) + +for dist in ws: + print('validating', repr(dist.project_name), dist.location) + try: + print('...checking requirements: ', end='') + req = str(dist.as_requirement()) + # dist.activate() is not enough to actually check requirements, we + # have to .require() it. + pkg_resources.require(req) + print('OK') + except Exception as e: + print('ERROR:', req, repr(e)) + ret = 1 + continue + + # Try to load top level modules. This should not have any side-effects. + try: + metalines = dist.get_metadata_lines('top_level.txt') + except (KeyError, EnvironmentError): + # distutils (i.e. #:use-setuptools? #f) will not install any metadata. + # This file is also missing for packages built using a PEP 517 builder + # such as poetry. + print('WARNING: cannot determine top-level modules') + continue + for name in metalines: + # Only available on Python 3. + if PathFinder and PathFinder.find_spec(name) is None: + # Ignore unavailable modules, often C modules, which were not + # installed at the top-level. Cannot use ModuleNotFoundError, + # because it is raised by failed imports too. + continue + try: + print('...trying to load module', name, end=': ') + importlib.import_module(name) + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + + # Try to load entry points of console scripts too, making sure they + # work. They should be removed if they don't. Other groups may not be + # safe, as they can depend on optional packages. + for group, v in dist.get_entry_map().items(): + if group not in {'console_scripts', 'gui_scripts'}: + continue + for name, ep in v.items(): + try: + print('...trying to load endpoint', group, name, end=': ') + ep.load() + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + +sys.exit(ret) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 00c3eb7774..f55e5a16da 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -26,7 +26,7 @@ ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> ;;; Copyright © 2016 Dylan Jeffers <sapientech@sapientech@openmailbox.org> ;;; Copyright © 2016 David Craven <david@craven.ch> -;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org> +;;; Copyright © 2016-2022 Marius Bakke <marius@gnu.org> ;;; Copyright © 2016, 2017 Stefan Reichör <stefan@xsteve.at> ;;; Copyright © 2016, 2017 Alex Vong <alexvong1995@gmail.com> ;;; Copyright © 2016, 2017, 2018 Arun Isaac <arunisaac@systemreboot.net> @@ -60,6 +60,7 @@ ;;; Copyright © 2020, 2021 Greg Hogan <code@greghogan.com> ;;; Copyright © 2022 Philip McGrath <philip@philipmcgrath.com> ;;; Copyright © 2022 jgart <jgart@dismail.de> +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> ;;; ;;; This file is part of GNU Guix. ;;; @@ -87,6 +88,7 @@ (define-module (gnu packages python) #:use-module (gnu packages hurd) #:use-module (gnu packages libffi) #:use-module (gnu packages pkg-config) + #:use-module (gnu packages python-build) #:use-module (gnu packages readline) #:use-module (gnu packages sqlite) #:use-module (gnu packages tcl) @@ -674,6 +676,54 @@ (define* (wrap-python3 python (define-public python-wrapper (wrap-python3 python)) (define-public python-minimal-wrapper (wrap-python3 python-minimal)) +;; The Python used in pyproject-build-system. +(define-public python-sans-pip + (hidden-package + (package/inherit python + (arguments + (substitute-keyword-arguments (package-arguments python) + ((#:configure-flags flags #~()) + #~(append '("--with-ensurepip=no") + (delete "--with-ensurepip=install" #$flags)))))))) + +(define-public python-sans-pip-wrapper + (wrap-python3 python-sans-pip)) + +(define-public python-toolchain + (package + (name "python-toolchain") + (version (package-version python)) + (source #f) + (build-system trivial-build-system) + (arguments + (list #:modules '((guix build union)) + #:builder + #~(begin + (use-modules (ice-9 match) + (srfi srfi-1) + (guix build union)) + (union-build #$output + (filter-map (match-lambda + ((_ . directory) directory)) + %build-inputs))))) + (inputs + (list python-sans-pip-wrapper + python-pip + python-pypa-build + python-setuptools + python-wheel)) + (native-search-paths + (package-native-search-paths python)) + (search-paths + (package-search-paths python)) + (license (package-license python)) + (synopsis "Python toolchain") + (description + "Python toolchain including Python itself, setuptools and pip. Use this +package if you need a minimal Python toolchain instead of just the +interpreter.") + (home-page (package-home-page python)))) + (define-public micropython (package (name "micropython") diff --git a/guix/build-system/pyproject.scm b/guix/build-system/pyproject.scm new file mode 100644 index 0000000000..1e365d4f21 --- /dev/null +++ b/guix/build-system/pyproject.scm @@ -0,0 +1,147 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build-system pyproject) + #:use-module ((gnu packages) #:select (search-auxiliary-file)) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (guix memoization) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix packages) + #:use-module (guix derivations) + #:use-module (guix search-paths) + #:use-module (guix build-system) + #:use-module (guix build-system gnu) + #:use-module (guix build-system python) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:export (%pyproject-build-system-modules + default-python + pyproject-build + pyproject-build-system)) + +;; Commentary: +;; +;; Standard build procedure for Python packages using 'pyproject.toml'. +;; This is implemented as an extension of 'python-build-system'. +;; +;; Code: + +(define %pyproject-build-system-modules + ;; Build-side modules imported by default. + `((guix build pyproject-build-system) + (guix build json) + ,@%python-build-system-modules)) + +(define (default-python) + "Return the default Python package." + ;; Lazily resolve the binding to avoid a circular dependency. + (let ((python (resolve-interface '(gnu packages python)))) + (module-ref python 'python-toolchain))) + +(define sanity-check.py + ;; TODO: Merge with sanity-check.py in the next rebuild cycle. + (search-auxiliary-file "python/sanity-check-next.py")) + +(define* (lower name + #:key source inputs native-inputs outputs system target + (python (default-python)) + #:allow-other-keys + #:rest arguments) + "Return a bag for NAME." + (define private-keywords + '(#:target #:python #:inputs #:native-inputs)) + + (and (not target) ;XXX: no cross-compilation + (bag + (name name) + (system system) + (host-inputs `(,@(if source + `(("source" ,source)) + '()) + ,@inputs + + ;; Keep the standard inputs of 'gnu-build-system'. + ,@(standard-packages))) + (build-inputs `(("python" ,python) + ("sanity-check.py" ,(local-file sanity-check.py)) + ,@native-inputs)) + (outputs (append outputs '(wheel))) + (build pyproject-build) + (arguments (strip-keyword-arguments private-keywords arguments))))) + +(define* (pyproject-build name inputs + #:key source + (tests? #t) + (configure-flags ''()) + (build-backend #f) + (test-backend #f) + (test-flags #f) + (phases '%standard-phases) + (outputs '("out" "wheel")) + (search-paths '()) + (system (%current-system)) + (guile #f) + (imported-modules %pyproject-build-system-modules) + (modules '((guix build pyproject-build-system) + (guix build utils)))) + "Build SOURCE using PYTHON, and with INPUTS." + (define build + (with-imported-modules imported-modules + #~(begin + (use-modules #$@(sexp->gexp modules)) + + #$(with-build-variables inputs outputs + #~(pyproject-build + #:name #$name + #:source #+source + #:configure-flags #$configure-flags + #:system #$system + #:build-backend #$build-backend + #:test-backend #$test-backend + #:test-flags #$test-flags + #:tests? #$tests? + #:phases #$(if (pair? phases) + (sexp->gexp phases) + phases) + #:outputs %outputs + #:search-paths '#$(sexp->gexp + (map search-path-specification->sexp + search-paths)) + #:inputs %build-inputs))))) + + + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system #:graft? #f))) + (gexp->derivation name build + #:system system + #:graft? #f ;consistent with 'gnu-build' + #:target #f + #:guile-for-build guile))) + +(define pyproject-build-system + (build-system + (name 'pyproject) + (description "The PEP517-compliant Python build system") + (lower lower))) + +;;; pyproject.scm ends here diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm new file mode 100644 index 0000000000..16a6db9537 --- /dev/null +++ b/guix/build/pyproject-build-system.scm @@ -0,0 +1,367 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net> +;;; Copyright © 2022 Marius Bakke <marius@gnu.org> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix build pyproject-build-system) + #:use-module ((guix build python-build-system) #:prefix python:) + #:use-module (guix build utils) + #:use-module (guix build json) + #:use-module (ice-9 match) + #:use-module (ice-9 ftw) + #:use-module (ice-9 format) + #:use-module (ice-9 rdelim) + #:use-module (ice-9 regex) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) + #:export (%standard-phases + add-installed-pythonpath + site-packages + python-version + pyproject-build)) + +;;; Commentary: +;;; +;;; PEP 517-compatible build system for Python packages. +;;; +;;; PEP 517 mandates the use of a TOML file called pyproject.toml at the +;;; project root, describing build and runtime dependencies, as well as the +;;; build system, which can be different from setuptools. This module uses +;;; that file to extract the build system used and call its wheel-building +;;; entry point build_wheel (see 'build). setuptools’ wheel builder is +;;; used as a fallback if either no pyproject.toml exists or it does not +;;; declare a build-system. It supports config_settings through the +;;; standard #:configure-flags argument. +;;; +;;; This wheel, which is just a ZIP file with a file structure defined +;;; by PEP 427 (https://www.python.org/dev/peps/pep-0427/), is then unpacked +;;; and its contents are moved to the appropriate locations in 'install. +;;; +;;; Then entry points, as defined by the PyPa Entry Point Specification +;;; (https://packaging.python.org/specifications/entry-points/) are read +;;; from a file called entry_points.txt in the package’s site-packages +;;; subdirectory and scripts are written to bin/. These are not part of a +;;; wheel and expected to be created by the installing utility. +;;; TODO: Add support for PEP-621 entry points. +;;; +;;; Caveats: +;;; - There is no support for in-tree build backends. +;;; +;;; Code: +;;; + +;; Re-export these variables from python-build-system as many packages +;; rely on these. +(define python-version python:python-version) +(define site-packages python:site-packages) +(define add-installed-pythonpath python:add-installed-pythonpath) + +;; Base error type. +(define-condition-type &python-build-error &error + python-build-error?) + +;; Raised when 'check cannot find a valid test system in the inputs. +(define-condition-type &test-system-not-found &python-build-error + test-system-not-found?) + +;; Raised when multiple wheels are created by 'build. +(define-condition-type &cannot-extract-multiple-wheels &python-build-error + cannot-extract-multiple-wheels?) + +;; Raised, when no wheel has been built by the build system. +(define-condition-type &no-wheels-built &python-build-error + no-wheels-built?) + +(define* (build #:key outputs build-backend configure-flags #:allow-other-keys) + "Build a given Python package." + + (define (pyproject.toml->build-backend file) + "Look up the build backend in a pyproject.toml file." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in 'concat))) + (if (eof-object? line) + #f + (let ((m (string-match "build-backend = [\"'](.+)[\"']" line))) + (if m (match:substring m 1) + (loop (read-line in 'concat))))))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + ;; There is no easy way to get data from Guile into Python via + ;; s-expressions, but we have JSON serialization already, which Python + ;; also supports out-of-the-box. + (config-settings (call-with-output-string (cut write-json configure-flags <>))) + ;; python-setuptools’ default backend supports setup.py *and* + ;; pyproject.toml. Allow overriding this automatic detection via + ;; build-backend. + (auto-build-backend (if (file-exists? "pyproject.toml") + (pyproject.toml->build-backend "pyproject.toml") + #f)) + ;; Use build system detection here and not in importer, because a) we + ;; have alot of legacy packages and b) the importer cannot update arbitrary + ;; fields in case a package switches its build system. + (use-build-backend (or + build-backend + auto-build-backend + "setuptools.build_meta"))) + (format #t "Using '~a' to build wheels, auto-detected '~a', override '~a'.~%" + use-build-backend auto-build-backend build-backend) + (mkdir-p wheel-dir) + ;; Call the PEP 517 build function, which drops a .whl into wheel-dir. + (invoke "python" "-c" "import sys, importlib, json +config_settings = json.loads (sys.argv[3]) +builder = importlib.import_module(sys.argv[1]) +builder.build_wheel(sys.argv[2], config_settings=config_settings)" + use-build-backend wheel-dir config-settings))) + +(define* (check #:key inputs outputs tests? test-backend test-flags #:allow-other-keys) + "Run the test suite of a given Python package." + (if tests? + ;; Unfortunately with PEP 517 there is no common method to specify test + ;; systems. Guess test system based on inputs instead. + (let* ((pytest (which "pytest")) + (nosetests (which "nosetests")) + (nose2 (which "nose2")) + (have-setup-py (file-exists? "setup.py")) + (use-test-backend + (or + test-backend + ;; Prefer pytest + (if pytest 'pytest #f) + (if nosetests 'nose #f) + (if nose2 'nose2 #f) + ;; But fall back to setup.py, which should work for most + ;; packages. XXX: would be nice not to depend on setup.py here? fails + ;; more often than not to find any tests at all. Maybe we can run + ;; `python -m unittest`? + (if have-setup-py 'setup.py #f)))) + (format #t "Using ~a~%" use-test-backend) + (match use-test-backend + ('pytest + (apply invoke (cons pytest (or test-flags '("-vv"))))) + ('nose + (apply invoke (cons nosetests (or test-flags '("-v"))))) + ('nose2 + (apply invoke (cons nose2 (or test-flags '("-v" "--pretty-assert"))))) + ('setup.py + (apply invoke (append '("python" "setup.py") (or test-flags '("test" "-v"))))) + ;; The developer should explicitly disable tests in this case. + (else (raise (condition (&test-system-not-found)))))) + (format #t "test suite not run~%"))) + +(define* (install #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Install a wheel file according to PEP 427" + ;; See https://www.python.org/dev/peps/pep-0427/#installing-a-wheel-distribution-1-0-py32-none-any-whl + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (out (assoc-ref outputs "out"))) + (define (extract file) + "Extract wheel (ZIP file) into site-packages directory" + ;; Use Python’s zipfile to avoid extra dependency + (invoke "python" "-m" "zipfile" "-e" file site-dir)) + + (define python-hashbang + (string-append "#!" python "/bin/python")) + + (define* (merge-directories source destination #:optional (post-move #f)) + "Move all files in SOURCE into DESTINATION, merging the two directories." + (format #t "Merging directory ~a into ~a~%" source destination) + (for-each + (lambda (file) + (format #t "~a/~a -> ~a/~a~%" source file destination file) + (mkdir-p destination) + (rename-file + (string-append source "/" file) + (string-append destination "/" file)) + (when post-move + (post-move file))) + (scandir source (negate (cut member <> '("." ".."))))) + (rmdir source)) + + (define (expand-data-directory directory) + "Move files from all .data subdirectories to their respective +destinations." + ;; Python’s distutils.command.install defines this mapping from source to + ;; destination mapping. + (let ((source (string-append directory "/scripts")) + (destination (string-append out "/bin"))) + (when (file-exists? source) + (merge-directories + source + destination + (lambda (f) + (let ((dest-path (string-append destination "/" f))) + (chmod dest-path #o755) + (substitute* dest-path (("#!python") python-hashbang))))))) + ;; data can create arbitrary directory structures. Most commonly + ;; it is used for share/. + (let ((source (string-append directory "/data")) + (destination out)) + (when (file-exists? source) + (merge-directories source destination))) + (let* ((distribution (car (string-split (basename directory) #\-))) + (source (string-append directory "/headers")) + (destination (string-append out "/include/python" (python-version python) "/" distribution))) + (when (file-exists? source) + (merge-directories source destination)))) + + (define (list-directories base predicate) + ;; Cannot use find-files here, because it’s recursive. + (scandir + base + (lambda (name) + (let ((stat (lstat (string-append base "/" name)))) + (and + (not (member name '("." ".."))) + (eq? (stat:type stat) 'directory) + (predicate name stat)))))) + + (let* ((wheel-output (assoc-ref outputs "wheel")) + (wheel-dir (if wheel-output wheel-output "dist")) + (wheels (map (cut string-append wheel-dir "/" <>) + (scandir wheel-dir (cut string-suffix? ".whl" <>))))) + (cond + ((> (length wheels) 1) ; This code does not support multiple wheels + ; yet, because their outputs would have to be + ; merged properly. + (raise (condition (&cannot-extract-multiple-wheels)))) + ((= (length wheels) 0) + (raise (condition (&no-wheels-built))))) + (for-each extract wheels)) + (let ((datadirs (map + (cut string-append site-dir "/" <>) + (list-directories site-dir (file-name-predicate "\\.data$"))))) + (for-each (lambda (directory) + (expand-data-directory directory) + (rmdir directory)) + datadirs)))) + +(define* (compile-bytecode #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Compile installed byte-code in site-packages." + (let* ((site-dir (site-packages inputs outputs)) + (python (assoc-ref inputs "python")) + (major-minor (map string->number + (take (string-split (python-version python) #\.) 2))) + (<3.7? (match major-minor + ((major minor) + (or (< major 3) (and (= major 3) (< minor 7))))))) + (if <3.7? + ;; These versions don’t have the hash invalidation modes and do + ;; not produce reproducible bytecode files. + (format #t "Skipping bytecode compilation for Python version ~a < 3.7~%" + (python-version python)) + (invoke "python" "-m" "compileall" "--invalidation-mode=unchecked-hash" + site-dir)))) + +(define* (create-entrypoints #:key inputs outputs (configure-flags '()) #:allow-other-keys) + "Implement Entry Points Specification +(https://packaging.python.org/specifications/entry-points/) by PyPa, +which creates runnable scripts in bin/ from entry point specification +file entry_points.txt. This is necessary, because wheels do not contain +these binaries and installers are expected to create them." + + (define (entry-points.txt->entry-points file) + "Specialized parser for Python configfile-like files, in particular +entry_points.txt. Returns a list of console_script and gui_scripts +entry points." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in)) + (inside #f) + (result '())) + (if (eof-object? line) + result + (let* ((group-match (string-match "^\\[(.+)\\]$" line)) + (group-name (if group-match (match:substring group-match 1) #f)) + (next-inside + (if (not group-name) + inside + (or + (string=? group-name "console_scripts") + (string=? group-name "gui_scripts")))) + (item-match (string-match "^([^ =]+)\\s*=\\s*([^:]+):(.+)$" line))) + (if (and inside item-match) + (loop (read-line in) next-inside (cons (list + (match:substring item-match 1) + (match:substring item-match 2) + (match:substring item-match 3)) + result)) + (loop (read-line in) next-inside result)))))))) + + (define (create-script path name module function) + "Create a Python script from an entry point’s NAME, MODULE and + FUNCTION and return write it to PATH/NAME." + (let ((interpreter (which "python")) + (file-path (string-append path "/" name))) + (format #t "Creating entry point for '~a.~a' at '~a'.~%" module function + file-path) + (call-with-output-file file-path + (lambda (port) + ;; Technically the script could also include search-paths, + ;; but having a generic 'wrap phases also handles manually + ;; written entry point scripts. + (format port "#!~a +# Auto-generated entry point script. +import sys +import ~a as mod +sys.exit (mod.~a ())~%" interpreter module function))) + (chmod file-path #o755))) + + (let* ((site-dir (site-packages inputs outputs)) + (out (assoc-ref outputs "out")) + (bin-dir (string-append out "/bin")) + (entry-point-files (find-files site-dir "^entry_points.txt$"))) + (mkdir-p bin-dir) + (for-each + (lambda (f) + (for-each + (lambda (ep) (apply create-script (cons bin-dir ep))) + (entry-points.txt->entry-points f))) + entry-point-files))) + +(define* (set-SOURCE-DATE-EPOCH #:rest _) + "Set the 'SOURCE_DATE_EPOCH' environment variable. This is used by tools +that incorporate timestamps as a way to tell them to use a fixed timestamp. +See https://reproducible-builds.org/specs/source-date-epoch/." + ;; Use a post-1980 timestamp because the Zip format used in wheels do + ;; not support timestamps before 1980. + (setenv "SOURCE_DATE_EPOCH" "315619200")) + +(define %standard-phases + ;; The build phase only builds C extensions and copies the Python sources, + ;; while the install phase copies then byte-compiles the sources to the + ;; prefix directory. The check phase is moved after the installation phase + ;; to ease testing the built package. + (modify-phases python:%standard-phases + (replace 'set-SOURCE-DATE-EPOCH set-SOURCE-DATE-EPOCH) + (replace 'build build) + (replace 'install install) + (delete 'check) + ;; Must be before tests, so they can use installed packages’ entry points. + (add-before 'wrap 'create-entrypoints create-entrypoints) + (add-after 'wrap 'check check) + (add-before 'check 'compile-bytecode compile-bytecode))) + +(define* (pyproject-build #:key inputs (phases %standard-phases) + #:allow-other-keys #:rest args) + "Build the given Python package, applying all of PHASES in order." + (apply python:python-build #:inputs inputs #:phases phases args)) + +;;; pyproject-build-system.scm ends here -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system'. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (2 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 04/22] build-system: Add pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-24 16:36 ` pelzflorian (Florian Pelz) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 06/22] gnu: python-autopage: Use pyproject-build-system Marius Bakke ` (16 subsequent siblings) 20 siblings, 1 reply; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * etc/news.scm: Add entry. --- etc/news.scm | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/etc/news.scm b/etc/news.scm index d9ffa036a2..c2ac842947 100644 --- a/etc/news.scm +++ b/etc/news.scm @@ -6,7 +6,7 @@ ;; Copyright © 2019, 2020 Konrad Hinsen <konrad.hinsen@fastmail.net> ;; Copyright © 2019, 2020, 2021 Julien Lepiller <julien@lepiller.eu> ;; Copyright © 2019–2022 Florian Pelz <pelzflorian@pelzflorian.de> -;; Copyright © 2020 Marius Bakke <mbakke@fastmail.com> +;; Copyright © 2020, 2022 Marius Bakke <marius@gnu.org> ;; Copyright © 2020, 2021 Mathieu Othacehe <m.othacehe@gmail.com> ;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;; Copyright © 2020, 2021, 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com> @@ -26,6 +26,26 @@ (channel-news (version 0) + (entry (commit "3f82a8b2d075f5980ffa8cdd9c49a5a8586e5b1b") + (title + (en "New build system for Python packages")) + (body + (en "A new @var{pyproject-build-system} has been added. This +is a redesign of @var{python-build-system} with support for @dfn{PEP 517} +and @file{pyproject.toml} files. It also has built-in support for various +test frameworks such as @command{pytest} and @code{nosetests}. + +There is a complementary @code{python-toolchain} package that comes with +updated versions of @command{pip}, @command{setuptools} and others. + +The build system will eventually be merged into @var{python-build-system} +but you are encouraged to use it for packages in the @code{guix} channel. +Third party channels may want to wait until the API is stable (see the +Guix manual for caveats). + +Despite the name, @var{pyproject-build-system} also works with the +``legacy'' @file{setup.py} format."))) + (entry (commit "c7ba5f38b80433b040d3946b8fc0b1e8621ba30a") (title (en "New @option{--emulate-fhs} option for @command{guix shell}") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system'. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke @ 2022-10-24 16:36 ` pelzflorian (Florian Pelz) 2022-10-24 18:01 ` Julien Lepiller 0 siblings, 1 reply; 65+ messages in thread From: pelzflorian (Florian Pelz) @ 2022-10-24 16:36 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587 Thank you for improving build systems. Could you add a German translation of the news: Marius Bakke <marius@gnu.org> writes: > + (entry (commit "3f82a8b2d075f5980ffa8cdd9c49a5a8586e5b1b") > + (title > + (en "New build system for Python packages")) (de "Neues Erstellungssystem für Python-Pakete")) > + (body > + (en "A new @var{pyproject-build-system} has been added. This > +is a redesign of @var{python-build-system} with support for @dfn{PEP 517} > +and @file{pyproject.toml} files. It also has built-in support for various > +test frameworks such as @command{pytest} and @code{nosetests}. > + > +There is a complementary @code{python-toolchain} package that comes with > +updated versions of @command{pip}, @command{setuptools} and others. > + > +The build system will eventually be merged into @var{python-build-system} > +but you are encouraged to use it for packages in the @code{guix} channel. > +Third party channels may want to wait until the API is stable (see the > +Guix manual for caveats). > + > +Despite the name, @var{pyproject-build-system} also works with the > +``legacy'' @file{setup.py} format."))) ``legacy'' @file{setup.py} format.") (de "Ein neues Erstellungssystem @var{pyproject-build-system} ist verfügbar. Es ist eine Neuauflage des @var{python-build-system}, die @dfn{PEP 517} und @file{pyproject.toml}-Dateien unterstützt. Auch wurde Unterstützung für Testrahmen wie @command{pytest} und @code{nosetests} eingebaut. Ergänzend gibt es ein Paket @code{python-toolchain} mit aktualisierten Versionen von @command{pip}, @command{setuptools} und mehr. Das Erstellungssystem wird in Zukunft Teil von @var{python-build-system} werden, aber wir würden es begrüßen, wenn Sie es für Pakete auf dem @code{guix}-Kanal verwenden würden. Drittanbieterkanäle warten vielleicht lieber auf eine stabile Programmierschnittstelle (siehe die im Guix-Handbuch genannten Einschränkungen). Trotz dem Namen funktioniert @var{pyproject-build-system} auch mit dem „alten“ @file{setup.py}-Format."))) == Regards, Florian ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system'. 2022-10-24 16:36 ` pelzflorian (Florian Pelz) @ 2022-10-24 18:01 ` Julien Lepiller 0 siblings, 0 replies; 65+ messages in thread From: Julien Lepiller @ 2022-10-24 18:01 UTC (permalink / raw) To: 58587, pelzflorian, marius [-- Attachment #1: Type: text/plain, Size: 3302 bytes --] And here's the French version: (fr "Nouveau système de construction pour les paquets Python") (fr "Un nouveau système de construction, @var{pyproject-build-system}, a été ajouté. Il s'agit d'une refonte du @var{python-build-system} qui rajoute la prise en charge de @dfn{PEP 517} et des fichiers @file{pyproject.toml}. Il intègre aussi la prise en charge de divers cadriciels de test comme @command{pytest} ou @code{nosetests}. Un paquet supplémentaire @code{python-toolchain} fournit des versions à jour de @command{pip}, @command{setuptools} et autres. Le système de construction finira par être intégré au @var{python-build-system} mais nous vous encourageons à l'utiliser pour les paquets du canal @code{guix}. Les canaux tiers devraient attendre que l'API se stabilise (voir le manuel de Guix pour les mises en garde). Contrairement à ce qu'indique son nom, @var{pyproject-build-system} fonctionne aussi avec « l'ancien » format @file{setup.py}.") Le 24 octobre 2022 18:36:56 GMT+02:00, "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> a écrit : >Thank you for improving build systems. Could you add a German >translation of the news: > >Marius Bakke <marius@gnu.org> writes: >> + (entry (commit "3f82a8b2d075f5980ffa8cdd9c49a5a8586e5b1b") >> + (title >> + (en "New build system for Python packages")) > >(de "Neues Erstellungssystem für Python-Pakete")) > >> + (body >> + (en "A new @var{pyproject-build-system} has been added. This >> +is a redesign of @var{python-build-system} with support for @dfn{PEP 517} >> +and @file{pyproject.toml} files. It also has built-in support for various >> +test frameworks such as @command{pytest} and @code{nosetests}. >> + >> +There is a complementary @code{python-toolchain} package that comes with >> +updated versions of @command{pip}, @command{setuptools} and others. >> + >> +The build system will eventually be merged into @var{python-build-system} >> +but you are encouraged to use it for packages in the @code{guix} channel. >> +Third party channels may want to wait until the API is stable (see the >> +Guix manual for caveats). >> + >> +Despite the name, @var{pyproject-build-system} also works with the >> +``legacy'' @file{setup.py} format."))) >``legacy'' @file{setup.py} format.") > (de "Ein neues Erstellungssystem @var{pyproject-build-system} ist >verfügbar. Es ist eine Neuauflage des @var{python-build-system}, die >@dfn{PEP 517} und @file{pyproject.toml}-Dateien unterstützt. Auch wurde >Unterstützung für Testrahmen wie @command{pytest} und @code{nosetests} >eingebaut. > >Ergänzend gibt es ein Paket @code{python-toolchain} mit aktualisierten Versionen >von @command{pip}, @command{setuptools} und mehr. > >Das Erstellungssystem wird in Zukunft Teil von @var{python-build-system} werden, >aber wir würden es begrüßen, wenn Sie es für Pakete auf dem @code{guix}-Kanal >verwenden würden. Drittanbieterkanäle warten vielleicht lieber auf eine >stabile Programmierschnittstelle (siehe die im Guix-Handbuch genannten >Einschränkungen). > >Trotz dem Namen funktioniert @var{pyproject-build-system} auch mit dem „alten“ >@file{setup.py}-Format."))) > >== > >Regards, >Florian > > > [-- Attachment #2: Type: text/html, Size: 3865 bytes --] ^ permalink raw reply [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 06/22] gnu: python-autopage: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (3 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 07/22] gnu: flair: Switch to pyproject-build-system Marius Bakke ` (15 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-autopage)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. --- gnu/packages/python-xyz.scm | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 1ded6f0384..42935cbaaa 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -259,6 +259,7 @@ (define-module (gnu packages python-xyz) #:use-module (guix utils) #:use-module (guix build-system gnu) #:use-module (guix build-system cmake) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system trivial) #:use-module (srfi srfi-1) @@ -6959,33 +6960,16 @@ (define-public python-autopage (sha256 (base32 "169ixll1ncm2a2pcc86665ikjv2lrzs10p6c1w4yj55p3gk3xgh1")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; Do a manual PEP 517 style build/install procedure until the - ;; python-build-system overhaul is merged. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (let ((circa-1980 (* 10 366 24 60 60))) - (setenv "SOURCE_DATE_EPOCH" (number->string circa-1980)) - (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))) (add-before 'check 'disable-e2e-tests (lambda _ ;; These tests rely on KeyboardInterrupts which do not ;; work in the build container. - (delete-file "autopage/tests/test_end_to_end.py"))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + (delete-file "autopage/tests/test_end_to_end.py")))))) (native-inputs (list python-pypa-build python-setuptools -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 07/22] gnu: flair: Switch to pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (4 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 06/22] gnu: python-autopage: Use pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 08/22] gnu: python-pydyf: Use pyproject-build-system Marius Bakke ` (14 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/bioinformatics.scm (flair)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove #:phases. [native-inputs]: Remove. --- gnu/packages/bioinformatics.scm | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index f2de09aa32..3588f4e3fd 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -56,6 +56,7 @@ (define-module (gnu packages bioinformatics) #:use-module (guix build-system meson) #:use-module (guix build-system ocaml) #:use-module (guix build-system perl) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system qt) #:use-module (guix build-system r) @@ -16928,33 +16929,15 @@ (define-public flair (sha256 (base32 "106swb2q7l20ki58fca1hg95q5f79bgp9gjb0clr2243ycrzyxf8")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - (list - #:tests? #false ;there are none - #:phases - #~(modify-phases %standard-phases - ;; TODO: implement as a feature of python-build-system (PEP-621, - ;; PEP-631, PEP-660) - (replace 'build - (lambda _ - (setenv "SETUPTOOLS_SCM_PRETEND_VERSION" #$version) - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (apply invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output - (find-files "dist" "\\.whl$"))))))) + (list #:tests? #false)) ;there are none (propagated-inputs (list python-mappy python-ncls python-pybedtools python-pysam python-tqdm)) - (native-inputs - (list python-pypa-build python-setuptools)) (home-page "https://flair.readthedocs.io/en/latest/") (synopsis "Full-length alternative isoform analysis of RNA") (description "This package implements FLAIR (Full-Length Alternative -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 08/22] gnu: python-pydyf: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (5 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 07/22] gnu: flair: Switch to pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 09/22] gnu: weasyprint: " Marius Bakke ` (13 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/pdf.scm (python-pydyf)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove #:phases. Add #:test-flags. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/pdf.scm | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/gnu/packages/pdf.scm b/gnu/packages/pdf.scm index b49e5264f9..ec75bca6e3 100644 --- a/gnu/packages/pdf.scm +++ b/gnu/packages/pdf.scm @@ -49,6 +49,7 @@ (define-module (gnu packages pdf) #:use-module (guix build-system gnu) #:use-module (guix build-system cmake) #:use-module (guix build-system meson) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system qt) #:use-module (guix build-system trivial) @@ -735,28 +736,14 @@ (define-public python-pydyf (uri (pypi-uri "pydyf" version)) (sha256 (base32 "18q43g5d9455msipcgd5fvnh8m4a2rz189slzfg80yycjw66rshs")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list - #:phases - #~(modify-phases %standard-phases - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "-c" "/dev/null"))))))) + #:test-flags #~'("-vv" "-c" "/dev/null"))) (propagated-inputs (list python-pillow)) (native-inputs (list ghostscript python-flit-core - python-pypa-build python-pytest)) (home-page "https://github.com/CourtBouillon/pydyf") (synopsis "Low-level PDF generator") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 09/22] gnu: weasyprint: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (6 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 08/22] gnu: python-pydyf: Use pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 10/22] gnu: python-glyphslib: " Marius Bakke ` (12 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/pdf.scm (weasyprint)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. Add #:test-flags. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/pdf.scm | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/gnu/packages/pdf.scm b/gnu/packages/pdf.scm index ec75bca6e3..8c871e4341 100644 --- a/gnu/packages/pdf.scm +++ b/gnu/packages/pdf.scm @@ -1497,9 +1497,11 @@ (define-public weasyprint (sha256 (base32 "0cn8gpgyic6pmrnhp0540nbgplpsd5aybi7k89anz6m1sshgjzgs")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list + #:test-flags #~(list "-vv" "-c" "/dev/null" + "-n" (number->string (parallel-job-count))) #:phases #~(modify-phases %standard-phases (add-after 'unpack 'patch-library-paths @@ -1520,21 +1522,7 @@ (define-public weasyprint (("'pangoft2-1.0-0'") (format #f "~s" (search-input-file inputs - "lib/libpangoft2-1.0.so")))))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "-c" "/dev/null" - "-n" (number->string (parallel-job-count)))))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + "lib/libpangoft2-1.0.so"))))))))) (inputs (list fontconfig glib harfbuzz pango)) (propagated-inputs (list gdk-pixbuf @@ -1552,7 +1540,6 @@ (define-public weasyprint (list font-dejavu ;tests depend on it ghostscript python-flit-core - python-pypa-build python-pytest python-pytest-xdist)) (home-page "https://weasyprint.org/") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 10/22] gnu: python-glyphslib: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (7 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 09/22] gnu: weasyprint: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 11/22] gnu: python-statmake: " Marius Bakke ` (11 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-glyphslib)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Add #:test-flags. Remove #:phases. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/fontutils.scm | 42 ++++++++++---------------------------- 1 file changed, 11 insertions(+), 31 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index 9b7f1ac879..eefde1d893 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -76,8 +76,9 @@ (define-module (gnu packages fontutils) #:use-module (guix build-system copy) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) - #:use-module (guix build-system python) #:use-module (guix build-system meson) + #:use-module (guix build-system pyproject) + #:use-module (guix build-system python) #:use-module (guix utils) #:use-module (srfi srfi-1)) @@ -591,39 +592,18 @@ (define-public python-glyphslib (sha256 (base32 "0mkkwd09g76hvif603ij5aqicxh47zvhgyyd0pjcjmpdy6dr70yw")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list - #:phases - #~(modify-phases %standard-phases - (replace 'build - (lambda _ - ;; The Zip format does not support pre-1980 time stamps. - (let ((circa-1980 (* 10 366 24 60 60))) - (setenv "SOURCE_DATE_EPOCH" (number->string circa-1980)) - (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (setenv "GUIX_PYTHONPATH" - (string-append (getcwd) ":" (getenv "GUIX_PYTHONPATH"))) - (invoke "pytest" "-vv" - ;; These fail because the test data has not yet been - ;; updated for newer FontTools: - ;; https://github.com/googlefonts/glyphsLib/issues/787 - ;; Re-enable for versions > 6.0.7. - "--ignore=tests/builder/designspace_gen_test.py" - "--ignore=tests/builder/interpolation_test.py" - ))))))) + #:test-flags #~'("-vv" + ;; These fail because the test data has not yet been + ;; updated for newer FontTools: + ;; https://github.com/googlefonts/glyphsLib/issues/787 + ;; Re-enable for versions > 6.0.7. + "--ignore=tests/builder/designspace_gen_test.py" + "--ignore=tests/builder/interpolation_test.py"))) (native-inputs - (list python-pypa-build - python-setuptools-scm - python-wheel + (list python-setuptools-scm ;; For tests. python-pytest -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 11/22] gnu: python-statmake: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (8 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 10/22] gnu: python-glyphslib: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 12/22] gnu: python-ufolib2: " Marius Bakke ` (10 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-statmake)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/fontutils.scm | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index eefde1d893..5c95f9a427 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -1561,12 +1561,11 @@ (define-public python-statmake (sha256 (base32 "0qavzspxhwnaayj5mxq6ncjjziggabxj157ls04h2rdrpq167706")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. (add-after 'unpack 'adjust-for-older-attrs ;; Our older attrs package is using the 'attr' rather than 'attrs' ;; namespace. @@ -1583,27 +1582,9 @@ (define-public python-statmake (("@attrs") "@attr") (("\\battrs\\.") - "attr.")))) - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" "tests" - ;;"-n" (number->string (parallel-job-count)) - ;; This test requires orjson, which needs the maturin - ;; build system and new Rust dependencies. - ;;"--ignore" "tests/test_preconf.py" - ))))))) + "attr."))))))) (native-inputs (list python-poetry-core - python-pypa-build python-pytest python-ufo2ft)) (propagated-inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 12/22] gnu: python-ufolib2: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (9 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 11/22] gnu: python-statmake: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 13/22] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke ` (9 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/fontutils.scm (python-ufolib2)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/fontutils.scm | 26 ++------------------------ 1 file changed, 2 insertions(+), 24 deletions(-) diff --git a/gnu/packages/fontutils.scm b/gnu/packages/fontutils.scm index 5c95f9a427..46fbc2370a 100644 --- a/gnu/packages/fontutils.scm +++ b/gnu/packages/fontutils.scm @@ -1612,31 +1612,9 @@ (define-public python-ufolib2 (uri (pypi-uri "ufoLib2" version)) (sha256 (base32 "0yx4i8q5rfyqhr2fj70a7z1bp1jv7bdlr64ww9z4nv9ycbda4x9j")))) - (build-system python-build-system) - (arguments - (list - #:phases - #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + (build-system pyproject-build-system) (native-inputs - (list python-pypa-build - python-pytest - python-setuptools-scm - python-wheel)) + (list python-pytest python-setuptools-scm)) (propagated-inputs (list python-attrs python-fonttools-full)) (home-page "https://github.com/fonttools/ufoLib2") (synopsis "Unified Font Object (UFO) font processing library") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 13/22] gnu: python-mypy-protobuf: Switch to pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (10 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 12/22] gnu: python-ufolib2: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 14/22] gnu: python-tempora: " Marius Bakke ` (8 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/protobuf.scm (python-mypy-protobuf)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/protobuf.scm | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/gnu/packages/protobuf.scm b/gnu/packages/protobuf.scm index a61d9d7492..e90e0ad3fa 100644 --- a/gnu/packages/protobuf.scm +++ b/gnu/packages/protobuf.scm @@ -31,6 +31,7 @@ (define-module (gnu packages protobuf) #:use-module (guix git-download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system emacs) #:use-module (guix build-system ruby) @@ -285,22 +286,11 @@ (define-public python-mypy-protobuf (sha256 (base32 "0z03h9k68qvnlyhpk0ndwp01bdx77vrjr6mybxq4ldilkkbksklk")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) (add-before 'check 'generate-protos-for-tests (lambda _ ;; Generate Python sources. @@ -324,7 +314,6 @@ (define-public python-mypy-protobuf (native-inputs (list python-grpc-stubs python-grpcio-tools - python-pypa-build python-pytest python-typing-extensions-next)) (propagated-inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 14/22] gnu: python-tempora: Switch to pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (11 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 13/22] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 15/22] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke ` (7 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-tempora)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Add #:test-flags. Remove #:phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/python-xyz.scm | 27 +++++---------------------- 1 file changed, 5 insertions(+), 22 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 42935cbaaa..f4113ca351 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -21199,30 +21199,13 @@ (define-public python-tempora (uri (pypi-uri "tempora" version)) (sha256 (base32 "09wirlk5vmxlhl9rnxp7g5qz2nsd6b0gnzk5fczbz0s8lsbz386b")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - (list - #:phases - #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - ;; Do not test the myproject.toml build as it tries to pull - ;; dependencies from the Internet. - (invoke "pytest" "-k" "not project"))))))) + ;; Do not test the myproject.toml build as it tries to pull + ;; dependencies from the Internet. + (list #:test-flags #~'("-vv" "-k" "not project"))) (native-inputs - (list python-pypa-build - python-freezegun + (list python-freezegun python-pytest python-pytest-black python-pytest-checkdocs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 15/22] gnu: python-pygmsh: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (12 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 14/22] gnu: python-tempora: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 16/22] gnu: zabbix-cli: Switch to pyproject-build-system Marius Bakke ` (6 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/simulation.scm (python-pygmsh)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/simulation.scm | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/gnu/packages/simulation.scm b/gnu/packages/simulation.scm index 39b5620822..57a7882765 100644 --- a/gnu/packages/simulation.scm +++ b/gnu/packages/simulation.scm @@ -57,6 +57,7 @@ (define-module (gnu packages simulation) #:use-module (guix svn-download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) @@ -828,7 +829,7 @@ (define-public python-pygmsh (sha256 (base32 "11flp2c4ynk1fhanf4mqyzrpd0gjbnv6afrwwc7xi3mb6ms69lr0")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases @@ -838,21 +839,8 @@ (define-public python-pygmsh ;; Due to lack of metadata, the gmsh Python package is not ;; detected although importable. (substitute* "pyproject.toml" - (("\"gmsh\",") "")))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-v" "tests")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) - (native-inputs (list python-pypa-build python-flit-core python-pytest)) + (("\"gmsh\",") ""))))))) + (native-inputs (list python-flit-core python-pytest)) (propagated-inputs (list gmsh python-meshio python-numpy)) (home-page "https://github.com/nschloe/pygmsh") (synopsis "Python frontend for Gmsh") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 16/22] gnu: zabbix-cli: Switch to pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (13 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 15/22] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 17/22] gnu: python-openapi-schema-validator: Use pyproject-build-system Marius Bakke ` (5 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/monitoring.scm (zabbix-cli)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove check phase. --- gnu/packages/monitoring.scm | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/gnu/packages/monitoring.scm b/gnu/packages/monitoring.scm index 51ead2fe9b..d2c0a335db 100644 --- a/gnu/packages/monitoring.scm +++ b/gnu/packages/monitoring.scm @@ -37,6 +37,7 @@ (define-module (gnu packages monitoring) #:use-module (guix git-download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix build-system perl) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix build-system gnu) #:use-module (guix build-system go) @@ -270,7 +271,7 @@ (define-public zabbix-cli (sha256 (base32 "1p8xkq3mxg476srwrgqax76vjzji0rjx32njmgnpa409vaqrbj5p")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases @@ -295,10 +296,7 @@ (define-public zabbix-cli (string-append #$output "/share/man/man1")) (copy-recursively "docs/_build/singlehtml" (string-append #$output "/share/doc/" - #$name "/html")))) - (replace 'check - (lambda _ - (invoke "pytest" "-vv")))))) + #$name "/html"))))))) (native-inputs (list python-pytest python-sphinx)) (inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 17/22] gnu: python-openapi-schema-validator: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (14 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 16/22] gnu: zabbix-cli: Switch to pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 18/22] gnu: python-openapi-spec-validator: " Marius Bakke ` (4 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-web.scm (python-openapi-schema-validator)[build-system]: Set to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove redundant phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/python-web.scm | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm index f5625cc305..370245de3a 100644 --- a/gnu/packages/python-web.scm +++ b/gnu/packages/python-web.scm @@ -79,6 +79,7 @@ (define-module (gnu packages python-web) #:use-module (guix download) #:use-module (guix git-download) #:use-module (guix build-system copy) + #:use-module (guix build-system pyproject) #:use-module (guix build-system python) #:use-module (guix gexp) #:use-module (guix utils) @@ -1601,7 +1602,7 @@ (define-public python-openapi-schema-validator (sha256 (base32 "1swm8h74nhg63nxk347blwq9f1qn6iiq3zisndcvm7axkq3pc2df")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases @@ -1609,21 +1610,8 @@ (define-public python-openapi-schema-validator (add-after 'unpack 'remove-coverage-pytest-options (lambda _ (substitute* "pyproject.toml" - (("^--cov.*") "")))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) - (native-inputs (list python-poetry-core python-pypa-build python-pytest)) + (("^--cov.*") ""))))))) + (native-inputs (list python-poetry-core python-pytest)) (propagated-inputs (list python-isodate python-jsonschema -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 18/22] gnu: python-openapi-spec-validator: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (15 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 17/22] gnu: python-openapi-schema-validator: Use pyproject-build-system Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 19/22] gnu: python-path: " Marius Bakke ` (3 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-web.scm (python-openapi-spec-validator)[build-system]: Set to PYPROJECT-BUILD-SYSTEM. [arguments]: Add #:test-flags. Remove obsolete phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/python-web.scm | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm index 370245de3a..cbc515fc64 100644 --- a/gnu/packages/python-web.scm +++ b/gnu/packages/python-web.scm @@ -1643,33 +1643,19 @@ (define-public python-openapi-spec-validator (sha256 (base32 "1q09sjh4hsc0c8yqbd97h5mp6rwh427y6zyn8kv8wljk6sa0fs4q")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list + ;; The example tests attempt to fetch resources from the Internet + ;; (see: https://github.com/p1c2u/openapi-spec-validator/issues/151). + #:test-flags #~'("-vv" "-k" "not Example and not Exampe") #:phases #~(modify-phases %standard-phases (add-after 'unpack 'remove-coverage-pytest-options (lambda _ (substitute* "pyproject.toml" - (("^--cov.*") "")))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest" "-vv" - ;; The example tests attempt to fetch resources from - ;; the Internet (see: - ;; https://github.com/p1c2u/openapi-spec-validator/issues/151). - "-k" "not Example and not Exampe")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) - (native-inputs (list python-poetry-core python-pypa-build python-pytest)) + (("^--cov.*") ""))))))) + (native-inputs (list python-poetry-core python-pytest)) (propagated-inputs (list python-jsonschema python-openapi-schema-validator -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 19/22] gnu: python-path: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (16 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 18/22] gnu: python-openapi-spec-validator: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 20/22] gnu: python-cattrs: " Marius Bakke ` (2 subsequent siblings) 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-path-bootstrap)[build-system]: Switch to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove #:phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. (python-path): Remove #:phases. Add #:test-flags. --- gnu/packages/python-xyz.scm | 42 +++++++++++-------------------------- 1 file changed, 12 insertions(+), 30 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index f4113ca351..72a516e6fb 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -12548,25 +12548,10 @@ (define-public python-path-bootstrap (uri (pypi-uri "path" version)) (sha256 (base32 "0lig13gxnfv98v790db1smvsbd3mnj7y8rwyiwhfi6xiqibygwms")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments - (list - #:tests? #f - #:phases - #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build/install procedures copied from - ;; python-isort. - (replace 'build - (lambda _ - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda* (#:key outputs #:allow-other-keys) - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) - (native-inputs (list python-pypa-build python-setuptools-scm)) + (list #:tests? #f)) + (native-inputs (list python-setuptools-scm)) (home-page "https://github.com/jaraco/path") (synopsis "Object-oriented file system path manipulation library") (description "@code{path} (formerly @code{path.py}) implements path @@ -12582,18 +12567,15 @@ (define-public python-path (package-arguments python-path-bootstrap) ((#:tests? _ #f) (not (%current-target-system))) - ((#:phases phases #~%standard-phases) - #~(modify-phases #$phases - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - ;; Do not test the myproject.toml build as it tries to pull - ;; dependencies from the Internet. - (invoke "pytest" "-vv" "-k" - (string-append - "not project " - ;; This tests assumes a root user exists. - "and not test_get_owner"))))))))) + ((#:test-flags flags #~'()) + #~(append (list "-vv" "-k" + (string-append + ;; Do not test the myproject.toml build as it tries + ;; to pull dependencies from the Internet. + "not project " + ;; This tests assumes a root user exists. + "and not test_get_owner")) + #$flags)))) (native-inputs (modify-inputs (package-native-inputs python-path-bootstrap) (append python-appdirs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 20/22] gnu: python-cattrs: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (17 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 19/22] gnu: python-path: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 21/22] gnu: python-scikit-build: Switch to pyproject-build-system Marius Bakke 2022-10-22 22:21 ` [bug#58587] [PATCH v3 22/22] gnu: python-deepmerge: Use pyproject-build-system Marius Bakke 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-cattrs)[build-system]: Set to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove obsolete phases. [native-inputs]: Remove PYTHON-PYPA-BUILD. --- gnu/packages/python-xyz.scm | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 72a516e6fb..a8b94b0acb 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -17028,12 +17028,11 @@ (define-public python-cattrs (sha256 (base32 "1n0h25gj6zd02kqyl040xpdvg4hpy1j92716sz0rg019xjqqijqb")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases - ;; XXX: PEP 517 manual build copied from python-isort. (add-after 'unpack 'adjust-for-older-attrs ;; Our older attrs package is using the 'attr' rather than 'attrs' ;; namespace. @@ -17042,14 +17041,6 @@ (define-public python-cattrs (substitute* (find-files "." "\\.py$") (("from attrs\\b") "from attr")))) - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl)))) (replace 'check (lambda* (#:key tests? #:allow-other-keys) (when tests? @@ -17067,7 +17058,6 @@ (define-public python-cattrs python-msgpack python-poetry-core python-pymongo ;for the bson module - python-pypa-build python-pytest python-pytest-xdist)) (propagated-inputs -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 21/22] gnu: python-scikit-build: Switch to pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (18 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 20/22] gnu: python-cattrs: " Marius Bakke @ 2022-10-22 22:20 ` Marius Bakke 2022-10-22 22:21 ` [bug#58587] [PATCH v3 22/22] gnu: python-deepmerge: Use pyproject-build-system Marius Bakke 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:20 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-scikit-build)[build-system]: Set to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove obsolete phases. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/python-xyz.scm | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index a8b94b0acb..bae4c385c0 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -27859,7 +27859,7 @@ (define-public python-scikit-build (uri (pypi-uri "scikit-build" version)) (sha256 (base32 "1wx1m9vnxnnz59lyaisgyxldp313kciyd4af8lf112vb8vbjy9yk")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases @@ -27870,11 +27870,6 @@ (define-public python-scikit-build (("^(CMAKE_DEFAULT_EXECUTABLE = ).*" _ head) (format #f "~a ~s~%" head (search-input-file inputs "bin/cmake")))))) - ;; XXX: PEP 517 manual build copied from python-isort. - (replace 'build - (lambda _ - (setenv "SOURCE_DATE_EPOCH" "315532800") - (invoke "python" "-m" "build" "--wheel" "--no-isolation" "."))) (replace 'check (lambda* (#:key tests? #:allow-other-keys) (when tests? @@ -27900,12 +27895,7 @@ (define-public python-scikit-build ;; nondeterministically (see: ;; https://github.com/scikit-build/scikit-build/issues/711). "and not test_generator_cleanup " - "and not test_generator_selection "))))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + "and not test_generator_selection ")))))))) (native-inputs (list cmake-minimal gfortran @@ -27916,15 +27906,13 @@ (define-public python-scikit-build python-mock python-packaging python-path - python-pypa-build python-pytest python-pytest-cov python-pytest-mock python-pytest-virtualenv python-pytest-xdist python-requests - python-setuptools-scm - python-wheel)) + python-setuptools-scm)) (propagated-inputs (list python-distro python-packaging python-wheel)) (home-page "https://github.com/scikit-build/scikit-build") -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH v3 22/22] gnu: python-deepmerge: Use pyproject-build-system. 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke ` (19 preceding siblings ...) 2022-10-22 22:20 ` [bug#58587] [PATCH v3 21/22] gnu: python-scikit-build: Switch to pyproject-build-system Marius Bakke @ 2022-10-22 22:21 ` Marius Bakke 20 siblings, 0 replies; 65+ messages in thread From: Marius Bakke @ 2022-10-22 22:21 UTC (permalink / raw) To: 58587 * gnu/packages/python-xyz.scm (python-deepmerge)[build-system]: Set to PYPROJECT-BUILD-SYSTEM. [arguments]: Remove obsolete phases. [native-inputs]: Remove PYTHON-PYPA-BUILD and PYTHON-WHEEL. --- gnu/packages/python-xyz.scm | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index bae4c385c0..cddd525b0c 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -30259,34 +30259,17 @@ (define-public python-deepmerge (uri (pypi-uri "deepmerge" version)) (sha256 (base32 "06hagzg8ccmjzqvszdxb52jgx5il8a1jdz41n4dpkyyjsfg7fi2b")))) - (build-system python-build-system) + (build-system pyproject-build-system) (arguments (list #:phases #~(modify-phases %standard-phases (add-before 'build 'set-version (lambda _ - (setenv "SETUPTOOLS_SCM_PRETEND_VERSION" #$version) - ;; ZIP does not support timestamps before 1980. - (setenv "SOURCE_DATE_EPOCH" "315532800"))) - (replace 'build - (lambda _ - (invoke "python" "-m" "build" "--wheel" - "--no-isolation" "."))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest")))) - (replace 'install - (lambda _ - (let ((whl (car (find-files "dist" "\\.whl$")))) - (invoke "pip" "--no-cache-dir" "--no-input" - "install" "--no-deps" "--prefix" #$output whl))))))) + (setenv "SETUPTOOLS_SCM_PRETEND_VERSION" #$version)))))) (native-inputs - (list python-pypa-build - python-setuptools-scm - python-pytest - python-wheel)) + (list python-setuptools-scm + python-pytest)) (home-page "https://deepmerge.readthedocs.io/en/latest/") (synopsis "Merge nested data structures") (description -- 2.38.0 ^ permalink raw reply related [flat|nested] 65+ messages in thread
* [bug#58587] [PATCH 00/14] Introducing pyproject-build-system. 2022-10-22 19:06 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke @ 2022-10-27 13:10 ` Maxim Cournoyer 2 siblings, 0 replies; 65+ messages in thread From: Maxim Cournoyer @ 2022-10-27 13:10 UTC (permalink / raw) To: Marius Bakke; +Cc: 58587, Lars-Dominik Braun, zimoun Hi Marius, I've yet to delve into the actual patches (I intend to do so shortly), but here's a quick reply to your last message. [...] > I've now added a section in the manual about API changes and stability, > and removed more duplicate code from pyproject-build-system. I also > added a news entry. > > I think this is now ready for merge, but have some concerns: > > * The python-toolchain package is a union that includes setuptools, > wheel and others. Packages that pick up a reference to the build-time > Python will needlessly include those other packages. It could be > worked around by propagating instead of using a union. That could be better, yes. They may still be picked up in wrap phases, but that's a Guix-wide problem (can't tell between native vs regular inputs on the build side when native compiling). > * Overriding #:test-flags means having to add back the implicit defaults > for verbose output. We could add them even when #:test-flags is > present so users don't need to add "-vv" manually. This is handled for Emacs packages by having a %default-test-flags or similar that users can cons to... but it's still very too easy to forget. Perhaps we should gradually move to most arguments "extending" the defaults rather than completely overriding them; such an idea was discussed for example for the #:modules and #:imported-modules arguments recently. > > I will merge this in a few days if there are no further comments. It > can still be tweaked directly on 'master' if necessary. > > Updated patches follow shortly. I'll try to have a quick look now, as I see this hasn't landed yet. Thank you for picking up this series! -- Thanks, Maxim ^ permalink raw reply [flat|nested] 65+ messages in thread
end of thread, other threads:[~2022-10-28 1:14 UTC | newest] Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-17 20:06 [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 02/14] gnu: pypy: Move to separate module Marius Bakke 2022-10-27 13:28 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 03/14] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke 2022-10-27 13:29 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 04/14] build-system: Add pyproject-build-system Marius Bakke 2022-10-27 14:23 ` Maxim Cournoyer 2022-10-27 15:17 ` Lars-Dominik Braun 2022-10-27 17:12 ` Maxim Cournoyer 2022-10-27 17:23 ` Marius Bakke 2022-10-27 19:32 ` bug#58587: " Marius Bakke 2022-10-28 1:12 ` [bug#58587] " Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 05/14] gnu: python-autopage: Use pyproject-build-system Marius Bakke 2022-10-27 14:24 ` Maxim Cournoyer 2022-10-17 20:11 ` [bug#58587] [PATCH 06/14] gnu: flair: Switch to pyproject-build-system Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 07/14] gnu: python-pydyf: Use pyproject-build-system Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 08/14] gnu: weasyprint: " Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 09/14] gnu: python-glyphslib: " Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 10/14] gnu: python-statmake: " Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 11/14] gnu: python-ufolib2: " Marius Bakke 2022-10-17 20:11 ` [bug#58587] [PATCH 12/14] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke 2022-10-17 20:12 ` [bug#58587] [PATCH 13/14] gnu: python-tempora: " Marius Bakke 2022-10-17 20:12 ` [bug#58587] [PATCH 14/14] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke 2022-10-27 14:26 ` Maxim Cournoyer 2022-10-27 13:25 ` [bug#58587] [PATCH 01/14] gnu: python-setuptools: Move to python-build Maxim Cournoyer 2022-10-27 14:08 ` Marius Bakke 2022-10-27 17:34 ` Maxim Cournoyer 2022-10-18 9:33 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system zimoun 2022-10-18 21:51 ` Marius Bakke 2022-10-19 9:49 ` zimoun 2022-10-19 23:11 ` Marius Bakke 2022-10-19 23:17 ` Marius Bakke 2022-10-20 8:08 ` zimoun 2022-10-22 19:06 ` Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 02/22] gnu: pypy: Move to separate module Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 04/22] build-system: Add pyproject-build-system Marius Bakke 2022-10-22 19:09 ` [bug#58587] [PATCH v2 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 01/22] gnu: python-setuptools: Move to python-build Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 02/22] gnu: pypy: Move to separate module Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 03/22] gnu: python-pip: Move to (gnu packages python-build) Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 04/22] build-system: Add pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 05/22] news: Add entry for 'pyproject-build-system' Marius Bakke 2022-10-24 16:36 ` pelzflorian (Florian Pelz) 2022-10-24 18:01 ` Julien Lepiller 2022-10-22 22:20 ` [bug#58587] [PATCH v3 06/22] gnu: python-autopage: Use pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 07/22] gnu: flair: Switch to pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 08/22] gnu: python-pydyf: Use pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 09/22] gnu: weasyprint: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 10/22] gnu: python-glyphslib: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 11/22] gnu: python-statmake: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 12/22] gnu: python-ufolib2: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 13/22] gnu: python-mypy-protobuf: Switch to pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 14/22] gnu: python-tempora: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 15/22] gnu: python-pygmsh: Use pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 16/22] gnu: zabbix-cli: Switch to pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 17/22] gnu: python-openapi-schema-validator: Use pyproject-build-system Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 18/22] gnu: python-openapi-spec-validator: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 19/22] gnu: python-path: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 20/22] gnu: python-cattrs: " Marius Bakke 2022-10-22 22:20 ` [bug#58587] [PATCH v3 21/22] gnu: python-scikit-build: Switch to pyproject-build-system Marius Bakke 2022-10-22 22:21 ` [bug#58587] [PATCH v3 22/22] gnu: python-deepmerge: Use pyproject-build-system Marius Bakke 2022-10-27 13:10 ` [bug#58587] [PATCH 00/14] Introducing pyproject-build-system Maxim Cournoyer
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).