From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CPl2ALAc919rOQAA0tVLHw (envelope-from ) for ; Thu, 07 Jan 2021 14:37:36 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id kAPkN68c91+bbQAAB5/wlQ (envelope-from ) for ; Thu, 07 Jan 2021 14:37:35 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A66099403C8 for ; Thu, 7 Jan 2021 14:37:34 +0000 (UTC) Received: from localhost ([::1]:52940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kxWPt-00051g-IG for larch@yhetil.org; Thu, 07 Jan 2021 09:37:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxVJe-00019h-Ub for guix-patches@gnu.org; Thu, 07 Jan 2021 08:27:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:35265) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kxVJe-0000Bj-HR for guix-patches@gnu.org; Thu, 07 Jan 2021 08:27:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kxVJe-0004ZL-CZ for guix-patches@gnu.org; Thu, 07 Jan 2021 08:27:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45712] [PATCHES] Improve Python package quality Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 07 Jan 2021 13:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45712 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45712@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.161002601017538 (code B ref -1); Thu, 07 Jan 2021 13:27:02 +0000 Received: (at submit) by debbugs.gnu.org; 7 Jan 2021 13:26:50 +0000 Received: from localhost ([127.0.0.1]:46810 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxVJM-0004Yi-Ij for submit@debbugs.gnu.org; Thu, 07 Jan 2021 08:26:50 -0500 Received: from lists.gnu.org ([209.51.188.17]:59520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxVJI-0004YZ-S3 for submit@debbugs.gnu.org; Thu, 07 Jan 2021 08:26:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxVJI-00010A-Ck for guix-patches@gnu.org; Thu, 07 Jan 2021 08:26:40 -0500 Received: from mout-p-102.mailbox.org ([80.241.56.152]:13292) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1kxVJC-0008Mf-9Y for guix-patches@gnu.org; Thu, 07 Jan 2021 08:26:39 -0500 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:105:465:1:1:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4DBRpD0d8gzQlZZ for ; Thu, 7 Jan 2021 14:26:28 +0100 (CET) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1610025985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=h2QO2yrUz6gaQFG+E05ghrfYRkyAEEgfIH/kQYaE+B4=; b=q3yCkxKj2kAJrv1KtyAq6oMU4pqp8aak8csiWMh57JRem1bP4b+ZHpbyuAhY9ENN5Y150H mvufVnhRFwQSzMaQqz02FACO7eFp0O8Du/COZ+96ctq4tWeuCfKNU/etmxN1csSGv9H9pm z8ianCnpxfHNH4njHEzvvIPPo6+nI58poSYBau4MWtgBBWq//pziZKJsUfLuVNID3IRoeU 8IgJG4sTO62RGwGc1U3K+sGrxX+5Krll1sWRUnsjGpQjUwR5RNtVRCFiab0y/iXkUFoeaR AEeN/DvRgJ962h+Hhf6YV+9p3/Aoi3mBRUvG2SWkcpwciOTkKQg6bi0I0Hxldw== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter05.heinlein-hosting.de (spamfilter05.heinlein-hosting.de [80.241.56.123]) (amavisd-new, port 10030) with ESMTP id I2gY7b1DIyAd for ; Thu, 7 Jan 2021 14:26:22 +0100 (CET) Date: Thu, 7 Jan 2021 14:26:20 +0100 From: Lars-Dominik Braun Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="Pbo5I8Q9EiMmYpOS" Content-Disposition: inline Content-Transfer-Encoding: 8bit X-MBO-SPAM-Probability: X-Rspamd-Score: -7.36 / 15.00 / 15.00 X-Rspamd-Queue-Id: 79BA41850 X-Rspamd-UID: 05b832 Received-SPF: pass client-ip=80.241.56.152; envelope-from=lars@6xq.net; helo=mout-p-102.mailbox.org X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Mailman-Approved-At: Thu, 07 Jan 2021 09:35:58 -0500 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.25 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=6xq.net header.s=MBO0001 header.b=q3yCkxKj; dmarc=fail reason="SPF not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: A66099403C8 X-Spam-Score: -1.25 X-Migadu-Scanner: scn0.migadu.com X-TUID: ahGTWw3lZd86 --Pbo5I8Q9EiMmYpOS Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi, as announced in https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00021.html I’ve been working on adding an additional phase to Python packages to check whether they actually work. I cleaned up my patch, added tests and now I’m pretty confident it works as expected. The first patch in this series adds this phase, while the other ones fix build failures caused by it. All of this should go to core-updates (or a separate wip-* branch?), since it causes a massive number of rebuilds. You can also pull my git repo at https://github.com/PromyLOPh/guix.git branch work-python-importcheck. Cheers, Lars --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="0001-build-system-python-Validate-installed-package.patch" Content-Transfer-Encoding: 8bit >From 7a9ac1ee220ec2cb3dc10da1a8455289aa5e3b99 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Sun, 3 Jan 2021 10:30:29 +0100 Subject: [PATCH 01/15] build-system/python: Validate installed package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a new phase validating usalibity of installed Python packages. * guix/build/python-build-system.scm (validate-loadable): New phase. (%standard-phases): Use it. * tests/builders.scm (python-dummy-*) Add test packages. ("python-build-system: …"): Add tests. --- guix/build/python-build-system.scm | 70 +++++++++++++ tests/builders.scm | 161 ++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 3 deletions(-) diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm index 09bd8465c8..15d4f0c54e 100644 --- a/guix/build/python-build-system.scm +++ b/guix/build/python-build-system.scm @@ -148,6 +148,75 @@ (format #t "test suite not run~%")) #t) +(define validate-script + "from __future__ import print_function # Python 2 support. +import pkg_resources, sys, importlib, traceback +try: + from importlib.machinery import PathFinder +except ImportError: + PathFinder = None +ret = 0 +# Only check site-packages installed by this package, but not dependencies +# (which pkg_resources.working_set would include). Path supplied via argv. +ws = pkg_resources.find_distributions(sys.argv[1]) +for dist in ws: + print('validating', repr(dist.project_name), dist.location) + try: + print('...checking requirements', end=': ') + req = str(dist.as_requirement()) + # dist.activate() is not enough to actually check requirements, we have to + # .require() it. + pkg_resources.require(req) + print('OK') + except Exception as e: + print('ERROR:', req, e) + ret = 1 + continue + # Try to load entry points of console scripts too, making sure they work. They + # should be removed if they don’t. Other groups may not be safe, as they can + # depend on optional packages. + for group, v in dist.get_entry_map().items(): + if group not in {'console_scripts', }: + continue + for name, ep in v.items(): + try: + print('...trying to load endpoint', group, name, end=': ') + ep.load() + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + continue + # And finally try to load top level modules. This should not have any + # side-effects. + for name in dist.get_metadata_lines('top_level.txt'): + # Only available on Python 3. + if PathFinder and PathFinder.find_spec(name) is None: + # Ignore unavailable modules. Cannot use ModuleNotFoundError, + # because it is raised by failed imports too. + continue + try: + print('...trying to load module', name, end=': ') + importlib.import_module(name) + print('OK') + except Exception: + print('ERROR:') + traceback.print_exc(file=sys.stdout) + ret = 1 + continue +sys.exit(ret)") + +(define* (validate-loadable #:key tests? inputs outputs #:allow-other-keys) + "Ensure packages depending on this package via setuptools work properly, +their advertised endpoints work and their top level modules are importable +without errors." + (add-installed-pythonpath inputs outputs) + ;; Make sure the working directory is empty (i.e. no Python modules in it) + (with-directory-excursion "/tmp" + (invoke "python" "-c" validate-script (site-packages inputs outputs))) + #t) + (define (python-version python) (let* ((version (last (string-split python #\-))) (components (string-split version #\.)) @@ -267,6 +336,7 @@ installed with setuptools." (replace 'install install) (add-after 'install 'check check) (add-after 'install 'wrap wrap) + (add-after 'check 'validate-loadable validate-loadable) (add-before 'strip 'rename-pth-file rename-pth-file))) (define* (python-build #:key inputs (phases %standard-phases) diff --git a/tests/builders.scm b/tests/builders.scm index fdcf38ded3..8fc0c07ee0 100644 --- a/tests/builders.scm +++ b/tests/builders.scm @@ -21,15 +21,15 @@ #:use-module (guix download) #:use-module (guix build-system) #:use-module (guix build-system gnu) + #:use-module (guix build-system python) #:use-module (guix store) + #:use-module (guix monads) #:use-module (guix utils) #:use-module (guix base32) #:use-module (guix derivations) #:use-module (gcrypt hash) #:use-module (guix tests) - #:use-module ((guix packages) - #:select (package? - package-derivation package-native-search-paths)) + #:use-module (guix packages) #:use-module (gnu packages bootstrap) #:use-module (ice-9 match) #:use-module (srfi srfi-1) @@ -78,4 +78,159 @@ (test-assert "gnu-build-system" (build-system? gnu-build-system)) + +(define python-dummy-ok + (package + (name "python-dummy-ok") + (version "0.1") + (source #f) ; source is generated in 'unpack + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'unpack + (lambda _ + (mkdir-p "src") + (chdir "src") + (mkdir-p "dummy") + (invoke "touch" "dummy/__init__.py") + (with-output-to-file "setup.py" + (lambda _ + (display "from setuptools import setup +setup( + name='dummy-ok', + version='0.1', + packages=['dummy'], + ) +"))) + #t))))) + (home-page #f) + (synopsis #f) + (description #f) + (license #f))) + +(define python2-dummy-ok + (package-with-python2 python-dummy-ok)) + +(define python-dummy-fail-requirements + (package + (name "python-dummy-fail-requirements") + (version "0.1") + (source #f) ; source is generated in 'unpack + (build-system python-build-system) + (arguments + `(#:tests? #f + #:phases + (modify-phases %standard-phases + (replace 'unpack + (lambda _ + (mkdir-p "src") + (chdir "src") + (mkdir-p "dummy") + (invoke "touch" "dummy/__init__.py") + (with-output-to-file "setup.py" + (lambda _ + (display "from setuptools import setup +setup( + name='dummy-fail-requirements', + version='0.1', + packages=['dummy'], + install_requires=['nonexistent'], + ) +"))) + #t))))) + (home-page #f) + (synopsis #f) + (description #f) + (license #f))) + +(define-public python2-dummy-fail-requirements + (package-with-python2 python-dummy-fail-requirements)) + +(define-public python-dummy-fail-import + (package + (name "python-dummy-fail-import") + (version "0.1") + (source #f) ; source is generated in 'unpack + (build-system python-build-system) + (arguments + `(#:tests? #f + #:phases + (modify-phases %standard-phases + (replace 'unpack + (lambda _ + (mkdir-p "src") + (chdir "src") + (mkdir-p "dummy") + (with-output-to-file "dummy/__init__.py" + (lambda _ + (display "import nonexistent"))) + (with-output-to-file "setup.py" + (lambda _ + (display "from setuptools import setup +setup( + name='dummy-fail-import', + version='0.1', + packages=['dummy'], + ) +"))) + #t))))) + (home-page #f) + (synopsis #f) + (description #f) + (license #f))) + +(define-public python2-dummy-fail-import + (package-with-python2 python-dummy-fail-import)) + +(define-public python-dummy-fail-console-script + (package + (name "python-dummy-fail-console-script") + (version "0.1") + (source #f) ; source is generated in 'unpack + (build-system python-build-system) + (arguments + `(#:tests? #f + #:phases + (modify-phases %standard-phases + (replace 'unpack + (lambda _ + (mkdir-p "src") + (chdir "src") + (mkdir-p "dummy") + (invoke "touch" "dummy/__init__.py") + (with-output-to-file "setup.py" + (lambda _ + (display "from setuptools import setup +setup( + name='dummy-fail-console-script', + version='0.1', + packages=['dummy'], + entry_points={'console_scripts': ['broken = dummy:nonexistent']}, + ) +"))) + #t))))) + (home-page #f) + (synopsis #f) + (description #f) + (license #f))) + +(define-public python2-dummy-fail-console-script + (package-with-python2 python-dummy-fail-console-script)) + +(with-external-store store + (unless store (test-skip 1)) + (test-assert "python-build-system: dummy-ok" + (let* ((drv (package-derivation store python-dummy-ok))) + (build-derivations store (list drv)))) + (unless store (test-skip 1)) + (test-assert "python-build-system: dummy-fail-requirements" + (not (false-if-exception (package-derivation store python-dummy-fail-requirements)))) + (unless store (test-skip 1)) + (test-assert "python-build-system: dummy-fail-import" + (not (false-if-exception (package-derivation store python-dummy-fail-import)))) + (unless store (test-skip 1)) + (test-assert "python-build-system: dummy-fail-console-script" + (not (false-if-exception (package-derivation store python-dummy-fail-console-script))))) + (test-end "builders") -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-gnu-pytest-6-Add-missing-propagated-input.patch" >From cf9ae80b59e86a60c27734c8cc27637757490d70 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:25:56 +0100 Subject: [PATCH 02/15] gnu: pytest@6: Add missing propagated-input. * gnu/packages/check.scm (python-pytest-6) [native-inputs]: Remove python-iniconfig. [propagated-inputs]: Move it here. --- gnu/packages/check.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index 1300f9e1a6..9d1e0b8173 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -964,13 +964,13 @@ and many external plugins.") (propagated-inputs (append (alist-delete "python-py" (package-propagated-inputs python-pytest)) - `(("python-py" ,python-py-next)))) + `(("python-py" ,python-py-next) + ("python-iniconfig" ,python-iniconfig)))) (native-inputs (append (alist-delete "python-pytest" (package-native-inputs python-pytest)) `(("python-pytest" ,python-pytest-6-bootstrap) - ("python-toml" ,python-toml) - ("python-iniconfig" ,python-iniconfig)))))) + ("python-toml" ,python-toml)))))) ;; Pytest 4.x are the last versions that support Python 2. (define-public python2-pytest -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-gnu-python-pytest-xdist-Add-missing-input-relax-pyte.patch" >From 9bc53a8e9706440668ec70d88db1ebc7d5e2c71d Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:27:55 +0100 Subject: [PATCH 03/15] gnu: python-pytest-xdist: Add missing input, relax pytest requirement. * gnu/packages/check.scm: (python-pytest-xdist) [arguments]: Relax pytest version requirements. [propagated-inputs]: Add python-pytest-forked. --- gnu/packages/check.scm | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index 9d1e0b8173..32a1a2d6a3 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -1216,20 +1216,27 @@ same arguments.") #t)))) (build-system python-build-system) (arguments - '(#:tests? #f)) ;FIXME: Some tests are failing. - ;; #:phases - ;; (modify-phases %standard-phases - ;; (delete 'check) - ;; (add-after 'install 'check - ;; (lambda* (#:key inputs outputs #:allow-other-keys) - ;; (add-installed-pythonpath inputs outputs) - ;; (zero? (system* "py.test" "-v"))))) + '(#:tests? #f ; Lots of tests fail. + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'patch + (lambda* (#:key inputs #:allow-other-keys) + ;; Relax pytest requirement. + (substitute* "setup.py" + (("pytest>=6\\.0\\.0") "pytest")) + #t)) + (replace 'check + (lambda* (#:key tests? inputs outputs #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "py.test" "-v"))))))) (native-inputs `(("python-setuptools-scm" ,python-setuptools-scm))) (propagated-inputs `(("python-execnet" ,python-execnet) ("python-pytest" ,python-pytest) - ("python-py" ,python-py))) + ("python-py" ,python-py) + ("python-pytest-forked" ,python-pytest-forked))) (home-page "https://github.com/pytest-dev/pytest-xdist") (synopsis -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0004-gnu-python-fixtures-bootstrap-Do-not-apply-loadable-.patch" >From 0974fc1c98ae1554ddd37a1ca40127bd5df95179 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:29:59 +0100 Subject: [PATCH 04/15] gnu: python-fixtures-bootstrap: Do not apply loadable check. * gnu/packages/check.scm (python-fixtures-bootstrap) [arguments]: Delete 'validate-loadable. --- gnu/packages/check.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index 32a1a2d6a3..fba408d88f 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -1541,7 +1541,12 @@ protocol."))) (base32 "1vxj29bzz3rd4pcy51d05wng9q9dh4jq6wx92yklsm7i6h1ddw7w")))) (build-system python-build-system) - (arguments `(#:tests? #f)) + (arguments + `(#:tests? #f + #:phases + (modify-phases %standard-phases + ;; Package is not loadable on its own at this stage. + (delete 'validate-loadable)))) (propagated-inputs `(("python-pbr-minimal" ,python-pbr-minimal) ("python-six" ,python-six))) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0005-gnu-python-pytest-pep8-Fix-package.patch" >From d2272e00c42e3ddde1b0532a4b1ad187816dc98f Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:35:11 +0100 Subject: [PATCH 05/15] gnu: python-pytest-pep8: Fix package. * gnu/packages/check.scm (python-pytest-pep8) [arguments]: Remove dependency on pytest-cache and add proper 'check phase. --- gnu/packages/check.scm | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index fba408d88f..796635e012 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -2043,7 +2043,19 @@ failures.") "06032agzhw1i9d9qlhfblnl3dw5hcyxhagn7b120zhrszbjzfbh3")))) (build-system python-build-system) (arguments - `(#:tests? #f)) ; Fails with recent pytest and pep8. See upstream issues #8 and #12. + `(#:tests? #t ; Fails with recent pytest and pep8. See upstream issues #8 and #12. + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'fix-dependencies + (lambda _ + (substitute* "setup.py" + (("'pytest-cache', ") "")) ; Included in recent pytest + #t)) + (replace 'check + (lambda* (#:key tests? inputs outputs #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "pytest" "-v"))))))) (native-inputs `(("python-pytest" ,python-pytest))) (propagated-inputs -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0006-gnu-python-pyfakefs-Disable-unreliable-test.patch" >From a252a9d180804a6cb0d2829acd99a1010b4e1984 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:42:40 +0100 Subject: [PATCH 06/15] gnu: python-pyfakefs: Disable unreliable test. * gnu/packages/check.scm (python-pyfakefs) [arguments]: Disable test. --- gnu/packages/check.scm | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index 796635e012..0535a1b2c4 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -2858,13 +2858,14 @@ grew out of the @dfn{Vc} project.") (arguments `(#:phases (modify-phases %standard-phases - ;; The default test suite does not run these extra tests. - (add-after 'check 'check-pytest-plugin + (replace 'check (lambda _ - (invoke - "python" "-m" "pytest" - "pyfakefs/pytest_tests/pytest_plugin_test.py") - #t))))) + (invoke "pytest" + "pyfakefs/tests" + ;; The default test suite does not run these extra tests. + ;"pyfakefs/pytest_tests/pytest_plugin_test.py" + ;; atime difference is larger than expected. + "-k" "not test_copy_real_file")))))) (native-inputs `(("python-pytest" ,python-pytest))) (build-system python-build-system) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0007-gnu-python-slugify-Add-missing-input.patch" >From ced04daa680ef5f261b6f23e0500aca60032d323 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:48:36 +0100 Subject: [PATCH 07/15] gnu: python-slugify: Add missing input. * gnu/packages/python-web.scm (python-slugify) [propagated-inputs]: Add python-text-unidecode. --- gnu/packages/python-web.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm index 622f5fc6e2..e3cf25a687 100644 --- a/gnu/packages/python-web.scm +++ b/gnu/packages/python-web.scm @@ -4398,7 +4398,8 @@ Python.") (sha256 (base32 "0w22fapghmzk3xdasc4dn7h8sl58l08d1h5zbf72dh80drv1g9b9")))) (propagated-inputs - `(("python-unidecode" ,python-unidecode))) + `(("python-unidecode" ,python-unidecode) + ("python-text-unidecode" ,python-text-unidecode))) (arguments `(#:phases (modify-phases %standard-phases -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0008-gnu-python-websockets-Fix-Python-package-name.patch" >From 4412f706693caa7fd888a637ea66b65afd34a15b Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:49:01 +0100 Subject: [PATCH 08/15] gnu: python-websockets: Fix Python package name. * gnu/packages/python-web.scm (python-websockets) [arguments]: Add new phase to fix package name. --- gnu/packages/python-web.scm | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm index e3cf25a687..e3318a18ce 100644 --- a/gnu/packages/python-web.scm +++ b/gnu/packages/python-web.scm @@ -5086,7 +5086,16 @@ Plus all the standard features of requests: (base32 "03s3ml6sbki24aajllf8aily0xzrn929zxi84p50zkkbikdd4raw")))) (build-system python-build-system) - (arguments '(#:tests? #f)) ; Tests not included in release tarball. + (arguments + '(#:tests? #f ; Tests not included in release tarball. + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'patch + (lambda* (#:key inputs #:allow-other-keys) + ;; Python package names use dot as separator. + (substitute* "setup.py" + (("websockets/extensions") "websockets.extensions")) + #t))))) (home-page "https://github.com/aaugustin/websockets") (synopsis "Python implementation of the WebSocket Protocol (RFC 6455 & 7692)") -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-gnu-python-black-Remove-blackd.patch" >From f1b8ada28652fa85fa62075afb462a0611a50a50 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:50:18 +0100 Subject: [PATCH 09/15] gnu: python-black: Remove blackd. * gnu/packages/python-xyz.scm (python-black) [arguments]: Add new phase to prevent installation of blackd. --- gnu/packages/python-xyz.scm | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 818a244378..a02960f1c6 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -4277,7 +4277,14 @@ matching of file paths.") (substitute* "tests/test_black.py" (("( *)def test_python38" match indent) (string-append indent "@unittest.skip(\"guix\")\n" match))) - #t))))) + #t)) + ;; Remove blackd, because it depends on python-aiohttp and + ;; python-aiohttp-cors. + (add-after 'unpack 'remove-entrypoint + (lambda _ + (substitute* "setup.py" + (("\\s*\"blackd=blackd:patched_main \\[d\\]\",\n") "") + (("\"blackd\", ") ""))))))) (propagated-inputs `(("python-click" ,python-click) ("python-attrs" ,python-attrs) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0010-gnu-python-traitlets-Add-missing-input.patch" >From df86a15fef25655c78faed179b25cd3c06e6b396 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 13:53:25 +0100 Subject: [PATCH 10/15] gnu: python-traitlets: Add missing input. * gnu/packages/python-xyz.scm (python-traitlets) [propagated-inputs]: Add python-six. --- gnu/packages/python-xyz.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index a02960f1c6..4b7fff5750 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -6973,7 +6973,8 @@ cluster down and deletes the throwaway profile.") (replace 'check (lambda _ (invoke "pytest" "-vv" "traitlets")))))) (propagated-inputs `(("python-ipython-genutils" ,python-ipython-genutils) - ("python-decorator" ,python-decorator))) + ("python-decorator" ,python-decorator) + ("python-six" ,python-six))) (native-inputs `(("python-pytest" ,python-pytest))) (properties `((python2-variant . ,(delay python2-traitlets)))) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0011-gnu-python-idna-ssl-Add-missing-input.patch" >From f41d09b6acf31fc36b37f8fb895fb9ded52fe825 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 14:12:56 +0100 Subject: [PATCH 11/15] gnu: python-idna-ssl: Add missing input. * gnu/packages/python-xyz.scm (python-idna-ssl) [propagated-inputs]: Add python-idna. --- gnu/packages/python-xyz.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 4b7fff5750..58ad3b6e55 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -9517,6 +9517,7 @@ is binding LibSass.") (build-system python-build-system) (arguments `(#:tests? #f)) ;circular dependency with python-aiohttp + (propagated-inputs `(("python-idna" ,python-idna))) (home-page "https://github.com/aio-libs/idna-ssl") (synopsis "Patch @code{ssl.match_hostname} for Unicode(idna) domains support") (description "Patch @code{ssl.match_hostname} for Unicode(idna) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0012-gnu-python-twisted-Remove-broken-console-scripts.patch" >From fe272f9e3a35a9a3b6b9994f3b6881fbe8facb7b Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 14:13:53 +0100 Subject: [PATCH 12/15] gnu: python-twisted: Remove broken console scripts. * gnu/packages/python-xyz.scm (python-twisted) [arguments]: Patch setup.py. --- gnu/packages/python-xyz.scm | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 58ad3b6e55..d8b61cd1d2 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -12716,7 +12716,14 @@ format.") "17d3hnxv9qndagzz63mdpyk99xj63p9gq586vjn0rxk8cl197nym")))) (build-system python-build-system) (arguments - '(#:tests? #f)) ; FIXME: some tests fail + '(#:tests? #f ; FIXME: some tests fail + #:phases + (modify-phases %standard-phases + ;; Remove scripts, because they depend on [conch] + (add-after 'unpack 'remove-entrypoint + (lambda _ + (substitute* "src/twisted/python/_setup.py" + (("\".+ = twisted\\.conch\\.scripts\\..+\",") ""))))))) (propagated-inputs `(("python-zope-interface" ,python-zope-interface) ("python-pyhamcrest" ,python-pyhamcrest) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0013-gnu-python-automat-Remove-broken-console-script.patch" >From fef86cb5fc9dede96cc458c3801818f7cd6912cd Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 14:15:12 +0100 Subject: [PATCH 13/15] gnu: python-automat: Remove broken console script. * gnu/packages/python-xyz.scm (python-automat) [arguments]: Patch setup.py. --- gnu/packages/python-xyz.scm | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index d8b61cd1d2..acd674878a 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -15151,7 +15151,15 @@ instead of servers and network commands.") ;; python-twisted depends on python-automat. Twisted is optional, but the ;; tests fail if it is not available. Also see ;; . - (arguments '(#:tests? #f)) + (arguments + `(#:tests? #f + #:phases + (modify-phases %standard-phases + ;; Remove script, because it depends on python-twisted. + (add-after 'unpack 'remove-entrypoint + (lambda _ + (substitute* "setup.py" + (("\"automat-visualize = automat._visualize:tool\"") ""))))))) (native-inputs `(("python-m2r" ,python-m2r) ("python-setuptools-scm" ,python-setuptools-scm) -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0014-gnu-python-packaging-bootstrap-Remove-dependency.patch" >From 9c0eb0a2ede51bc234cd5c4d7a1347fa3f83e843 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 14:16:05 +0100 Subject: [PATCH 14/15] gnu: python-packaging-bootstrap: Remove dependency. * gnu/packages/python-xyz.scm (python-packaging-bootstrap) [arguments]: Remove dependency from setup.py, which we do not provide for this variant. --- gnu/packages/python-xyz.scm | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index acd674878a..e78016221f 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -16049,10 +16049,18 @@ information.") (package/inherit python-packaging (name "python-packaging-bootstrap") + (arguments + (substitute-keyword-arguments (package-arguments python-packaging) + ((#:phases phases) + `(modify-phases ,phases + (add-after 'unpack 'fix-dependencies + (lambda* (#:key tests? #:allow-other-keys) + (substitute* "setup.py" (("\"six\"") "")) + #t)))) + ((#:tests? _ #f) #f))) (native-inputs '()) (propagated-inputs - `(("python-pyparsing" ,python-pyparsing))) - (arguments '(#:tests? #f))))) + `(("python-pyparsing" ,python-pyparsing)))))) (define-public python2-packaging-bootstrap (hidden-package -- 2.26.2 --Pbo5I8Q9EiMmYpOS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0015-gnu-python-traceback2-Add-missing-dependency.patch" >From ecc8eebadbeb8c75ac5025d7bce581423d4d1894 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Thu, 7 Jan 2021 14:17:20 +0100 Subject: [PATCH 15/15] gnu: python-traceback2: Add missing dependency. * gnu/packages/python-xyz.scm (python-traceback2) [propagated-inputs]: Add python-six. --- gnu/packages/python-xyz.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index e78016221f..1df9807626 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -17266,7 +17266,8 @@ lines are read from a single file.") (native-inputs `(("python-pbr" ,python-pbr-minimal))) (propagated-inputs - `(("python-linecache2" ,python-linecache2))) + `(("python-linecache2" ,python-linecache2) + ("python-six" ,python-six))) (home-page "https://github.com/testing-cabal/traceback2") (synopsis "Backports of the traceback module") -- 2.26.2 --Pbo5I8Q9EiMmYpOS--