From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id oOMRGikAX2d8FgAAe85BDQ:P1 (envelope-from ) for ; Sun, 15 Dec 2024 16:13:29 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id oOMRGikAX2d8FgAAe85BDQ (envelope-from ) for ; Sun, 15 Dec 2024 17:13:29 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=fo+5jycp; dkim=fail ("headers rsa verify failed") header.d=6xq.net header.s=MBO0001 header.b="b3R/OMqa"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1734279209; a=rsa-sha256; cv=none; b=BBhS99q4TE888U1S9nKr3ed2veC5kJL3Sl46kE1rehYeZdfcAdOlfa1g4z64h2eUZ6C0bb 3iwLqXIHgwjibRYn9sY2a/Y6ydjwwkMsqaO8sWXO7qEvGBkFLOcT5GOd/pF/GXAxlLyvNs uwHBropE6Vkjb6f+iLKgRUT+4XURsU8dgosQCZQSmYy7Wklj2mYPMiPoN8Cpenc2BhBeB9 f2pL0LGKVd3/LGGUbxT+e+bLzEB7R5pq1gGzBVXbjHIDIEcjH9qDrf7RtB5L2iyCwXNErj Zx8Y7mrYWC3JVqLQi727SUiqFqHQ3UbK13bzfn+APZyiRVB2BcmksTl+0qYrzQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=fo+5jycp; dkim=fail ("headers rsa verify failed") header.d=6xq.net header.s=MBO0001 header.b="b3R/OMqa"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=6xq.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1734279209; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=81TMEFGlcE6UvXLAjdxl+f62B7GAiObyFqB6nEKTUA8=; b=kSy6D5G1cQyZTsLv9N5KQOzs8qHDn5NXctdRaw2LdbA+DL/bhSH4bENjv+LNb92X7G4vLU pNoSgAPPfPX/rD0T6+oyGp1qM4qT6gBmB3T6z5caVkDWVrIvl+TG/lxKDd6b5I544FRmha K1Y0obEDUGY/5W/ZqkJOtiDloyErvdE7V+4HeiNv90Bmm38tTr9+St7INHpumsOpCaZdMQ +fAvcnR12MJxXFPJYS/rD/0JKz1Dya3PtzBcbjTi+zT90RLgXYvMcM7xBTmL6n0GkX8xSy Y1GBgPK13rHB72AoXXiH7lMNAP10VDytmazY7/Dz98YLEmYqwz+v3x5rSzPTEA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 381CC956FD for ; Sun, 15 Dec 2024 17:13:27 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tMrF2-0002DQ-3V; Sun, 15 Dec 2024 11:13:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tMrEw-0002D9-3v for bug-guix@gnu.org; Sun, 15 Dec 2024 11:13:08 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tMrEs-0004sx-0E for bug-guix@gnu.org; Sun, 15 Dec 2024 11:13:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=In-Reply-To:MIME-Version:References:From:Date:To:Subject; bh=81TMEFGlcE6UvXLAjdxl+f62B7GAiObyFqB6nEKTUA8=; b=fo+5jycp5pnqT3rvWhhkQptCDp4QwFi/tCQfuCt9b8i2fBqNMEGGMgX1/otMyuGK/zJOjMMyiH0PJKJV9PF+d3PaJDkqKi9teSuk4Bz1kb4uPDTzAxJVks5cgUWP7LQVvoOPrgNYH5+HXFVMIuHxM+4h0axThiZtZpOSZpKxSONcrZkSRu5EaZqcyLh4xLFeR0dnPNL6n8qU6GGLLilns7qJdao0fL8HrTGFJQ9oZjiKy774jRCyFzxNqluQ/TXKE2LLyl7H29QvroFSTNOQvQxA+6TZmmSvSzSItb2wbqi/d/UE7U9Al0qkn4bHocEDefxIEfBmONgSKGucugDTpw==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tMrEr-0005l4-Ld for bug-guix@gnu.org; Sun, 15 Dec 2024 11:13:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#69997: Should =?UTF-8?Q?=E2=80=98guix?= import =?UTF-8?Q?pypi=E2=80=99?= get dependencies from pyproject files? Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 15 Dec 2024 16:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69997 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Tanguy LE CARROUR , 69997@debbugs.gnu.org, Sharlatan Hellseher Received: via spool by 69997-submit@debbugs.gnu.org id=B69997.173427917922117 (code B ref 69997); Sun, 15 Dec 2024 16:13:01 +0000 Received: (at 69997) by debbugs.gnu.org; 15 Dec 2024 16:12:59 +0000 Received: from localhost ([127.0.0.1]:51689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMrEn-0005ke-Vr for submit@debbugs.gnu.org; Sun, 15 Dec 2024 11:12:59 -0500 Received: from mout-p-102.mailbox.org ([80.241.56.152]:51444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tMrEk-0005kI-9d for 69997@debbugs.gnu.org; Sun, 15 Dec 2024 11:12:56 -0500 Received: from smtp2.mailbox.org (smtp2.mailbox.org [10.196.197.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4YB7Pq1w8vz9sSH; Sun, 15 Dec 2024 17:12:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1734279135; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=81TMEFGlcE6UvXLAjdxl+f62B7GAiObyFqB6nEKTUA8=; b=b3R/OMqabwpzT6bh4ZJB5bVSYDkxEZjH14eqlFxM6ic6M+xcrmdtjhCFP9cZSdKiETN+QM 0CWj2ENP1Z0vePSvt4cc03aRO0yuFazDZlP31/tTPICggGQgkNANfyEMRXNs2QEIIrVDv2 DeIANAq1Ue685Y0z9sHC6dFgtl4eamHcUbipZPX1/aNdGSUu50+dcoyzcuZiAwKIt2Pfsj kl/lTGmC9UJE7QPwnzDp17FFdHccAHDEP2TmgHZRaUoDEsoj3AX07qV6ul1/G4MG6kk3Pz ueJQnMC1r1sHI+tJrx6afb3ex83cA0PYcw/K5FdqLGZj92lBXGEcDTd+kaQSQw== Date: Sun, 15 Dec 2024 17:12:12 +0100 From: Lars-Dominik Braun Message-ID: References: <877chqtvsk.fsf@inria.fr> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="kpsUVzBC96Nk7Jsm" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <877chqtvsk.fsf@inria.fr> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -3.93 X-Spam-Score: -3.93 X-Migadu-Queue-Id: 381CC956FD X-Migadu-Scanner: mx10.migadu.com X-TUID: MOd5bXtHtsFP --kpsUVzBC96Nk7Jsm Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi, > Should ‘guix import pypi’ attempt to get dependency information from > ‘pyproject.toml’, in addition to ‘requirements.txt’ and wheel ‘METADATA’ > as it already does? attached patches allow parsing the standardized pyproject.toml fields for dependencies. This won’t work for poetry (we need a different version parser for that), but it’s a start. Lars --kpsUVzBC96Nk7Jsm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0001-import-pypi-Support-extracting-dependencies-from-pyp.patch" >From c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25 Mon Sep 17 00:00:00 2001 Message-ID: From: Lars-Dominik Braun Date: Sun, 15 Dec 2024 13:22:00 +0100 Subject: [PATCH 1/4] import: pypi: Support extracting dependencies from pyproject.toml. * guix/import/pypi.scm (guess-requirements): Support extracting dependencies from pyproject.toml. * tests/pypi.scm: ("pypi->guix-package, no requires.txt, but wheel."): Renamed from "pypi->guix-package, wheels", remove requires.txt file, because the current implementation cannot detect invalid files. ("pypi->guix-package, no usable requirement file, no wheel."): Renamed from "pypi->guix-package, no usable requirement file.". (test-pyproject.toml): New variable. ("pypi->guix-package, no wheel, no requires.txt, but pyproject.toml"): New test. ("pypi->guix-package, no wheel, but requires.txt and pyproject.toml"): Ditto. Change-Id: Ib525750eb6ff4139a8209420042b28ae3c850764 --- guix/import/pypi.scm | 74 +++++++++++++++++++++++-------- tests/pypi.scm | 101 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 152 insertions(+), 23 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 7b9f54a200..7915d65d23 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -57,6 +57,7 @@ (define-module (guix import pypi) #:use-module (guix import utils) #:use-module (guix import json) #:use-module (json) + #:use-module (guix build toml) #:use-module (guix packages) #:use-module (guix upstream) #:use-module ((guix licenses) #:prefix license:) @@ -386,7 +387,42 @@ (define (guess-requirements source-url wheel-url archive) (if wheel-url (and (url-fetch wheel-url temp) (read-wheel-metadata temp)) - #f)))) + (list '() '()))))) + + (define (guess-requirements-from-pyproject.toml dir) + (let* ((pyproject.toml-files (find-files dir (lambda (abs-file-name _) + (string-match "/pyproject.toml$" + abs-file-name)))) + (pyproject.toml (match pyproject.toml-files + (() + (warning (G_ "Cannot guess requirements from \ +pyproject.toml file, because it does not exist.~%")) + '()) + (else (parse-toml-file (first pyproject.toml-files))))) + (pyproject-build-requirements + (or (recursive-assoc-ref pyproject.toml '("build-system" "requires")) '())) + (pyproject-dependencies + (or (recursive-assoc-ref pyproject.toml '("project" "dependencies")) '())) + ;; This is more of a convention, since optional-dependencies is a table of arbitrary values. + (pyproject-test-dependencies + (or (recursive-assoc-ref pyproject.toml '("project" "optional-dependencies" "test")) '()))) + (if (null? pyproject.toml) + #f + (list (map specification->requirement-name pyproject-dependencies) + (map specification->requirement-name + (append pyproject-build-requirements + pyproject-test-dependencies)))))) + + (define (guess-requirements-from-requires.txt dir) + (let ((requires.txt-files (find-files dir (lambda (abs-file-name _) + (string-match "\\.egg-info/requires.txt$" + abs-file-name))))) + (match requires.txt-files + (() + (warning (G_ "Cannot guess requirements from source archive: \ +no requires.txt file found.~%")) + #f) + (else (parse-requires.txt (first requires.txt-files)))))) (define (guess-requirements-from-source) ;; Return the package's requirements by guessing them from the source. @@ -398,27 +434,29 @@ (define (guess-requirements source-url wheel-url archive) (if (string=? "zip" (file-extension source-url)) (invoke "unzip" archive "-d" dir) (invoke "tar" "xf" archive "-C" dir))) - (let ((requires.txt-files - (find-files dir (lambda (abs-file-name _) - (string-match "\\.egg-info/requires.txt$" - abs-file-name))))) - (match requires.txt-files - (() - (warning (G_ "Cannot guess requirements from source archive:\ - no requires.txt file found.~%")) - (list '() '())) - (else (parse-requires.txt (first requires.txt-files))))))) + (list (guess-requirements-from-pyproject.toml dir) + (guess-requirements-from-requires.txt dir)))) (begin (warning (G_ "Unsupported archive format; \ cannot determine package dependencies from source archive: ~a~%") (basename source-url)) - (list '() '())))) - - ;; First, try to compute the requirements using the wheel, else, fallback to - ;; reading the "requires.txt" from the egg-info directory from the source - ;; archive. - (or (guess-requirements-from-wheel) - (guess-requirements-from-source))) + (list #f #f)))) + + (define (merge a b) + "Given lists A and B with two iteams each, combine A1 and B1, as well as A2 and B2." + (match (list a b) + (((first-propagated first-native) (second-propagated second-native)) + (list (append first-propagated second-propagated) (append first-native second-native))))) + + ;; requires.txt and the metadata of a wheel contain redundant information, + ;; so fetch only one of them, preferring requires.txt from the source + ;; distribution, which we always fetch, since the source tarball also + ;; contains pyproject.toml. + (match (guess-requirements-from-source) + ((from-pyproject.toml #f) + (merge (or from-pyproject.toml '(() ())) (or (guess-requirements-from-wheel) '(() ())))) + ((from-pyproject.toml from-requires.txt) + (merge (or from-pyproject.toml '(() ())) from-requires.txt)))) (define (compute-inputs source-url wheel-url archive) "Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, return diff --git a/tests/pypi.scm b/tests/pypi.scm index c9aee34d8b..fe00e429b7 100644 --- a/tests/pypi.scm +++ b/tests/pypi.scm @@ -112,6 +112,20 @@ (define test-requires.txt-beaker "\ coverage ") +(define test-pyproject.toml "\ +[build-system] +requires = [\"dummy-build-dep-a\", \"dummy-build-dep-b\"] + +[project] +dependencies = [ + \"dummy-dep-a\", + \"dummy-dep-b\", +] + +[project.optional-dependencies] +test = [\"dummy-test-dep-a\", \"dummy-test-dep-b\"] +") + (define test-metadata "\ Classifier: Programming Language :: Python :: 3.7 Requires-Dist: baz ~= 3 @@ -325,13 +339,90 @@ (define-syntax-rule (with-pypi responses body ...) (x (pk 'fail x #f)))))) +(test-assert "pypi->guix-package, no wheel, no requires.txt, but pyproject.toml" + (let ((tarball (pypi-tarball + "foo-1.0.0" + `(("pyproject.toml" ,test-pyproject.toml)))) + (twice (lambda (lst) (append lst lst)))) + (with-pypi (twice `(("/foo-1.0.0.tar.gz" 200 ,(file-dump tarball)) + ("/foo-1.0.0-py2.py3-none-any.whl" 404 "") + ("/foo/json" 200 ,(lambda (port) + (display (foo-json) port))))) + ;; Not clearing the memoization cache here would mean returning the value + ;; computed in the previous test. + (invalidate-memoization! pypi->guix-package) + (match (pypi->guix-package "foo") + (`(package + (name "python-foo") + (version "1.0.0") + (source (origin + (method url-fetch) + (uri (pypi-uri "foo" version)) + (sha256 + (base32 ,(? string? hash))))) + (build-system pyproject-build-system) + (propagated-inputs (list python-dummy-dep-a python-dummy-dep-b)) + (native-inputs (list python-dummy-build-dep-a python-dummy-build-dep-b + python-dummy-test-dep-a python-dummy-test-dep-b)) + (home-page "http://example.com") + (synopsis "summary") + (description "summary.") + (license license:lgpl2.0)) + (and (string=? default-sha256/base32 hash) + (equal? (pypi->guix-package "foo" #:version "1.0.0") + (pypi->guix-package "foo")) + (guard (c ((error? c) #t)) + (pypi->guix-package "foo" #:version "42")))) + (x + (pk 'fail x #f)))))) + +(test-assert "pypi->guix-package, no wheel, but requires.txt and pyproject.toml" + (let ((tarball (pypi-tarball + "foo-1.0.0" + `(("foo-1.0.0/pyproject.toml" ,test-pyproject.toml) + ("foo-1.0.0/bizarre.egg-info/requires.txt" + ,test-requires.txt)))) + (twice (lambda (lst) (append lst lst)))) + (with-pypi (twice `(("/foo-1.0.0.tar.gz" 200 ,(file-dump tarball)) + ("/foo-1.0.0-py2.py3-none-any.whl" 404 "") + ("/foo/json" 200 ,(lambda (port) + (display (foo-json) port))))) + ;; Not clearing the memoization cache here would mean returning the value + ;; computed in the previous test. + (invalidate-memoization! pypi->guix-package) + (match (pypi->guix-package "foo") + (`(package + (name "python-foo") + (version "1.0.0") + (source (origin + (method url-fetch) + (uri (pypi-uri "foo" version)) + (sha256 + (base32 ,(? string? hash))))) + (build-system pyproject-build-system) + ;; Information from requires.txt and pyproject.toml is combined. + (propagated-inputs (list python-bar python-dummy-dep-a python-dummy-dep-b + python-foo)) + (native-inputs (list python-dummy-build-dep-a python-dummy-build-dep-b + python-dummy-test-dep-a python-dummy-test-dep-b + python-pytest)) + (home-page "http://example.com") + (synopsis "summary") + (description "summary.") + (license license:lgpl2.0)) + (and (string=? default-sha256/base32 hash) + (equal? (pypi->guix-package "foo" #:version "1.0.0") + (pypi->guix-package "foo")) + (guard (c ((error? c) #t)) + (pypi->guix-package "foo" #:version "42")))) + (x + (pk 'fail x #f)))))) + (test-skip (if (which "zip") 0 1)) -(test-assert "pypi->guix-package, wheels" +(test-assert "pypi->guix-package, no requires.txt, but wheel." (let ((tarball (pypi-tarball "foo-1.0.0" - '(("foo-1.0.0/foo.egg-info/requires.txt" - "wrong data \ -to make sure we're testing wheels")))) + '(("foo-1.0.0/foo.egg-info/.empty" "")))) (wheel (wheel-file "foo-1.0.0" `(("METADATA" ,test-metadata))))) (with-pypi `(("/foo-1.0.0.tar.gz" 200 ,(file-dump tarball)) @@ -362,7 +453,7 @@ (define-syntax-rule (with-pypi responses body ...) (x (pk 'fail x #f)))))) -(test-assert "pypi->guix-package, no usable requirement file." +(test-assert "pypi->guix-package, no usable requirement file, no wheel." (let ((tarball (pypi-tarball "foo-1.0.0" '(("foo.egg-info/.empty" ""))))) (with-pypi `(("/foo-1.0.0.tar.gz" 200 ,(file-dump tarball)) base-commit: cfd4f56f75a20b6732d463180d211f796c9032e5 -- 2.45.2 --kpsUVzBC96Nk7Jsm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0002-import-pypi-Add-python-wheel-to-native-inputs-if-set.patch" >From 0abdb392bf10a99291114fc7e162a3845f25c696 Mon Sep 17 00:00:00 2001 Message-ID: <0abdb392bf10a99291114fc7e162a3845f25c696.1734278914.git.lars@6xq.net> In-Reply-To: References: From: Lars-Dominik Braun Date: Sun, 15 Dec 2024 13:30:59 +0100 Subject: [PATCH 2/4] import: pypi: Add python-wheel to native inputs if setuptools is used. * guix/import/pypi.scm (compute-inputs): Add missing python-wheel if necessary. Change-Id: Iedad213a6684856e48349289c4d9beba953f396b --- guix/import/pypi.scm | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 7915d65d23..52ec6e4ee6 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -470,12 +470,18 @@ (define (compute-inputs source-url wheel-url archive) (type type)))) (sort deps string-ci. (let ((dependencies (guess-requirements source-url wheel-url archive))) (match dependencies ((propagated native) (append (requirements->upstream-inputs propagated 'propagated) - (requirements->upstream-inputs native 'native)))))) + (requirements->upstream-inputs (add-missing-native-inputs native) 'native)))))) (define* (pypi-package-inputs pypi-package #:optional version) "Return the list of for PYPI-PACKAGE. This procedure -- 2.45.2 --kpsUVzBC96Nk7Jsm Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="0003-import-pypi-Default-to-setuptools-as-build-system-in.patch" Content-Transfer-Encoding: 8bit >From 0c9708bf7b387f2100cdf375353982fbca9b364e Mon Sep 17 00:00:00 2001 Message-ID: <0c9708bf7b387f2100cdf375353982fbca9b364e.1734278914.git.lars@6xq.net> In-Reply-To: References: From: Lars-Dominik Braun Date: Sun, 15 Dec 2024 16:56:53 +0100 Subject: [PATCH 3/4] import: pypi: Default to setuptools as build system input. * guix/import/pypi.scm (guess-requirements): Default to setuptools if pyproject.toml does not exist. Change-Id: I600bd0a44342847878e3a2a7041bd7e7c7d30769 --- guix/import/pypi.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 52ec6e4ee6..bba7361307 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -448,15 +448,21 @@ (define (guess-requirements source-url wheel-url archive) (((first-propagated first-native) (second-propagated second-native)) (list (append first-propagated second-propagated) (append first-native second-native))))) + (define default-pyproject.toml-dependencies + ;; If there is no pyproject.toml, we assume it’s an old-style setuptools-based project. + '(() ("setuptools"))) + ;; requires.txt and the metadata of a wheel contain redundant information, ;; so fetch only one of them, preferring requires.txt from the source ;; distribution, which we always fetch, since the source tarball also ;; contains pyproject.toml. (match (guess-requirements-from-source) ((from-pyproject.toml #f) - (merge (or from-pyproject.toml '(() ())) (or (guess-requirements-from-wheel) '(() ())))) + (merge (or from-pyproject.toml default-pyproject.toml-dependencies) + (or (guess-requirements-from-wheel) '(() ())))) ((from-pyproject.toml from-requires.txt) - (merge (or from-pyproject.toml '(() ())) from-requires.txt)))) + (merge (or from-pyproject.toml default-pyproject.toml-dependencies) + from-requires.txt)))) (define (compute-inputs source-url wheel-url archive) "Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, return -- 2.45.2 --kpsUVzBC96Nk7Jsm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0004-import-pypi-Move-deduplication-to-final-processing-s.patch" >From 8ab434690c870deb95bfbf61adc60a6a38d084bb Mon Sep 17 00:00:00 2001 Message-ID: <8ab434690c870deb95bfbf61adc60a6a38d084bb.1734278914.git.lars@6xq.net> In-Reply-To: References: From: Lars-Dominik Braun Date: Sun, 15 Dec 2024 17:02:44 +0100 Subject: [PATCH 4/4] import: pypi: Move deduplication to final processing step. * guix/import/pypi.scm (parse-requires.txt): Remove deduplication. (parse-wheel-metadata): Remove deduplication. (compute-inputs): Instead do it here on all the collected inputs. Change-Id: I2504cc693e9bf2e4cc44fd37b5823904dbaaa925 --- guix/import/pypi.scm | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index bba7361307..530b7d6879 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -283,12 +283,7 @@ (define (parse-requires.txt requires.txt) (let ((line (read-line port))) (cond ((eof-object? line) - ;; Duplicates can occur, since the same requirement can be - ;; listed multiple times with different conditional markers, e.g. - ;; pytest >= 3 ; python_version >= "3.3" - ;; pytest < 3 ; python_version < "3.3" - (map (compose reverse delete-duplicates) - (list required-deps test-deps))) + (list required-deps test-deps)) ((or (string-null? line) (comment? line)) (loop required-deps test-deps inside-test-section? optional?)) ((section-header? line) @@ -342,8 +337,7 @@ (define (parse-wheel-metadata metadata) (let ((line (read-line port))) (cond ((eof-object? line) - (map (compose reverse delete-duplicates) - (list required-deps test-deps))) + (list required-deps test-deps)) ((and (requires-dist-header? line) (not (extra? line))) (loop (cons (specification->requirement-name (requires-dist-value line)) @@ -486,8 +480,10 @@ (define (compute-inputs source-url wheel-url archive) (let ((dependencies (guess-requirements source-url wheel-url archive))) (match dependencies ((propagated native) - (append (requirements->upstream-inputs propagated 'propagated) - (requirements->upstream-inputs (add-missing-native-inputs native) 'native)))))) + (append (requirements->upstream-inputs (delete-duplicates propagated) + 'propagated) + (requirements->upstream-inputs (delete-duplicates (add-missing-native-inputs native)) + 'native)))))) (define* (pypi-package-inputs pypi-package #:optional version) "Return the list of for PYPI-PACKAGE. This procedure -- 2.45.2 --kpsUVzBC96Nk7Jsm--