* Re: Supporting upstream supported Python versions [not found] <mailman.562.1652773501.1048.guix-devel@gnu.org> @ 2022-06-01 7:50 ` Ricardo Wurmus 2022-09-03 10:39 ` zimoun 0 siblings, 1 reply; 5+ messages in thread From: Ricardo Wurmus @ 2022-06-01 7:50 UTC (permalink / raw) To: guix-devel; +Cc: jgart Hi, > From: jgart <jgart@dismail.de> > To: Guix Devel <guix-devel@gnu.org> > Subject: Supporting upstream supported Python versions > Message-ID: <20220517003848.GB18763@gac> […] > What is the opinion on supporting current upstream supported versions > of python? > > The master branch only has 3.9 but I'd like to have substitutes available > for 3.7, 3.8 and 3.10: The Guix Past channel now also has python-3.8. I recently needed to be able to build a profile with Python packages that were built with Python 3.8, but without replacing each and every instance of Python (because I don’t want to build librsvg, gtk+, etc). Here’s the manifest I came up with: --8<---------------cut here---------------start------------->8--- ;;; Manifest to build the latest Python packages with Python 3.8. (use-modules (guix packages) ; for "package", "package-arguments"... (guix build-system python) (guix utils) (ice-9 match)) ;; The list of Python packages (or rather specifications) that we want ;; to build with an older Python. (define packages (list "python-numpy" "python-pytorch" "python-matplotlib" "python-scipy" "python-scikit-learn" "python-seaborn")) \f (define old-python (specification->package "python@3.8")) (define old-python-wrapper ;; We use wrap-python3 to create a "python" executable. Python ;; itself only comes with "python3". Python 3.8 is available in the ;; guix-past channel. ((@@ (gnu packages python) wrap-python3) old-python)) (define (python-package? package) (or (eq? python-build-system (package-build-system package)) ;; Special cases: packages that produce Python modules but don't ;; use the Python build system. (member (package-name package) (list "pybind11")))) ;; I'd love to just use modify-inputs here, but this python-wrapper vs ;; python complication forces me to do this manually. (define (replace-python-in-inputs inputs python-wrapper python) "Some packages have an explicit Python in the inputs, or use a separate output of the Python package. We use PYTHON-WRAPPER wherever the \"python\" executable may be needed and the plain PYTHON package where we need selected outputs." (map (match-lambda (("python" package out) `("python" ,python ,out)) (("python" package) `("python" ,python-wrapper)) (anything anything)) inputs)) (define (package-with-different-python pkg python-wrapper python) "Return a new package based on PKG that uses PYTHON during the build." (package/inherit pkg (arguments (if (eq? (package-build-system pkg) python-build-system) (ensure-keyword-arguments (package-arguments pkg) `(#:python ,python-wrapper #:tests? #false)) ;running tests is slow, so why bother? (package-arguments pkg))) (native-inputs (replace-python-in-inputs (package-native-inputs pkg) python-wrapper python)) (inputs (replace-python-in-inputs (package-inputs pkg) python-wrapper python)) (propagated-inputs (replace-python-in-inputs (package-propagated-inputs pkg) python-wrapper python)))) ;; This is a recursive package transformer. When given a package ;; "pkg" it checks if it is a Python package by looking at its build ;; system; if that is the case, it will return a package variant that ;; is built with the old Python. It does this recursively, so all ;; dependencies are also modified. (define use-old-python (package-mapping (lambda (pkg) (if (python-package? pkg) (let ((modified (package-with-different-python pkg old-python-wrapper old-python))) (match (package-name pkg) ;; This package also needs a newer version of setuptools; ;; Python 3.8 comes with an older version of setuptools. ((or "python-importlib-metadata" "python-ipython") (package/inherit modified (native-inputs (modify-inputs (package-native-inputs modified) (prepend (specification->package "python-setuptools")))))) ;; This package expects typing.py to export ;; _SpecialGenericAlias, but this version of Python does ;; not define it. ("python-typing-inspect" (package/inherit modified (arguments (substitute-keyword-arguments (package-arguments modified) ((#:phases phases '%standard-phases) `(modify-phases ,phases (add-after 'unpack 'do-not-import-SpecialGenericAlias (lambda _ (substitute* "typing_inspect.py" (("_SpecialGenericAlias") "_GenericAlias")))))))))) ;; The other packages don't need special treatment. (else modified))) pkg)) ;; Stop recursion when we hit a package that is not considered a ;; Python package. This could be a package that uses Python or ;; Python modules as inputs but does not itself produce a Python ;; module. This is to avoid needlessly rebuilding big things like ;; GTK+. (negate python-package?) #:deep? #false)) ;; Apply the transformer to the list of packages. (define python-packages-with-old-python (map (compose use-old-python specification->package) packages)) ;; Build a manifest from the list of modified packages and the old ;; Python variant itself. (packages->manifest (cons old-python python-packages-with-old-python)) --8<---------------cut here---------------end--------------->8--- -- Ricardo ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Supporting upstream supported Python versions 2022-06-01 7:50 ` Supporting upstream supported Python versions Ricardo Wurmus @ 2022-09-03 10:39 ` zimoun 0 siblings, 0 replies; 5+ messages in thread From: zimoun @ 2022-09-03 10:39 UTC (permalink / raw) To: Ricardo Wurmus, guix-devel; +Cc: jgart Hi Ricardo, I am late to the party. :-) On Wed, 01 Jun 2022 at 09:50, Ricardo Wurmus <rekado@elephly.net> wrote: > Here’s the manifest I came up with: By curiosity, why does package-with-explicit-python from (guix build-system python) not fit the bill? Cheers, simon ^ permalink raw reply [flat|nested] 5+ messages in thread
* Supporting upstream supported Python versions @ 2022-05-17 5:38 jgart 2022-05-17 7:26 ` zimoun 2022-05-17 7:44 ` Tobias Geerinckx-Rice 0 siblings, 2 replies; 5+ messages in thread From: jgart @ 2022-05-17 5:38 UTC (permalink / raw) To: Guix Devel Hi Guixers, What is the opinion on supporting current upstream supported versions of python? The master branch only has 3.9 but I'd like to have substitutes available for 3.7, 3.8 and 3.10: https://endoflife.date/python Today, I needed python 3.8 and I had to resort to using asdf (https://asdf-vm.com/) to get a binary. I'd prefer to use Guix for getting all my needed python versions as substitutes. I think that providing this is important if we want Guix to be a competitive tool in the python ecosystem. Could we make all upstream supported versions of python available in master? This is currently the experience for ruby having various supported versions in Guix master. I'd like to have available all supported python versions also. I'm happy to work on this. wdyt ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Supporting upstream supported Python versions 2022-05-17 5:38 jgart @ 2022-05-17 7:26 ` zimoun 2022-05-17 7:44 ` Tobias Geerinckx-Rice 1 sibling, 0 replies; 5+ messages in thread From: zimoun @ 2022-05-17 7:26 UTC (permalink / raw) To: jgart, Guix Devel Hi, On Tue, 17 May 2022 at 00:38, jgart <jgart@dismail.de> wrote: > What is the opinion on supporting current upstream supported versions > of python? > > The master branch only has 3.9 but I'd like to have substitutes available > for 3.7, 3.8 and 3.10: By substitutes, do you mean python@x.y? Or all python-* packages compiled with python@x.y instead of python@3.9? > Could we make all upstream supported versions of python available in > master? From my point of view, we can support several version of the CPython VM, say python@3.10 (python-next), python@3.9 (current default), and various older CPython interpreters as python@3.8, 3.7, say. But only the substitutes for python-* packages compiled with the current default (here python@3.9). Somehow, we do not have the manpower (I do not know about he build farm) for maintaining more. For instance, give a look at: <https://ci.guix.gnu.org/eval/304888/dashboard> and type “python-”; well, it’s not so bad but it is not all green though. Somehow, we have the experience of maintaining 2 versions (CPython@3 and CPython@2 ;-)) and it is a piece of work to maintain, so it would be really similar when multiplying the number of CPython@3.x. Note, however, it is easy to apply ’package-with-explicit-python’ from ’(guix build-system python). Well, IMHO, supporting several version of CPython is good idea. :-) Cheers, simon ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Supporting upstream supported Python versions 2022-05-17 5:38 jgart 2022-05-17 7:26 ` zimoun @ 2022-05-17 7:44 ` Tobias Geerinckx-Rice 1 sibling, 0 replies; 5+ messages in thread From: Tobias Geerinckx-Rice @ 2022-05-17 7:44 UTC (permalink / raw) To: guix-devel, jgart, Guix Devel On 17 May 2022 05:38:48 UTC, jgart <jgart@dismail.de> wrote: >What is the opinion on supporting current upstream supported versions >of python? > >The master branch only has 3.9 but I'd like to have substitutes available >for 3.7, 3.8 and 3.10: Go for it if you're willing to maintain them. Kind regards, T G-R Sent on the go. Excuse or enjoy my brevity. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-09-03 10:43 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <mailman.562.1652773501.1048.guix-devel@gnu.org> 2022-06-01 7:50 ` Supporting upstream supported Python versions Ricardo Wurmus 2022-09-03 10:39 ` zimoun 2022-05-17 5:38 jgart 2022-05-17 7:26 ` zimoun 2022-05-17 7:44 ` Tobias Geerinckx-Rice
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).