From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id gHTxALUhZmL6JQEAbAwnHQ (envelope-from ) for ; Mon, 25 Apr 2022 06:21:09 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id MNXqALUhZmIX6wAA9RJhRA (envelope-from ) for ; Mon, 25 Apr 2022 06:21:09 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 506C92A106 for ; Mon, 25 Apr 2022 06:21:08 +0200 (CEST) Received: from localhost ([::1]:42328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1niqDj-00085p-Bl for larch@yhetil.org; Mon, 25 Apr 2022 00:21:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38902) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nipvb-00080Z-M5 for guix-patches@gnu.org; Mon, 25 Apr 2022 00:02:27 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38801) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nipvZ-0001nX-PF for guix-patches@gnu.org; Mon, 25 Apr 2022 00:02:23 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nipvZ-0008FT-KN for guix-patches@gnu.org; Mon, 25 Apr 2022 00:02:21 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#55104] [PATCH 075/232] gnu: python-scipy: Update to 1.8.0 and enable parallel build. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 25 Apr 2022 04:02:21 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55104 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 55104@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 55104-submit@debbugs.gnu.org id=B55104.165085933131508 (code B ref 55104); Mon, 25 Apr 2022 04:02:21 +0000 Received: (at 55104) by debbugs.gnu.org; 25 Apr 2022 04:02:11 +0000 Received: from localhost ([127.0.0.1]:60893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nipvO-0008Bp-IQ for submit@debbugs.gnu.org; Mon, 25 Apr 2022 00:02:11 -0400 Received: from mail-qk1-f180.google.com ([209.85.222.180]:43569) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nipty-0007sV-7D for 55104@debbugs.gnu.org; Mon, 25 Apr 2022 00:00:47 -0400 Received: by mail-qk1-f180.google.com with SMTP id a186so9956107qkc.10 for <55104@debbugs.gnu.org>; Sun, 24 Apr 2022 21:00:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Cu21VibzAT5voxXRFV8prEpe+NPO19P+tqIWIlA8rlw=; b=Qsxnv2W0m0IG7NAsVCLFqzpfHC3nYFLE0WztfSzwuG2fSPtszrekfEPvOd5cGBncKX PwqhnbQ0ED4f8izzQipPc+pq71s+0kCkMh4k58DvRBuOP3qQkf1GOFYp9u4VvczhBHhq kT+f+7cBjviFCfoGcv5kp+cZLifbK663HglfvD+GR8O4fog3B0R+SPyORRNb6yscPlD8 P9Ix6orcW9U0WFnlSH7NN2Y4iWPpddDCLEhQhD94mo97IFk0FFaK3YqdyTscEavbAmD8 espx25mVq/GYqhSsQ1QUyhEL9LSh3poDRFzg6lbbuk/GSDIIKHFkxQg1+IYN/p30/ecs rJUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Cu21VibzAT5voxXRFV8prEpe+NPO19P+tqIWIlA8rlw=; b=JyPolgQwN55Q/yZoo9av1AnoTRM2X0oE/BO3eDBYFu2jBf2kPmag8BbmAMV72Irko3 0b4YK9YEJV+TlxGNfEI0TjE91/9FKcodtKK1tCNmTlrjDgnFynDU+hXW4qZR16K39SyP X6F5Am3mXNiqIrgeKzeX+PxUJBotIbv4ikP+aEX3zdlf+FhcNsnONDnkTtR1U4sYc8L8 aK7aQQon3jg65ezGDtsCHgTkXVS+BLfltGzJqsDHE8zV3FXFbMSpxvbmk91ww821FchZ bskPpplgoPFb7yDCC4jRjEI/hhQhyr9UeCsqmq7b0wWWX9X4HprlsbjgNbcuQRmDsfkr xlZw== X-Gm-Message-State: AOAM532H/f3QO8g/eqgQNSBM39ieYzF8Rp9xL7F8C8xNfshgjQhbMk4M TcZ1sRgDqtxqoiWwR+H+b34g+U0IjlOQPA== X-Google-Smtp-Source: ABdhPJy/9W/Olu3+Cw1G6zStAeSFwyGVJiV/a4ikKN4K5Hg61D30hfofBDUhRz6Zconl1MLl1MZHIw== X-Received: by 2002:a05:620a:2590:b0:680:f8c3:582 with SMTP id x16-20020a05620a259000b00680f8c30582mr8902891qko.723.1650859236594; Sun, 24 Apr 2022 21:00:36 -0700 (PDT) Received: from localhost.localdomain (dsl-150-232.b2b2c.ca. [66.158.150.232]) by smtp.gmail.com with ESMTPSA id w82-20020a376255000000b0069ee3f0ae63sm4478142qkb.45.2022.04.24.21.00.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Apr 2022 21:00:35 -0700 (PDT) From: Maxim Cournoyer Date: Sun, 24 Apr 2022 23:56:41 -0400 Message-Id: <20220425035918.25683-75-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220425035918.25683-1-maxim.cournoyer@gmail.com> References: <20220425035918.25683-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1650860468; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Cu21VibzAT5voxXRFV8prEpe+NPO19P+tqIWIlA8rlw=; b=uallrq14Kwop+VDportyoX1wYkkwHYPGMl4yB5frWNIyZftSY6NLkYl6pU3dAwMcNKN9sj wvao3wZQoIoTvmySUFHTPwFXmcmc1UAjoD8BgC+DbuaodcVSEroMFB4HlDND8fOgTGTyZ/ CCp5lh0nG5LmIWciNMRYQ2INoGc0x/Y6Mg1yHLZHZ6Gr4rnlLjxata//u2yYI4tQJYipRJ eZGRapjK7t+OaKg1KkXhovg7u36QGboBH0zab4jZ7tofXIxLdZuGBRzZEIhzc2i98OkyAV PTKz/Hp/6vxXkmeV14tggxhmHcIy9qDLhJzkQPDvNYXX32CE659AqjBMkSOYnQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1650860468; a=rsa-sha256; cv=none; b=lZwyoxhP7l1hqKhxKHnZ62R7tEuvPJCtza6VjhUhxFT5I9brUUqdTEQxsXdBPmJQ/4B3O8 B+tZahFHDdfyyYSUJ80XZTAfX2/kTwe7NoG+lO5A5I/A8szhDL+2ekhW+AIuqu/QEW6b9S 5iacFjejStjH6w/uj14pg48Wo+woASGvX9fU9Hvbubu/xnbmqOC2y4MPqDh8Shd5AUhXEG /kzAHPCEpov1Fl0V4ykZs8D1Z3wxpqtSybdCeqlZTWeFZHVUbrCVyoLhgpQxduHw5g1woo X475T0aWeTXlDp+xmjJWByMKN7fJWstrL0zC3x0H7fYH1y0yFyIOclSQFF9Z3g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Qsxnv2W0; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 7.19 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Qsxnv2W0; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 506C92A106 X-Spam-Score: 7.19 X-Migadu-Scanner: scn0.migadu.com X-TUID: 5oysXiZCQTt7 * gnu/packages/python-science.scm (python-scipy): Update to 1.8.0. [modules]: New argument. [phases]{disable-broken-tests}: Delete phase. {configure-openblas}: Streamline configuration template. {parallelize-build}: New phase. {install-doc}: Move before 'check phase. Remove now extraneous 'add-installed-pythonpath' call. Remove the SPHINXBUILD and PYVER Make variables, as well as conf.py patching. Provide SPHINXOPTS to build doc in parallel. Simply recursively copy the 'html' output directory to install the doc. {check}: Remove extraneous INPUTS and OUTPUTS arguments. Start tests with the 'runtests.py' launcher, which accepts a '-j' argument to run the tests in parallel. [native-inputs]: Sort inputs. Add python-pytest-xdist and python-threadpoolctl. [home-page]: Update. --- gnu/packages/python-science.scm | 91 +++++++++++++-------------------- 1 file changed, 36 insertions(+), 55 deletions(-) diff --git a/gnu/packages/python-science.scm b/gnu/packages/python-science.scm index dfe6a66e61..1852a16047 100644 --- a/gnu/packages/python-science.scm +++ b/gnu/packages/python-science.scm @@ -72,17 +72,20 @@ (define-module (gnu packages python-science) (define-public python-scipy (package (name "python-scipy") - (version "1.7.3") + (version "1.8.0") (source (origin (method url-fetch) (uri (pypi-uri "scipy" version)) (sha256 - (base32 "1gxsnw6viz2j3sm8ak2a8l7fcn4b2zm3kzfm8w57xxyyrzx7an5b")))) + (base32 "1gghkwn93niyasm36333xbqrnn3yiadq9d97wnc9mg14nzbg5m1i")))) (outputs '("out" "doc")) (build-system python-build-system) (arguments (list + #:modules '((guix build utils) + (guix build python-build-system) + (ice-9 format)) #:phases #~(modify-phases %standard-phases (add-after 'unpack 'disable-pythran @@ -92,81 +95,59 @@ (define-public python-scipy (lambda _ ;; Change from /homeless-shelter to /tmp for write permission. (setenv "HOME" "/tmp"))) - (add-after 'unpack 'disable-broken-tests - (lambda _ - (substitute* "scipy/sparse/linalg/dsolve/tests/test_linsolve.py" - (("^( +)def test_threads_parallel\\(self\\):" m indent) - (string-append indent - "@pytest.mark.skip(reason=\"Disabled by Guix\")\n" - m))) - (substitute* "scipy/sparse/linalg/eigen/arpack/tests/test_arpack.py" - (("^def test_parallel_threads\\(\\):" m) - (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n" - m))))) (add-before 'build 'configure-openblas (lambda _ (call-with-output-file "site.cfg" (lambda (port) (format port - "[blas] + "\ +[blas] libraries = openblas library_dirs = ~a/lib -include_dirs = ~a/include +include_dirs = ~:*~a/include -# backslash-n to make emacs happy -\n[atlas] -library_dirs = ~a/lib -atlas_libs = openblas -" - #$(this-package-input "openblas") - #$(this-package-input "openblas") - #$(this-package-input "openblas")))))) - (add-after 'install 'install-doc - (lambda* (#:key inputs outputs #:allow-other-keys) +[atlas] +library_dirs = ~:*~a/lib +atlas_libs = openblas~%" #$(this-package-input "openblas")))))) + (add-before 'build 'parallelize-build + (lambda _ + (setenv "NPY_NUM_BUILD_JOBS" + (number->string (parallel-job-count))))) + (add-before 'check 'install-doc + (lambda* (#:key outputs #:allow-other-keys) (let* ((data (string-append (assoc-ref outputs "doc") "/share")) (doc (string-append data "/doc/" #$name "-" #$version)) - (html (string-append doc "/html")) - (pyver (string-append "PYVER=" - #$(version-major+minor - (package-version python)))) - ;; By default it tries to run sphinx-build through the Python - ;; interpreter which won't work with our shell wrapper. - (sphinxbuild "SPHINXBUILD=LANG=C sphinx-build")) - ;; Make installed package available for building the - ;; documentation - (add-installed-pythonpath inputs outputs) + (html (string-append doc "/html"))) (with-directory-excursion "doc" - ;; Fix generation of images for mathematical expressions. - (substitute* (find-files "source" "conf\\.py") - (("pngmath_use_preview = True") - "pngmath_use_preview = False")) + ;; Build doc. + (invoke "make" "html" + ;; Building the documentation takes a very long time. + ;; Parallelize it. + (string-append "SPHINXOPTS=-j" + (number->string (parallel-job-count)))) + ;; Install doc. (mkdir-p html) - (invoke "make" "html" pyver sphinxbuild) - (with-directory-excursion "build/html" - (for-each (lambda (file) - (let* ((dir (dirname file)) - (tgt-dir (string-append html "/" dir))) - (install-file file html))) - (find-files "."))))))) + (copy-recursively "build/html" html))))) (replace 'check - (lambda* (#:key tests? inputs outputs #:allow-other-keys) + (lambda* (#:key tests? #:allow-other-keys) (when tests? - (with-directory-excursion "/tmp" - (invoke "python" "-c" - "import scipy; scipy.test(verbose=2)")))))))) + (invoke "./runtests.py" "-vv" "--no-build" "--mode=fast" + "-j" (number->string (parallel-job-count))))))))) (propagated-inputs (list python-numpy python-matplotlib python-pyparsing)) (inputs (list openblas pybind11)) (native-inputs - (list python-cython + (list gfortran + perl + python-cython + python-numpydoc python-pydata-sphinx-theme python-pytest + python-pytest-xdist python-sphinx python-sphinx-panels - python-numpydoc - gfortran - perl + python-threadpoolctl which)) - (home-page "https://www.scipy.org/") + (home-page "https://scipy.org/") (synopsis "The Scipy library provides efficient numerical routines") (description "The SciPy library is one of the core packages that make up the SciPy stack. It provides many user-friendly and efficient numerical -- 2.34.0