[bug#48046] [PATCH]: Gnu add astropy

[Vinicius Monego <monego@posteo.net>]

[-- Attachment #1: Type: text/plain, Size: 2539 bytes --]

Hi Sharlatan,

Thanks for continuing the work on the astropy package, I managed to
finish it this time. I am resending your patch with the following
modifications:

- Moved the package definition from the bottom to the middle of the
file (to avoid merge conflicts)
- Removed all optional inputs and propagated the remaining. I left only
those listed in install_requires, setup_requires, test_requires and
test[extras] in setup.cfg
- Changed synopsis and description
- Changed package labels to match the package name    
- Made the compiler file writable instead of deleting it
- Deleted the makdir-astropy phase (it wasn't needed)
- Added license for the jquery bundle that is not replaced

and then I made my own improvements on that patch: enabling tests and
unbundling some external libraries.

I removed the optional packages because astropy is a core package,
which will be a dependency for its many extensions. It's important that
it builds with a high probability of success or the chain will break.
Some of its optional dependencies, e.g. Pandas, have a broken build in
aarch64 at the moment. The "full" astropy package could be installed
easily from a manifest file and the tests can run again with
astropy.test().

> the project heavily depends on TOX which requires pip to install
> missing dependencies for itself.

I don't think that a project can heavily depend on tox, all it does is
manage a virtual environment with dependencies to run the tests. Guix
does the same so tox is redundant here. Tests will still run with the
testing framework.

Two more suggestions for future Python patches:

> +         (replace 'check
> +           (lambda* (#:key inputs outputs #:allow-other-keys)
> +             (add-installed-pythonpath inputs outputs)
> +             (invoke "pytest" "-vv")))

When a project contains tests as part of the application code, as in
Astropy, tests should run with "pytest --pyargs module". See Pytest
Integration Pratices:
https://docs.pytest.org/en/documentation-restructure/background/goodpractices.html

It's also good practice in Guix to use (when tests?) when overriding
the check phase to allow --without-tests=pkg.
  
> ImportError: You appear to be trying to import astropy from within a
> source checkout or from an editable installation without building the
> extension modules first. Either run:

I fixed this error by running the second command before the tests.

If you don't mind the modifications I did, I will call this patchset
complete and wait for a committer to review.

Vinicius

[-- Attachment #2: 0001-gnu-Add-python-astropy.patch --]
[-- Type: text/x-patch, Size: 2651 bytes --]

From d7245f0852a1a2f39bdba277b4287e2a753955a6 Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Mon, 26 Apr 2021 20:52:09 +0100
Subject: [PATCH 1/6] gnu: Add python-astropy.

* gnu/packages/astronomy.scm (python-astropy): New variable.
---
 gnu/packages/astronomy.scm | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/gnu/packages/astronomy.scm b/gnu/packages/astronomy.scm
index 6d1c4ddfe6..b32f3f7cf5 100644
--- a/gnu/packages/astronomy.scm
+++ b/gnu/packages/astronomy.scm
@@ -547,6 +547,54 @@ accurately in real time at any rate desired.")
      `(#:configure-flags '("-DENABLE_GTK=ON" "-DENABLE_QT=OFF")
        #:tests? #f))))
 
+(define-public python-astropy
+  (package
+    (name "python-astropy")
+    (version "4.2.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "astropy" version))
+       (sha256
+        (base32 "09w4q64c6bykcdp8xdq5fgsdjqrcihqhqjszqjp3s5a1493kwj7d"))))
+    (build-system python-build-system)
+    (arguments
+     ;; NOTE: (Sharlatan-20210426T204315+0100): Tests require build astropy
+     ;; module, it needs a good review on how to enable them.
+     `(#:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setenv-astropy-system-all
+           (lambda _
+             (setenv "ASTROPY_USE_SYSTEM_ALL" "1")))
+         ;; Permission denied: './astropy/_compiler.c'.
+         (add-before 'install 'writable-compiler
+           (lambda _ (make-file-writable "astropy/_compiler.c"))))))
+    (native-inputs
+     `(("pkg-config" ,pkg-config)
+       ("python-coverage" ,python-coverage)
+       ("python-cython" ,python-cython)
+       ("python-extension-helpers" ,python-extension-helpers)
+       ("python-ipython" ,python-ipython)
+       ("python-objgraph" ,python-objgraph)
+       ("python-setuptools-scm" ,python-setuptools-scm)
+       ("python-sgp4" ,python-sgp4)
+       ("python-skyfield" ,python-skyfield)))
+    (inputs
+     `(("cfitsio" ,cfitsio)
+       ("wcslib" ,wcslib)))
+    (propagated-inputs
+     `(("python-numpy" ,python-numpy)
+       ("python-pyerfa" ,python-pyerfa)))
+    (home-page "https://www.astropy.org/")
+    (synopsis "Core package for Astronomy in Python")
+    (description
+     "Astropy is a single core package for Astronomy in Python.  It contains
+much of the core functionality and some common tools needed for performing
+astronomy and astrophysics.")
+    (license (list license:bsd-3     ;astropy
+                   license:expat)))) ;jquery
+
 (define-public libnova
   (package
     (name "libnova")
-- 
2.31.1


[-- Attachment #3: 0002-gnu-python-pytest-astropy-Propagate-inputs.patch --]
[-- Type: text/x-patch, Size: 2582 bytes --]

From 7a119c602124c24555aacd0a39d634127e7b65d0 Mon Sep 17 00:00:00 2001
From: Vinicius Monego <monego@posteo.net>
Date: Sat, 22 May 2021 14:49:54 -0300
Subject: [PATCH 2/6] gnu: python-pytest-astropy: Propagate inputs.

* gnu/packages/python-check.scm (python-pytest-astropy)[arguments]: Add new
phase to skip a version check bug.
[native-inputs]: Remove python-pytest. Move python-attrs, python-hypothesis,
python-pytest-arraydiff, python-pytest-astropy-header, python-pytest-cov,
python-pytest-filter-subpackage, python-pytest-mock, python-pytest-openfiles,
python-pytest-remotedata ...
[propagated-inputs]: ... here.
---
 gnu/packages/python-check.scm | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index f0b454eec9..b75d0fb155 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -286,12 +286,20 @@ Astropy project, but is optimized for use with astropy-related projects.")
         (base32 "18j6z6y2fvykmcs5z0mldhhaxxn6wzpnhlm2ps7m8r5z5kmh1631"))))
     (build-system python-build-system)
     (arguments
-     ;; No tests provided
-     '(#:tests? #f))
+     `(#:tests? #f ; there are no tests
+       #:phases
+       (modify-phases %standard-phases
+         ;; There is a bug somewhere that makes pytest-filter-subpackage appear
+         ;; as version 0.0.0 to setup.py.  Remove it from the requirements.
+         (add-after 'unpack 'remove-requirement
+           (lambda _
+             (substitute* "setup.cfg"
+               ((".*pytest-filter-subpackage.*") "")))))))
     (native-inputs
+     `(("python-setuptools-scm" ,python-setuptools-scm)))
+    (propagated-inputs
      `(("attrs" ,python-attrs)
        ("hypothesis" ,python-hypothesis)
-       ("pytest" ,python-pytest)
        ("pytest-arraydiff" ,python-pytest-arraydiff)
        ("pytest-astropy-header" ,python-pytest-astropy-header)
        ("pytest-cov" ,python-pytest-cov)
@@ -299,8 +307,7 @@ Astropy project, but is optimized for use with astropy-related projects.")
        ("pytest-filter-subpackage" ,python-pytest-filter-subpackage)
        ("pytest-mock" ,python-pytest-mock)
        ("pytest-openfiles" ,python-pytest-openfiles)
-       ("pytest-remotedata" ,python-pytest-remotedata)
-       ("setuptools-scm" ,python-setuptools-scm)))
+       ("pytest-remotedata" ,python-pytest-remotedata)))
     (home-page "https://github.com/astropy/pytest-astropy")
     (synopsis
      "Metapackage for all the testing machinery used by the Astropy Project")
-- 
2.31.1


[-- Attachment #4: 0003-gnu-python-pyerfa-Adjust-inputs.patch --]
[-- Type: text/x-patch, Size: 1062 bytes --]

From 256038d0d9eae6509d96bd370404f30a723715f0 Mon Sep 17 00:00:00 2001
From: Vinicius Monego <monego@posteo.net>
Date: Sat, 22 May 2021 14:58:15 -0300
Subject: [PATCH 3/6] gnu: python-pyerfa: Adjust inputs.

* gnu/packages/astronomy.scm (python-pyerfa)[inputs]: Move python-numpy to ...
[propagated-inputs]: ... here.
---
 gnu/packages/astronomy.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/astronomy.scm b/gnu/packages/astronomy.scm
index b32f3f7cf5..4c5c86a3dc 100644
--- a/gnu/packages/astronomy.scm
+++ b/gnu/packages/astronomy.scm
@@ -992,8 +992,9 @@ JPL ephemerides use to predict raw (x,y,z) planetary positions.")
        ("setuptools-scm" ,python-setuptools-scm)
        ("pytest-doctestplus" ,python-pytest-doctestplus)))
     (inputs
-     `(("liberfa" ,erfa)
-       ("numpy" ,python-numpy)))
+     `(("liberfa" ,erfa)))
+    (propagated-inputs
+     `(("numpy" ,python-numpy)))
     (home-page "https://github.com/liberfa/pyerfa")
     (synopsis "Python bindings for ERFA")
     (description
-- 
2.31.1


[-- Attachment #5: 0004-gnu-Add-wcslib-7.3.patch --]
[-- Type: text/x-patch, Size: 1116 bytes --]

From fda2610f09b0849645d17b906ade3c8fdc493cb5 Mon Sep 17 00:00:00 2001
From: Vinicius Monego <monego@posteo.net>
Date: Sat, 22 May 2021 15:14:59 -0300
Subject: [PATCH 4/6] gnu: Add wcslib-7.3.

* gnu/packages/astronomy.scm (wcslib-7.3): New variable.
---
 gnu/packages/astronomy.scm | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/gnu/packages/astronomy.scm b/gnu/packages/astronomy.scm
index 4c5c86a3dc..6e0fdeb969 100644
--- a/gnu/packages/astronomy.scm
+++ b/gnu/packages/astronomy.scm
@@ -205,6 +205,20 @@ coordinate systems in a @dfn{FITS} (Flexible Image Transport System) image
 header.")
     (license license:lgpl3+)))
 
+(define-public wcslib-7.3
+  (package
+    (inherit wcslib)
+    (name "wcslib")
+    (version "7.3")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-" version
+             ".tar.bz2"))
+       (sha256
+        (base32 "0q99k61l2zh6irzkd5195aama37mlm0nivamz6j6r8l2ad1cy0ab"))))))
+
 (define-public weightwatcher
   (package
     (name "weightwatcher")
-- 
2.31.1


[-- Attachment #6: 0005-gnu-python-astropy-Enable-tests.patch --]
[-- Type: text/x-patch, Size: 3754 bytes --]

From d226f4d8560d78da88ae9ebfdc48abf443b572e0 Mon Sep 17 00:00:00 2001
From: Vinicius Monego <monego@posteo.net>
Date: Sat, 22 May 2021 17:13:35 -0300
Subject: [PATCH 5/6] gnu: python-astropy: Enable tests.

* gnu/packages/astronomy.scm (python-astropy)[arguments]: Enable tests.
[phases]{setenv-astropy-system-all}: Move to after 'unpack. Rename to
'preparations'. Set HOME to /tmp. Allow xfail tests to pass.
{writable-compiler}: New phase.
{check}: Override phase.
[native-inputs]: Add python-pytest, python-pytest-astropy, python-pytest-xdist.
[inputs]: Remove wcslib. Add wcslib-7.3.
---
 gnu/packages/astronomy.scm | 37 ++++++++++++++++++++++++++++---------
 1 file changed, 28 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/astronomy.scm b/gnu/packages/astronomy.scm
index 6e0fdeb969..d9785696bb 100644
--- a/gnu/packages/astronomy.scm
+++ b/gnu/packages/astronomy.scm
@@ -573,17 +573,31 @@ accurately in real time at any rate desired.")
         (base32 "09w4q64c6bykcdp8xdq5fgsdjqrcihqhqjszqjp3s5a1493kwj7d"))))
     (build-system python-build-system)
     (arguments
-     ;; NOTE: (Sharlatan-20210426T204315+0100): Tests require build astropy
-     ;; module, it needs a good review on how to enable them.
-     `(#:tests? #f
-       #:phases
+     `(#:phases
        (modify-phases %standard-phases
-         (add-before 'build 'setenv-astropy-system-all
+         (add-after 'unpack 'preparations
            (lambda _
-             (setenv "ASTROPY_USE_SYSTEM_ALL" "1")))
-         ;; Permission denied: './astropy/_compiler.c'.
+             ;; Use our own libraries in place of bundles.
+             (setenv "ASTROPY_USE_SYSTEM_ALL" "1")
+             ;; Some tests require a writable home.
+             (setenv "HOME" "/tmp")
+             ;; Relax xfail tests.
+             (substitute* "setup.cfg"
+               (("xfail_strict = true") "xfail_strict = false"))))
+         ;; This file is opened in both install and check phases.
          (add-before 'install 'writable-compiler
-           (lambda _ (make-file-writable "astropy/_compiler.c"))))))
+           (lambda _ (make-file-writable "astropy/_compiler.c")))
+         (add-before 'check 'writable-compiler
+           (lambda _ (make-file-writable "astropy/_compiler.c")))
+         (replace 'check
+           (lambda* (#:key inputs outputs tests? #:allow-other-keys)
+             (when tests?
+               (add-installed-pythonpath inputs outputs)
+               ;; Extensions have to be rebuilt before running the tests.
+               (invoke "python" "setup.py" "build_ext" "--inplace")
+               (invoke "python" "-m" "pytest" "--pyargs" "astropy"
+                       ;; Skip tests that need remote data.
+                       "-m" "not remote_data")))))))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("python-coverage" ,python-coverage)
@@ -591,12 +605,17 @@ accurately in real time at any rate desired.")
        ("python-extension-helpers" ,python-extension-helpers)
        ("python-ipython" ,python-ipython)
        ("python-objgraph" ,python-objgraph)
+       ("python-pytest" ,python-pytest)
+       ("python-pytest-astropy" ,python-pytest-astropy)
+       ("python-pytest-xdist" ,python-pytest-xdist)
        ("python-setuptools-scm" ,python-setuptools-scm)
        ("python-sgp4" ,python-sgp4)
        ("python-skyfield" ,python-skyfield)))
     (inputs
      `(("cfitsio" ,cfitsio)
-       ("wcslib" ,wcslib)))
+       ;; Astropy 4.2.1 bundles wcslib 7.3 and that is the only version it
+       ;; accepts.  Version 7.5 will not be validated in the build.
+       ("wcslib" ,wcslib-7.3)))
     (propagated-inputs
      `(("python-numpy" ,python-numpy)
        ("python-pyerfa" ,python-pyerfa)))
-- 
2.31.1


[-- Attachment #7: 0006-gnu-python-astropy-Unbundle-ply-and-configobj.patch --]
[-- Type: text/x-patch, Size: 2737 bytes --]

From be70ce67532c8b5c14bebdf3a4c0ce483bc86530 Mon Sep 17 00:00:00 2001
From: Vinicius Monego <monego@posteo.net>
Date: Sat, 22 May 2021 15:09:37 -0300
Subject: [PATCH 6/6] gnu: python-astropy: Unbundle ply and configobj.

* gnu/packages/astronomy.scm (python-astropy)[arguments]: Replace references
to external ply and configobj and delete the bundles.
[propagated-inputs]: Add python-ply, python-configobj.
---
 gnu/packages/astronomy.scm | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/astronomy.scm b/gnu/packages/astronomy.scm
index d9785696bb..17617eef9b 100644
--- a/gnu/packages/astronomy.scm
+++ b/gnu/packages/astronomy.scm
@@ -583,7 +583,25 @@ accurately in real time at any rate desired.")
              (setenv "HOME" "/tmp")
              ;; Relax xfail tests.
              (substitute* "setup.cfg"
-               (("xfail_strict = true") "xfail_strict = false"))))
+               (("xfail_strict = true") "xfail_strict = false"))
+             ;; Delete ply and configobj because we have them.  They are not
+             ;; covered by ASTROPY_USE_SYSTEM_ALL.
+             (with-directory-excursion "astropy/extern"
+               (for-each delete-file-recursively '("ply" "configobj")))
+             ;; Replace all references to external ply.
+             (let ((ply-files '("coordinates/angle_utilities.py"
+                                "units/format/cds.py"
+                                "units/format/ogip.py"
+                                "units/format/generic.py")))
+               (with-directory-excursion "astropy"
+                 (map (lambda (file)
+                        (substitute* file (("astropy.extern.ply")
+                                           "ply")))
+                      ply-files)))
+             ;; Replace reference to external configobj.
+             (with-directory-excursion "astropy/config"
+               (substitute* "configuration.py"
+                 (("from astropy.extern.configobj ") "")))             ))
          ;; This file is opened in both install and check phases.
          (add-before 'install 'writable-compiler
            (lambda _ (make-file-writable "astropy/_compiler.c")))
@@ -617,7 +635,9 @@ accurately in real time at any rate desired.")
        ;; accepts.  Version 7.5 will not be validated in the build.
        ("wcslib" ,wcslib-7.3)))
     (propagated-inputs
-     `(("python-numpy" ,python-numpy)
+     `(("python-configobj" ,python-configobj)
+       ("python-numpy" ,python-numpy)
+       ("python-ply" ,python-ply)
        ("python-pyerfa" ,python-pyerfa)))
     (home-page "https://www.astropy.org/")
     (synopsis "Core package for Astronomy in Python")
-- 
2.31.1