all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
@ 2022-12-13 18:00 Roman Scherer
  2022-12-20 17:07 ` Guillaume Le Vaillant
  0 siblings, 1 reply; 7+ messages in thread
From: Roman Scherer @ 2022-12-13 18:00 UTC (permalink / raw)
  To: 60039


[-- Attachment #1.1: Type: text/plain, Size: 176 bytes --]


Hello Guix,

this patch series updates the GDAL, Netcdf and libtiff packages. It also
adds support for lz4 and openjpeg to GDAL.

Could you please review it?

Thanks, Roman.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-gdal-Update-to-3.6.0.patch --]
[-- Type: text/x-diff, Size: 1137 bytes --]

From 0fcc8aa4999271d0f50aac8bed9a9c826c88a392 Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Sat, 3 Dec 2022 18:15:04 +0100
Subject: [PATCH 1/4] gnu: gdal: Update to 3.6.0.

* gnu/packages/geo.scm (gdal): Update to 3.6.0.
---
 gnu/packages/geo.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index abfa5963b9..bfce9399c3 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1043,7 +1043,7 @@ (define-public spatialite-gui
 (define-public gdal
   (package
     (name "gdal")
-    (version "3.5.1")
+    (version "3.6.0")
     (source (origin
               (method url-fetch)
               (uri (string-append
@@ -1051,7 +1051,7 @@ (define-public gdal
                      version ".tar.gz"))
               (sha256
                (base32
-                "1l93q7xf6qx9ck5axfkf3ygmvclxkvrjd8x00ckn7j0d0750ci3w"))
+                "1nri6p495chlm63dcylq2wwmvlf90pn2kr3jf5ybkby6q3p7jshd"))
               (modules '((guix build utils)))
               (snippet
                 `(begin
-- 
2.38.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.3: 0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch --]
[-- Type: text/x-diff, Size: 775 bytes --]

From d740188e07a5368fd0e01a8aa24a6e515ecebc9e Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Sat, 3 Dec 2022 18:59:02 +0100
Subject: [PATCH 2/4] gnu: gdal: Add support for lz4 and openjpeg.

* gnu/packages/geo.scm (gdal): Add support for lz4 and openjpeg.
---
 gnu/packages/geo.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index bfce9399c3..ba348dbb7d 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1084,8 +1084,10 @@ (define-public gdal
            libpng
            libtiff
            libwebp
+           lz4
            netcdf
            openssl
+           openjpeg
            pcre2
            postgresql ; libpq
            proj
-- 
2.38.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.4: 0003-gnu-libtiff-Update-to-4.4.0.patch --]
[-- Type: text/x-diff, Size: 1137 bytes --]

From a6deb05aabbbea9b0e0ccb79887e524ddd99e589 Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Sat, 10 Dec 2022 12:16:21 +0100
Subject: [PATCH 3/4] gnu: libtiff: Update to 4.4.0.

* gnu/packages/image.scm (libtiff): Update to 4.4.0.
---
 gnu/packages/image.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/image.scm b/gnu/packages/image.scm
index bc1f25627b..9427d022d9 100644
--- a/gnu/packages/image.scm
+++ b/gnu/packages/image.scm
@@ -569,7 +569,7 @@ (define-public libicns
 (define-public libtiff
   (package
    (name "libtiff")
-   (version "4.3.0")
+   (version "4.4.0")
    (source
      (origin
        (method url-fetch)
@@ -577,7 +577,7 @@ (define-public libtiff
                            version ".tar.gz"))
        (sha256
         (base32
-         "1j3snghqjbhwmnm5vz3dr1zm68dj15mgbx1wqld7vkl7n2nfaihf"))))
+         "1vdbk3sc497c58kxmp02irl6nqkfm9rjs3br7g59m59qfnrj6wli"))))
    (build-system gnu-build-system)
    (outputs '("out"
               "doc"))                           ;1.8 MiB of HTML documentation
-- 
2.38.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.5: 0004-gnu-netcdf-Update-to-4.9.0.patch --]
[-- Type: text/x-diff, Size: 5014 bytes --]

From 4ab999dfcab8b999d6256e328667a07aa7cb5e76 Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Sat, 10 Dec 2022 12:18:20 +0100
Subject: [PATCH 4/4] gnu: netcdf: Update to 4.9.0.

* gnu/packages/maths.scm (netcdf): Update to 4.9.0.
* gnu/packages/patches/netcdf-4.9.0-fix-cmake-typo.patch
---
 gnu/packages/maths.scm                        | 46 +++++++++++++------
 .../patches/netcdf-4.9.0-fix-cmake-typo.patch | 23 ++++++++++
 2 files changed, 55 insertions(+), 14 deletions(-)
 create mode 100644 gnu/packages/patches/netcdf-4.9.0-fix-cmake-typo.patch

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index aee050735e..057e9651c2 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1850,16 +1850,16 @@ (define-public itpp
 (define-public netcdf
   (package
     (name "netcdf")
-    (version "4.7.4")
+    (version "4.9.0")
     (source
      (origin
        (method url-fetch)
        (uri (string-append
-             "https://www.unidata.ucar.edu/downloads/netcdf/ftp/"
-             "netcdf-c-" version ".tar.gz"))
+             "https://downloads.unidata.ucar.edu/netcdf-c/" version
+             "/netcdf-c-" version ".tar.gz"))
        (sha256
         (base32
-         "1a2fpp15a2rl1m50gcvvzd9y6bavl6vjf9zzf63sz5gdmq06yiqf"))
+         "0j8b814mjdqvqanzmrxpq8hn33n22cdzb3gf9vhya24wnwi615ac"))
        (modules '((guix build utils)))
        (snippet
         ;; Make sure this variable is defined only once.  Failing to do so
@@ -1867,20 +1867,35 @@ (define-public netcdf
         ;; multiple-definition link error with GCC 10.
         '(substitute* "ncdump/ocprint.c"
            (("^int ocdebug") "static int ocdebug")))
-       (patches (search-patches "netcdf-date-time.patch"))))
-    (build-system gnu-build-system)
+       (patches (search-patches "netcdf-date-time.patch"
+                                "netcdf-4.9.0-fix-cmake-typo.patch"))))
+    (build-system cmake-build-system)
     (native-inputs
-     (list m4 doxygen graphviz))
+     (list m4 doxygen graphviz unzip))
     (inputs
      `(("hdf4" ,hdf4-alt)
-       ("hdf5" ,hdf5)
+       ("hdf5" ,hdf5-1.12)
        ("curl" ,curl)
        ("zlib" ,zlib)
-       ("libjpeg" ,libjpeg-turbo)))
+       ("libjpeg" ,libjpeg-turbo)
+       ("libxml2" ,libxml2)))
     (arguments
-     `(#:configure-flags '("--enable-doxygen" "--enable-dot" "--enable-hdf4")
-
-       #:phases (modify-phases %standard-phases
+     `(#:configure-flags
+       '("-DENABLE_BYTERANGE=ON"
+         "-DENABLE_CDF5=ON"
+         "-DENABLE_DAP_LONG_TESTS=OFF"
+         "-DENABLE_DAP_REMOTE_TESTS=OFF"
+         "-DENABLE_EXAMPLE_TESTS=OFF"
+         "-DENABLE_EXTRA_TESTS=OFF"
+         "-DENABLE_FILTER_TESTING=OFF"
+         "-DENABLE_LARGE_FILE_TESTS=OFF"
+         "-DENABLE_UNIT_TESTS=OFF"
+         "-DSOURCE_DATE_EPOCH=0")
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'disable-nc-byterange-test
+           (lambda _
+             (invoke "sed" "-i" "/test_byterange/d" "nc_test/CMakeLists.txt")))
          (add-before 'configure 'fix-source-date
            (lambda _
              ;; As we ${SOURCE_DATE_EPOCH} evaluates to "1" in the build
@@ -1900,8 +1915,11 @@ (define-public netcdf
              (substitute* "libnetcdf.settings"
                (("(/gnu/store/)([0-9A-Za-z]*)" all prefix hash)
                 (string-append prefix (string-take hash 10) "...")))
-             #t)))
-
+             #t))
+         (add-before 'check 'fix-test-rcmerge
+           (lambda _
+             ;; Set HOME, to fix the test-rcmerge.
+             (setenv "HOME" "/tmp"))))
        #:parallel-tests? #f))           ;various race conditions
     (home-page "https://www.unidata.ucar.edu/software/netcdf/")
     (synopsis "Library for scientific data")
diff --git a/gnu/packages/patches/netcdf-4.9.0-fix-cmake-typo.patch b/gnu/packages/patches/netcdf-4.9.0-fix-cmake-typo.patch
new file mode 100644
index 0000000000..b9c0ed5c10
--- /dev/null
+++ b/gnu/packages/patches/netcdf-4.9.0-fix-cmake-typo.patch
@@ -0,0 +1,23 @@
+From 80a6611cfca98a5979c5ac69cba56a49af8c1e48 Mon Sep 17 00:00:00 2001
+From: Yuriy Chernyshov <thegeorg@yandex-team.com>
+Date: Sun, 19 Jun 2022 19:29:20 +0300
+Subject: [PATCH] Fix typo in CMakeLists.txt
+
+At the time generated dynamic library is named `libnetcdf.so.19}` which looks like a typo.
+---
+ CMakeLists.txt | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 7bf8641bec..18366d24ad 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -39,5 +39,5 @@ SET(NC_VERSION ${netCDF_VERSION})
+ SET(PACKAGE_VERSION ${VERSION})
+
+ # These values should match those in configure.ac
+-SET(netCDF_LIB_VERSION 19})
++SET(netCDF_LIB_VERSION 19)
+ SET(netCDF_SO_VERSION 19)
+
+ # Version of the dispatch table. This must match the value in
-- 
2.38.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 528 bytes --]

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-13 18:00 [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support Roman Scherer
@ 2022-12-20 17:07 ` Guillaume Le Vaillant
  2022-12-20 19:10   ` Roman Scherer
  0 siblings, 1 reply; 7+ messages in thread
From: Guillaume Le Vaillant @ 2022-12-20 17:07 UTC (permalink / raw)
  To: Roman Scherer; +Cc: 60039

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]

Roman Scherer <roman.scherer@burningswell.com> skribis:

> Hello Guix,
>
> this patch series updates the GDAL, Netcdf and libtiff packages. It also
> adds support for lz4 and openjpeg to GDAL.
>
> Could you please review it?
>
> Thanks, Roman.
>
> [2. text/x-diff; 0001-gnu-gdal-Update-to-3.6.0.patch]...

Hi,

Instead of this patch 1, I pushed the patch from issue 60159 that updates
gdal to 3.6.1.


> [3. text/x-diff; 0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch]...

Patch 2 pushed as 3c6f7b53cea7ea5dc8176fec02271bc3770d7fc1.


> [4. text/x-diff; 0003-gnu-libtiff-Update-to-4.4.0.patch]...

As libtiff as over 9000 dependents, it has to be updated on the
core-updates branch instead of master (see [1]). However, libtiff is
already at version 4.4.0 on core-updates, so patch 3 is not necessary
and libtiff 4.4.0 will end up in master when core-updates gets merged.

[1] https://guix.gnu.org/manual/devel/en/guix.html#Submitting-Patches


> [5. text/x-diff; 0004-gnu-netcdf-Update-to-4.9.0.patch]...

When adding or removing a patch file to "gnu/packages/patches/...", the
'dist_patch_DATA' variable in "gnu/packages/local.mk" has to be updated
to track the necessary patch files.
Moreover, as this patch fixes only one line, it is also possible to add
a custom phase with a '(substitute* ...)' form in the 'arguments' field
instead of adding a patch file.
Could you send an updated patch 4?
Thanks.

Also, many tests are disabled in 'configure-flags'. Are they not working
at all?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-20 17:07 ` Guillaume Le Vaillant
@ 2022-12-20 19:10   ` Roman Scherer
  2022-12-22 10:41     ` Guillaume Le Vaillant
  0 siblings, 1 reply; 7+ messages in thread
From: Roman Scherer @ 2022-12-20 19:10 UTC (permalink / raw)
  To: Guillaume Le Vaillant; +Cc: 60039


[-- Attachment #1.1: Type: text/plain, Size: 495 bytes --]


Hi Guillaume,

here's the updated patch for Netcdf. I checked the tests again, I
believe I left some of them off by accident after trying a couple of
things. So, I enabled some of them again. They pass. However, I left the
remote and large file tests still turned off.

I believe the remote tests we do not want anyway on Guix CICD
system. Is that correct? And the large file tests seem to take ages. Can
we ignore them, because they are super annoying?

Thanks for the review.

Wdyt?

Roman.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0005-gnu-netcdf-Update-to-4.9.0.patch --]
[-- Type: text/x-diff, Size: 3809 bytes --]

From eb2a03112532f6c1482ebaf1634cf3ac67303507 Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Tue, 20 Dec 2022 20:06:57 +0100
Subject: [PATCH] gnu: netcdf: Update to 4.9.0.

* gnu/packages/maths.scm (netcdf): Update to 4.9.0.
---
 gnu/packages/maths.scm | 50 +++++++++++++++++++++++++++++++-----------
 1 file changed, 37 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 050450e12c..a5de0f4016 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1852,16 +1852,16 @@ (define-public itpp
 (define-public netcdf
   (package
     (name "netcdf")
-    (version "4.7.4")
+    (version "4.9.0")
     (source
      (origin
        (method url-fetch)
        (uri (string-append
-             "https://www.unidata.ucar.edu/downloads/netcdf/ftp/"
-             "netcdf-c-" version ".tar.gz"))
+             "https://downloads.unidata.ucar.edu/netcdf-c/" version
+             "/netcdf-c-" version ".tar.gz"))
        (sha256
         (base32
-         "1a2fpp15a2rl1m50gcvvzd9y6bavl6vjf9zzf63sz5gdmq06yiqf"))
+         "0j8b814mjdqvqanzmrxpq8hn33n22cdzb3gf9vhya24wnwi615ac"))
        (modules '((guix build utils)))
        (snippet
         ;; Make sure this variable is defined only once.  Failing to do so
@@ -1870,19 +1870,40 @@ (define-public netcdf
         '(substitute* "ncdump/ocprint.c"
            (("^int ocdebug") "static int ocdebug")))
        (patches (search-patches "netcdf-date-time.patch"))))
-    (build-system gnu-build-system)
+    (build-system cmake-build-system)
     (native-inputs
-     (list m4 doxygen graphviz))
+     (list m4 doxygen graphviz unzip))
     (inputs
      `(("hdf4" ,hdf4-alt)
-       ("hdf5" ,hdf5)
+       ("hdf5" ,hdf5-1.12)
        ("curl" ,curl)
        ("zlib" ,zlib)
-       ("libjpeg" ,libjpeg-turbo)))
+       ("libjpeg" ,libjpeg-turbo)
+       ("libxml2" ,libxml2)))
     (arguments
-     `(#:configure-flags '("--enable-doxygen" "--enable-dot" "--enable-hdf4")
-
-       #:phases (modify-phases %standard-phases
+     `(#:configure-flags
+       '("-DENABLE_BYTERANGE=ON"
+         "-DENABLE_CDF5=ON"
+         "-DENABLE_DAP_LONG_TESTS=ON"
+         "-DENABLE_DAP_REMOTE_TESTS=OFF"
+         "-DENABLE_EXAMPLE_TESTS=ON"
+         "-DENABLE_EXTRA_TESTS=ON"
+         "-DENABLE_FILTER_TESTING=ON"
+         "-DENABLE_LARGE_FILE_TESTS=OFF"
+         "-DENABLE_UNIT_TESTS=ON"
+         "-DSOURCE_DATE_EPOCH=0")
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'fix-cmake-typo
+           (lambda _
+             ;; Fix a typo in CMakeLists.txt
+             (substitute* "CMakeLists.txt"
+               (("SET\\(netCDF_LIB_VERSION 19\\}\\)")
+                "SET(netCDF_LIB_VERSION 19)"))
+             #t))
+         (add-before 'configure 'disable-nc-byterange-test
+           (lambda _
+             (invoke "sed" "-i" "/test_byterange/d" "nc_test/CMakeLists.txt")))
          (add-before 'configure 'fix-source-date
            (lambda _
              ;; As we ${SOURCE_DATE_EPOCH} evaluates to "1" in the build
@@ -1902,8 +1923,11 @@ (define-public netcdf
              (substitute* "libnetcdf.settings"
                (("(/gnu/store/)([0-9A-Za-z]*)" all prefix hash)
                 (string-append prefix (string-take hash 10) "...")))
-             #t)))
-
+             #t))
+         (add-before 'check 'fix-test-rcmerge
+           (lambda _
+             ;; Set HOME, to fix the test-rcmerge.
+             (setenv "HOME" "/tmp"))))
        #:parallel-tests? #f))           ;various race conditions
     (home-page "https://www.unidata.ucar.edu/software/netcdf/")
     (synopsis "Library for scientific data")
-- 
2.38.1


[-- Attachment #1.3: Type: text/plain, Size: 1673 bytes --]


Guillaume Le Vaillant <glv@posteo.net> writes:

> [[PGP Signed Part:Undecided]]
> Roman Scherer <roman.scherer@burningswell.com> skribis:
>
>> Hello Guix,
>>
>> this patch series updates the GDAL, Netcdf and libtiff packages. It also
>> adds support for lz4 and openjpeg to GDAL.
>>
>> Could you please review it?
>>
>> Thanks, Roman.
>>
>> [2. text/x-diff; 0001-gnu-gdal-Update-to-3.6.0.patch]...
>
> Hi,
>
> Instead of this patch 1, I pushed the patch from issue 60159 that updates
> gdal to 3.6.1.
>
>
>> [3. text/x-diff; 0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch]...
>
> Patch 2 pushed as 3c6f7b53cea7ea5dc8176fec02271bc3770d7fc1.
>
>
>> [4. text/x-diff; 0003-gnu-libtiff-Update-to-4.4.0.patch]...
>
> As libtiff as over 9000 dependents, it has to be updated on the
> core-updates branch instead of master (see [1]). However, libtiff is
> already at version 4.4.0 on core-updates, so patch 3 is not necessary
> and libtiff 4.4.0 will end up in master when core-updates gets merged.
>
> [1] https://guix.gnu.org/manual/devel/en/guix.html#Submitting-Patches
>
>
>> [5. text/x-diff; 0004-gnu-netcdf-Update-to-4.9.0.patch]...
>
> When adding or removing a patch file to "gnu/packages/patches/...", the
> 'dist_patch_DATA' variable in "gnu/packages/local.mk" has to be updated
> to track the necessary patch files.
> Moreover, as this patch fixes only one line, it is also possible to add
> a custom phase with a '(substitute* ...)' form in the 'arguments' field
> instead of adding a patch file.
> Could you send an updated patch 4?
> Thanks.
>
> Also, many tests are disabled in 'configure-flags'. Are they not working
> at all?
>
> [[End of PGP Signed Part]]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 528 bytes --]

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-20 19:10   ` Roman Scherer
@ 2022-12-22 10:41     ` Guillaume Le Vaillant
  2022-12-22 19:07       ` Roman Scherer
  0 siblings, 1 reply; 7+ messages in thread
From: Guillaume Le Vaillant @ 2022-12-22 10:41 UTC (permalink / raw)
  To: Roman Scherer; +Cc: 60039

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

Roman Scherer <roman.scherer@burningswell.com> skribis:

> Hi Guillaume,
>
> here's the updated patch for Netcdf. I checked the tests again, I
> believe I left some of them off by accident after trying a couple of
> things. So, I enabled some of them again. They pass. However, I left the
> remote and large file tests still turned off.
>
> I believe the remote tests we do not want anyway on Guix CICD
> system. Is that correct? And the large file tests seem to take ages. Can
> we ignore them, because they are super annoying?
>
> Thanks for the review.
>
> Wdyt?
>
> Roman.
>
> [2. text/x-diff; 0005-gnu-netcdf-Update-to-4.9.0.patch]...

Hi,

The build environment doesn't have network access, so indeed remote
tests have to be disabled.

The netcdf-parallel-openmpi package fails to build with the updated
netcdf. I think it's because the package definition for
netcdf-parallel-openmpi inherits from the package definition for netcdf,
so it has to be updated to take into consideration the build-system
change of netcdf.

There are also some dependents that fail to build (cdo, python-h5netcdf,
python-meshio, qgis). I saw some error messages about conflicting
versions of hdf5, probably because in your patch netcdf uses hdf5-1.12
and the dependents also have hdf5 (v1.10) in their dependency graph in
some way. Maybe this could be fixed by using hdf5 instead of hdf5-1.12
for netcdf...

Could take a look?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-22 10:41     ` Guillaume Le Vaillant
@ 2022-12-22 19:07       ` Roman Scherer
  2022-12-23 10:58         ` bug#60039: " Guillaume Le Vaillant
  0 siblings, 1 reply; 7+ messages in thread
From: Roman Scherer @ 2022-12-22 19:07 UTC (permalink / raw)
  To: Guillaume Le Vaillant; +Cc: 60039


[-- Attachment #1.1: Type: text/plain, Size: 833 bytes --]


Hi Guillaume,

sorry about that. Here is another patch. The failed dependencies you
mentioned are working now. Another reason why netcdf-parallel-openmpi
failed was that I changed the build system previously to cmake. I went
back to use the original gnu build system. I changed it to cmake
initially because I saw that in Arch Linux's PKGBUILD and I remember
having had some problems initially. I guess it was related to the tests
I now patched.

While at the topic. Do we prefer any build system over the other in
general in Guix, like cmake vs gnu. Does one have more features than the
other (I heard something about cross compilation)?

And another question. How did you find the failing dependencies in the
first place? Did you build all the dependencies of the netcdf packages
with --sources=all?

Thanks for the help.

Roman


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0006-gnu-netcdf-Update-to-4.9.0.patch --]
[-- Type: text/x-diff, Size: 3260 bytes --]

From a5ad4273294d0d92eec1eeab2173f12bfe36b74b Mon Sep 17 00:00:00 2001
From: r0man <roman@burningswell.com>
Date: Thu, 22 Dec 2022 19:41:14 +0100
Subject: [PATCH] gnu: netcdf: Update to 4.9.0.

* gnu/packages/maths.scm (netcdf): Update to 4.9.0.
---
 gnu/packages/maths.scm | 33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 050450e12c..0d1c65636b 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1852,16 +1852,16 @@ (define-public itpp
 (define-public netcdf
   (package
     (name "netcdf")
-    (version "4.7.4")
+    (version "4.9.0")
     (source
      (origin
        (method url-fetch)
        (uri (string-append
-             "https://www.unidata.ucar.edu/downloads/netcdf/ftp/"
-             "netcdf-c-" version ".tar.gz"))
+             "https://downloads.unidata.ucar.edu/netcdf-c/" version
+             "/netcdf-c-" version ".tar.gz"))
        (sha256
         (base32
-         "1a2fpp15a2rl1m50gcvvzd9y6bavl6vjf9zzf63sz5gdmq06yiqf"))
+         "0j8b814mjdqvqanzmrxpq8hn33n22cdzb3gf9vhya24wnwi615ac"))
        (modules '((guix build utils)))
        (snippet
         ;; Make sure this variable is defined only once.  Failing to do so
@@ -1874,13 +1874,18 @@ (define-public netcdf
     (native-inputs
      (list m4 doxygen graphviz))
     (inputs
-     `(("hdf4" ,hdf4-alt)
+     `(("curl" ,curl)
+       ("hdf4" ,hdf4-alt)
        ("hdf5" ,hdf5)
-       ("curl" ,curl)
-       ("zlib" ,zlib)
-       ("libjpeg" ,libjpeg-turbo)))
+       ("libjpeg" ,libjpeg-turbo)
+       ("libxml2" ,libxml2)
+       ("unzip" ,unzip)
+       ("zlib" ,zlib)))
     (arguments
-     `(#:configure-flags '("--enable-doxygen" "--enable-dot" "--enable-hdf4")
+     `(#:configure-flags '("--enable-doxygen"
+                           "--enable-dot"
+                           "--enable-hdf4"
+                           "--disable-dap-remote-tests")
 
        #:phases (modify-phases %standard-phases
          (add-before 'configure 'fix-source-date
@@ -1891,8 +1896,7 @@ (define-public netcdf
              ;; package not reproducible.
              (substitute* "./configure"
                (("date -u -d \"\\$\\{SOURCE_DATE_EPOCH\\}\"")
-                "date --date='@0'"))
-             #t))
+                "date --date='@0'"))))
          (add-after 'configure 'patch-settings
            (lambda _
              ;; libnetcdf.settings contains the full filename of the compilers
@@ -1901,8 +1905,11 @@ (define-public netcdf
              ;; store items.
              (substitute* "libnetcdf.settings"
                (("(/gnu/store/)([0-9A-Za-z]*)" all prefix hash)
-                (string-append prefix (string-take hash 10) "...")))
-             #t)))
+                (string-append prefix (string-take hash 10) "...")))))
+         (add-before 'check 'fix-test-rcmerge
+           (lambda _
+             ;; Set HOME, to fix the test-rcmerge test.
+             (setenv "HOME" "/tmp"))))
 
        #:parallel-tests? #f))           ;various race conditions
     (home-page "https://www.unidata.ucar.edu/software/netcdf/")
-- 
2.38.1


[-- Attachment #1.3: Type: text/plain, Size: 1580 bytes --]


Guillaume Le Vaillant <glv@posteo.net> writes:

> [[PGP Signed Part:Undecided]]
> Roman Scherer <roman.scherer@burningswell.com> skribis:
>
>> Hi Guillaume,
>>
>> here's the updated patch for Netcdf. I checked the tests again, I
>> believe I left some of them off by accident after trying a couple of
>> things. So, I enabled some of them again. They pass. However, I left the
>> remote and large file tests still turned off.
>>
>> I believe the remote tests we do not want anyway on Guix CICD
>> system. Is that correct? And the large file tests seem to take ages. Can
>> we ignore them, because they are super annoying?
>>
>> Thanks for the review.
>>
>> Wdyt?
>>
>> Roman.
>>
>> [2. text/x-diff; 0005-gnu-netcdf-Update-to-4.9.0.patch]...
>
> Hi,
>
> The build environment doesn't have network access, so indeed remote
> tests have to be disabled.
>
> The netcdf-parallel-openmpi package fails to build with the updated
> netcdf. I think it's because the package definition for
> netcdf-parallel-openmpi inherits from the package definition for netcdf,
> so it has to be updated to take into consideration the build-system
> change of netcdf.
>
> There are also some dependents that fail to build (cdo, python-h5netcdf,
> python-meshio, qgis). I saw some error messages about conflicting
> versions of hdf5, probably because in your patch netcdf uses hdf5-1.12
> and the dependents also have hdf5 (v1.10) in their dependency graph in
> some way. Maybe this could be fixed by using hdf5 instead of hdf5-1.12
> for netcdf...
>
> Could take a look?
>
> [[End of PGP Signed Part]]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 528 bytes --]

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* bug#60039: Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-22 19:07       ` Roman Scherer
@ 2022-12-23 10:58         ` Guillaume Le Vaillant
  2022-12-23 11:13           ` [bug#60039] " Roman Scherer
  0 siblings, 1 reply; 7+ messages in thread
From: Guillaume Le Vaillant @ 2022-12-23 10:58 UTC (permalink / raw)
  To: Roman Scherer; +Cc: 60039-done

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

Roman Scherer <roman.scherer@burningswell.com> skribis:

> Hi Guillaume,
>
> sorry about that. Here is another patch. The failed dependencies you
> mentioned are working now. Another reason why netcdf-parallel-openmpi
> failed was that I changed the build system previously to cmake. I went
> back to use the original gnu build system. I changed it to cmake
> initially because I saw that in Arch Linux's PKGBUILD and I remember
> having had some problems initially. I guess it was related to the tests
> I now patched.
>
> While at the topic. Do we prefer any build system over the other in
> general in Guix, like cmake vs gnu. Does one have more features than the
> other (I heard something about cross compilation)?

Usually the best build system is the one that upstream developers use.
If several equivalent build systems can be used, it depends what works
best or what is easier in Guix...


> And another question. How did you find the failing dependencies in the
> first place? Did you build all the dependencies of the netcdf packages
> with --sources=all?

You can find the dependents of a package with:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix refresh -l <package-name>
--8<---------------cut here---------------end--------------->8---

So you can rebuild all the dependents of a package with something like:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix build $(./pre-inst-env guix refresh -l <package-name> | cut -d ':' -f 2)
--8<---------------cut here---------------end--------------->8---


Patch pushed as 66188398c446bdf9ce044fa539536e9b54c28c60 with a complete
commit message.
Thanks.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support
  2022-12-23 10:58         ` bug#60039: " Guillaume Le Vaillant
@ 2022-12-23 11:13           ` Roman Scherer
  0 siblings, 0 replies; 7+ messages in thread
From: Roman Scherer @ 2022-12-23 11:13 UTC (permalink / raw)
  To: Guillaume Le Vaillant; +Cc: 60039-done

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

Thank you!

On Fri, Dec 23, 2022, 12:11 Guillaume Le Vaillant <glv@posteo.net> wrote:

> Roman Scherer <roman.scherer@burningswell.com> skribis:
>
> > Hi Guillaume,
> >
> > sorry about that. Here is another patch. The failed dependencies you
> > mentioned are working now. Another reason why netcdf-parallel-openmpi
> > failed was that I changed the build system previously to cmake. I went
> > back to use the original gnu build system. I changed it to cmake
> > initially because I saw that in Arch Linux's PKGBUILD and I remember
> > having had some problems initially. I guess it was related to the tests
> > I now patched.
> >
> > While at the topic. Do we prefer any build system over the other in
> > general in Guix, like cmake vs gnu. Does one have more features than the
> > other (I heard something about cross compilation)?
>
> Usually the best build system is the one that upstream developers use.
> If several equivalent build systems can be used, it depends what works
> best or what is easier in Guix...
>
>
> > And another question. How did you find the failing dependencies in the
> > first place? Did you build all the dependencies of the netcdf packages
> > with --sources=all?
>
> You can find the dependents of a package with:
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix refresh -l <package-name>
> --8<---------------cut here---------------end--------------->8---
>
> So you can rebuild all the dependents of a package with something like:
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix build $(./pre-inst-env guix refresh -l <package-name>
> | cut -d ':' -f 2)
> --8<---------------cut here---------------end--------------->8---
>
>
> Patch pushed as 66188398c446bdf9ce044fa539536e9b54c28c60 with a complete
> commit message.
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 2427 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-12-23 11:14 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-12-13 18:00 [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg support Roman Scherer
2022-12-20 17:07 ` Guillaume Le Vaillant
2022-12-20 19:10   ` Roman Scherer
2022-12-22 10:41     ` Guillaume Le Vaillant
2022-12-22 19:07       ` Roman Scherer
2022-12-23 10:58         ` bug#60039: " Guillaume Le Vaillant
2022-12-23 11:13           ` [bug#60039] " Roman Scherer

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.