[PATCH] gnu: Add hdf4

[PATCH] gnu: Add hdf4

[Thomas Danckaert]

[-- Attachment #1: Type: Text/Plain, Size: 999 bytes --]

Hi Guix,

this patch continues the work done by Jeremy Robst to package HDF4

(https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00069.html)

 - Back then, people reported occasional build failures. I've disabled
   parallel tests and don't have any problems on my system anymore
   (maybe someone else can test?).

 - I've added a patch to allow a shared build with Fortran interface,
   and some substitutions to remove bogus RPATH settings from the
   Makefiles (otherwise, build directories are included in the RPATH).

 - I've also added a variant “hdf4-alt”: by default, HDF4 includes a
   netCDF API to access HDF4 files, which clashes with the symbols in
   the real netCDF library when you try to use both libraries in your
   program.  This variant uses the “--disable-netcdf” configure flag
   to avoid this.  We could call it “hdf4-nonetcdf”, but the name
   “hdf4-alt” is used by Debian, so maybe it's good to use the same?

best,

Thomas

[-- Attachment #2: 0001-gnu-Add-hdf4.patch --]
[-- Type: Text/X-Patch, Size: 7159 bytes --]

From dd7eca9a3cfec364a05aa0f52b941f09b4dea039 Mon Sep 17 00:00:00 2001
From: Thomas Danckaert <thomas.danckaert@gmail.com>
Date: Wed, 28 Sep 2016 10:34:58 +0200
Subject: [PATCH] gnu: Add hdf4

* gnu/packages/maths.scm (hdf4, hdf4-alt): New variables.
* gnu/packages/patches/hdf4-reproducibility.patch: New file.
* gnu/packages/patches/hdf4-shared-fortran.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add patches.
---
 gnu/local.mk                                    |  2 +
 gnu/packages/maths.scm                          | 65 +++++++++++++++++++++++++
 gnu/packages/patches/hdf4-reproducibility.patch | 47 ++++++++++++++++++
 gnu/packages/patches/hdf4-shared-fortran.patch  | 25 ++++++++++
 4 files changed, 139 insertions(+)
 create mode 100644 gnu/packages/patches/hdf4-reproducibility.patch
 create mode 100644 gnu/packages/patches/hdf4-shared-fortran.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 1ce95f2..1b65e74 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -570,6 +570,8 @@ dist_patch_DATA =						\
   %D%/packages/patches/gtk3-respect-GUIX_GTK3_PATH.patch	\
   %D%/packages/patches/gtk3-respect-GUIX_GTK3_IM_MODULE_FILE.patch \
   %D%/packages/patches/gtkglext-disable-disable-deprecated.patch \
+  %D%/packages/patches/hdf4-shared-fortran.patch 		\
+  %D%/packages/patches/hdf4-reproducibility.patch 		\
   %D%/packages/patches/hdf5-config-date.patch			\
   %D%/packages/patches/hdf-eos5-build-shared.patch 		\
   %D%/packages/patches/hdf-eos5-remove-gctp.patch		\
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index d5e82bb..72abe58 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -445,6 +445,71 @@ computations.")
     (home-page "https://github.com/OkoSanto/GCTP")
     (license license:public-domain))) ;https://www2.usgs.gov/laws/info_policies.html
 
+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://support.hdfgroup.org/ftp/HDF/releases/HDF"
+                           version "/src/hdf-" version ".tar.bz2"))
+       (sha256
+        (base32 "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))
+       (patches (search-patches "hdf4-reproducibility.patch"
+                                "hdf4-shared-fortran.patch"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:parallel-tests? #f
+       #:configure-flags '("--enable-shared")
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patchbuild
+           (lambda _
+             (substitute*
+                 '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+               (("/bin/rm") "rm")
+               (("/bin/mkdir") "mkdir"))
+             (substitute*
+                 (map (lambda (dir) (string-append dir "/Makefile.in"))
+                      '("hdf" "hdf/examples" "hdf/fortran" "hdf/src"
+                        "hdf/test" "hdf/util" "mfhdf" "mfhdf/dumper"
+                        "mfhdf/examples" "mfhdf/fortran" "mfhdf/hdfimport"
+                        "mfhdf/hdiff" "mfhdf/hrepack" "mfhdf/libsrc"
+                        "mfhdf/ncgen" "mfhdf/ncdump" "mfhdf/nctest"
+                        "mfhdf/test" "mfhdf/xdr"))
+               (("@HDF_BUILD_XDR_TRUE@XDR_ADD = \
+-R\\$\\(abs_top_builddir\\)/mfhdf/xdr/\\.libs") "")
+               (("@HDF_BUILD_SHARED_TRUE@AM_LDFLAGS = \
+-R\\$\\(abs_top_builddir\\)/mfhdf/libsrc/\\.libs \
+-R\\$\\(abs_top_builddir\\)/hdf/src/\\.libs \\$\\(XDR_ADD\\)") "")))))))
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "HDF4 is a library and multi-object file format for storing
+and managing data between machines.  HDF4 is an older hierarchical data format,
+incompatible with HDF5.")
+    (license
+     (license:non-copyleft
+      "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
+(define-public hdf4-alt
+  (package
+    (inherit hdf4)
+    (name "hdf4-alt")
+    (arguments
+     (substitute-keyword-arguments (package-arguments hdf4)
+       ((#:configure-flags flags) `(cons* "--disable-netcdf" ,flags))))
+    (synopsis
+     "HDF4 without netCDF API, can be combined with the regular netCDF library")))
+
 (define-public hdf5
   (package
     (name "hdf5")
diff --git a/gnu/packages/patches/hdf4-reproducibility.patch b/gnu/packages/patches/hdf4-reproducibility.patch
new file mode 100644
index 0000000..2c66bec
--- /dev/null
+++ b/gnu/packages/patches/hdf4-reproducibility.patch
@@ -0,0 +1,47 @@
+Remove/patch unreproducible config data.
+---
+ configure           | 9 ++++++++-
+ libhdf4.settings.in | 6 +++---
+ 2 files changed, 11 insertions(+), 4 deletions(-)
+
+diff --git a/configure b/configure
+index eb9f346..ebab94d 100755
+--- a/configure
++++ b/configure
+@@ -23163,7 +23163,14 @@ H4_VERSION="`cut -d' ' -f3 $srcdir/README.txt | head -1`"
+ 
+ 
+ ## Configuration date
+- CONFIG_DATE="`date`"
++CONFIG_DATE="`date -u`"
++if test -n "$SOURCE_DATE_EPOCH"; then
++  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
++               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
++  if test -z "$CONFIG_DATE"; then
++    as_fn_error $? "malformed SOURCE_DATE_EPOCH" "$LINENO" 5
++  fi
++fi
+ 
+ ## User doing the configuration
+  CONFIG_USER="`whoami`@`hostname`"
+diff --git a/libhdf4.settings.in b/libhdf4.settings.in
+index 1d9bbb9..4180d72 100644
+--- a/libhdf4.settings.in
++++ b/libhdf4.settings.in
+@@ -5,10 +5,10 @@ General Information:
+ -------------------
+ 		   HDF4 Version: @H4_VERSION@
+ 		  Configured on: @CONFIG_DATE@
+-		  Configured by: @CONFIG_USER@
++		  Configured by: guix
+ 		 Configure mode: @CONFIG_MODE@
+-		    Host system: @host_cpu@-@host_vendor@-@host_os@
+-              Uname information: @UNAME_INFO@
++		    Host system: guix
++              Uname information: guix
+ 		      Libraries: @STATIC_SHARED@
+ 	     Installation point: @prefix@
+ 
+-- 
+2.10.0
+
diff --git a/gnu/packages/patches/hdf4-shared-fortran.patch b/gnu/packages/patches/hdf4-shared-fortran.patch
new file mode 100644
index 0000000..60db355
--- /dev/null
+++ b/gnu/packages/patches/hdf4-shared-fortran.patch
@@ -0,0 +1,25 @@
+Allow building a shared library with Fortran.
+---
+ configure | 6 ------
+ 1 file changed, 6 deletions(-)
+
+diff --git a/configure b/configure
+index f37d2fa..eb9f346 100755
+--- a/configure
++++ b/configure
+@@ -6141,12 +6141,6 @@ if test "X${enable_shared}" != "Xyes"; then
+     enable_shared="no"
+ fi
+ 
+-if test "X${enable_shared}" = "Xyes"; then
+-    if test "X${BUILD_FORTRAN}" = "Xyes"; then
+-        as_fn_error $? "Cannot build shared fortran libraries. Please configure with --disable-fortran flag." "$LINENO" 5
+-    fi
+-fi
+-
+ 
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ln -s works" >&5
+ $as_echo_n "checking whether ln -s works... " >&6; }
+-- 
+2.10.0
+
-- 
2.10.0

Re: [PATCH] gnu: Add hdf4

[Ludovic Courtès]

Hello!

Thomas Danckaert <post@thomasdanckaert.be> skribis:

> this patch continues the work done by Jeremy Robst to package HDF4
>
> (https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00069.html)

Nice!

>  - Back then, people reported occasional build failures. I've disabled
>    parallel tests and don't have any problems on my system anymore
>    (maybe someone else can test?).

Will do.

>  - I've added a patch to allow a shared build with Fortran interface,
>    and some substitutions to remove bogus RPATH settings from the
>    Makefiles (otherwise, build directories are included in the RPATH).
>
>  - I've also added a variant “hdf4-alt”: by default, HDF4 includes a
>    netCDF API to access HDF4 files, which clashes with the symbols in
>    the real netCDF library when you try to use both libraries in your
>    program.  This variant uses the “--disable-netcdf” configure flag
>    to avoid this.  We could call it “hdf4-nonetcdf”, but the name
>    “hdf4-alt” is used by Debian, so maybe it's good to use the same?

Would “hdf4-minimal” sound appropriate here?  That’s a convention we use
in similar cases.  Otherwise “hdf4-alt” is fine with me.

> From dd7eca9a3cfec364a05aa0f52b941f09b4dea039 Mon Sep 17 00:00:00 2001
> From: Thomas Danckaert <thomas.danckaert@gmail.com>
> Date: Wed, 28 Sep 2016 10:34:58 +0200
> Subject: [PATCH] gnu: Add hdf4
>
> * gnu/packages/maths.scm (hdf4, hdf4-alt): New variables.
> * gnu/packages/patches/hdf4-reproducibility.patch: New file.
> * gnu/packages/patches/hdf4-shared-fortran.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add patches.

Please credit Jeremy in a “Co-authored-by” tag.

I only have cosmetic comments:

> +             (substitute*
> +                 (map (lambda (dir) (string-append dir "/Makefile.in"))
> +                      '("hdf" "hdf/examples" "hdf/fortran" "hdf/src"
> +                        "hdf/test" "hdf/util" "mfhdf" "mfhdf/dumper"
> +                        "mfhdf/examples" "mfhdf/fortran" "mfhdf/hdfimport"
> +                        "mfhdf/hdiff" "mfhdf/hrepack" "mfhdf/libsrc"
> +                        "mfhdf/ncgen" "mfhdf/ncdump" "mfhdf/nctest"
> +                        "mfhdf/test" "mfhdf/xdr"))

Maybe simply (find-files "." "^Makefile\\.in$")?

> --- /dev/null
> +++ b/gnu/packages/patches/hdf4-reproducibility.patch
> @@ -0,0 +1,47 @@
> +Remove/patch unreproducible config data.
> +---
> + configure           | 9 ++++++++-
> + libhdf4.settings.in | 6 +++---
> + 2 files changed, 11 insertions(+), 4 deletions(-)
> +
> +diff --git a/configure b/configure
> +index eb9f346..ebab94d 100755
> +--- a/configure
> ++++ b/configure
> +@@ -23163,7 +23163,14 @@ H4_VERSION="`cut -d' ' -f3 $srcdir/README.txt | head -1`"
> + 
> + 
> + ## Configuration date
> +- CONFIG_DATE="`date`"
> ++CONFIG_DATE="`date -u`"
> ++if test -n "$SOURCE_DATE_EPOCH"; then
> ++  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
> ++               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
> ++  if test -z "$CONFIG_DATE"; then
> ++    as_fn_error $? "malformed SOURCE_DATE_EPOCH" "$LINENO" 5
> ++  fi
> ++fi

It’d be enough to just do:

- CONFIG_DATE="`date`"
+ CONFIG_DATE="$SOURCE_DATE_EPOCH"

The smaller the patch, the better.  :-)

Otherwise LGTM.  Could you send an updated patch?

Thank you!

Ludo’.

Re: [PATCH] gnu: Add hdf4

[Thomas Danckaert]

[-- Attachment #1: Type: Text/Plain, Size: 2052 bytes --]

From: ludo@gnu.org (Ludovic Courtès)
Subject: Re: [PATCH] gnu: Add hdf4
Date: Wed, 12 Oct 2016 14:54:40 +0200

>>  - I've also added a variant “hdf4-alt”: by default, HDF4 includes 
>> a
>>    netCDF API to access HDF4 files, which clashes with the symbols 
>> in
>>    the real netCDF library when you try to use both libraries in 
>> your
>>    program.  This variant uses the “--disable-netcdf” configure 
>> flag
>>    to avoid this.  We could call it “hdf4-nonetcdf”, but the name
>>    “hdf4-alt” is used by Debian, so maybe it's good to use the 
>> same?
>
> Would “hdf4-minimal” sound appropriate here?  That’s a convention 
> we use
> in similar cases.  Otherwise “hdf4-alt” is fine with me.

With --disable-netcdf, the library still includes all the same 
features, the netCDF API function names are just mangled to avoid 
conflicts.  So it's not really more “minimal” than the regular HDF4 
:-)  I'd either go for something that explicitly mentions netCDF, or 
hdf4-alt because that's what Debian uses.

>> +             (substitute*
>> +                 (map (lambda (dir) (string-append dir 
>> "/Makefile.in"))
>> +                      '("hdf" "hdf/examples" "hdf/fortran" 
>> "hdf/src"
>> +                        "hdf/test" "hdf/util" "mfhdf" 
>> "mfhdf/dumper"
>> +                        "mfhdf/examples" "mfhdf/fortran" 
>> "mfhdf/hdfimport"
>> +                        "mfhdf/hdiff" "mfhdf/hrepack" 
>> "mfhdf/libsrc"
>> +                        "mfhdf/ncgen" "mfhdf/ncdump" 
>> "mfhdf/nctest"
>> +                        "mfhdf/test" "mfhdf/xdr"))
>
> Maybe simply (find-files "." "^Makefile\\.in$")?

Yes, I was convinced something like that had to exist, but am not 
good at reading the docs, it seems.

> - CONFIG_DATE="`date`"
> + CONFIG_DATE="$SOURCE_DATE_EPOCH"
>
> The smaller the patch, the better.  :-)

I went with

+CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH"`

(which just results in 1 Jan 1970, but anyway)

cheers,

Thomas

[-- Attachment #2: 0001-gnu-Add-hdf4.patch --]
[-- Type: Text/X-Patch, Size: 6375 bytes --]

From 3778ec2dfea4e5d7c0712ae1d354b463c8ee6694 Mon Sep 17 00:00:00 2001
From: Thomas Danckaert <thomas.danckaert@gmail.com>
Date: Wed, 28 Sep 2016 10:34:58 +0200
Subject: [PATCH] gnu: Add hdf4

* gnu/packages/maths.scm (hdf4, hdf4-alt): New variables.
* gnu/packages/patches/hdf4-reproducibility.patch: New file.
* gnu/packages/patches/hdf4-shared-fortran.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add patches.

Co-Authored-By: Jeremy Robst <jpro@bas.ac.uk>
---
 gnu/local.mk                                    |  2 +
 gnu/packages/maths.scm                          | 59 +++++++++++++++++++++++++
 gnu/packages/patches/hdf4-reproducibility.patch | 35 +++++++++++++++
 gnu/packages/patches/hdf4-shared-fortran.patch  | 25 +++++++++++
 4 files changed, 121 insertions(+)
 create mode 100644 gnu/packages/patches/hdf4-reproducibility.patch
 create mode 100644 gnu/packages/patches/hdf4-shared-fortran.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 1ce95f2..1b65e74 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -570,6 +570,8 @@ dist_patch_DATA =						\
   %D%/packages/patches/gtk3-respect-GUIX_GTK3_PATH.patch	\
   %D%/packages/patches/gtk3-respect-GUIX_GTK3_IM_MODULE_FILE.patch \
   %D%/packages/patches/gtkglext-disable-disable-deprecated.patch \
+  %D%/packages/patches/hdf4-shared-fortran.patch 		\
+  %D%/packages/patches/hdf4-reproducibility.patch 		\
   %D%/packages/patches/hdf5-config-date.patch			\
   %D%/packages/patches/hdf-eos5-build-shared.patch 		\
   %D%/packages/patches/hdf-eos5-remove-gctp.patch		\
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index d5e82bb..07934e3 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -445,6 +445,65 @@ computations.")
     (home-page "https://github.com/OkoSanto/GCTP")
     (license license:public-domain))) ;https://www2.usgs.gov/laws/info_policies.html
 
+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://support.hdfgroup.org/ftp/HDF/releases/HDF"
+                           version "/src/hdf-" version ".tar.bz2"))
+       (sha256
+        (base32 "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))
+       (patches (search-patches "hdf4-reproducibility.patch"
+                                "hdf4-shared-fortran.patch"))))
+
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:parallel-tests? #f
+       #:configure-flags '("--enable-shared")
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patchbuild
+           (lambda _
+             (substitute*
+                 '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+               (("/bin/rm") "rm")
+               (("/bin/mkdir") "mkdir"))
+             (substitute* (find-files "." "^Makefile\\.in$")
+               (("@HDF_BUILD_XDR_TRUE@XDR_ADD = \
+-R\\$\\(abs_top_builddir\\)/mfhdf/xdr/\\.libs") "")
+               (("@HDF_BUILD_SHARED_TRUE@AM_LDFLAGS = \
+-R\\$\\(abs_top_builddir\\)/mfhdf/libsrc/\\.libs \
+-R\\$\\(abs_top_builddir\\)/hdf/src/\\.libs \\$\\(XDR_ADD\\)") "")))))))
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "HDF4 is a library and multi-object file format for storing
+and managing data between machines.  HDF4 is an older hierarchical data format,
+incompatible with HDF5.")
+    (license
+     (license:non-copyleft
+      "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
+(define-public hdf4-alt
+  (package
+    (inherit hdf4)
+    (name "hdf4-alt")
+    (arguments
+     (substitute-keyword-arguments (package-arguments hdf4)
+       ((#:configure-flags flags) `(cons* "--disable-netcdf" ,flags))))
+    (synopsis
+     "HDF4 without netCDF API, can be combined with the regular netCDF library")))
+
 (define-public hdf5
   (package
     (name "hdf5")
diff --git a/gnu/packages/patches/hdf4-reproducibility.patch b/gnu/packages/patches/hdf4-reproducibility.patch
new file mode 100644
index 0000000..9b5fea6
--- /dev/null
+++ b/gnu/packages/patches/hdf4-reproducibility.patch
@@ -0,0 +1,35 @@
+Remove/patch unreproducible config data.
+
+diff --git a/configure b/configure
+index eb9f346..ebab94d 100755
+--- a/configure
++++ b/configure
+@@ -23163,7 +23163,7 @@ H4_VERSION="`cut -d' ' -f3 $srcdir/README.txt | head -1`"
+ 
+ 
+ ## Configuration date
+- CONFIG_DATE="`date`"
++CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH"`
+ 
+ ## User doing the configuration
+  CONFIG_USER="`whoami`@`hostname`"
+diff --git a/libhdf4.settings.in b/libhdf4.settings.in
+index 1d9bbb9..4180d72 100644
+--- a/libhdf4.settings.in
++++ b/libhdf4.settings.in
+@@ -5,10 +5,10 @@ General Information:
+ -------------------
+ 		   HDF4 Version: @H4_VERSION@
+ 		  Configured on: @CONFIG_DATE@
+-		  Configured by: @CONFIG_USER@
++		  Configured by: guix
+ 		 Configure mode: @CONFIG_MODE@
+-		    Host system: @host_cpu@-@host_vendor@-@host_os@
+-              Uname information: @UNAME_INFO@
++		    Host system: guix
++              Uname information: guix
+ 		      Libraries: @STATIC_SHARED@
+ 	     Installation point: @prefix@
+ 
+-- 
+2.10.0
diff --git a/gnu/packages/patches/hdf4-shared-fortran.patch b/gnu/packages/patches/hdf4-shared-fortran.patch
new file mode 100644
index 0000000..60db355
--- /dev/null
+++ b/gnu/packages/patches/hdf4-shared-fortran.patch
@@ -0,0 +1,25 @@
+Allow building a shared library with Fortran.
+---
+ configure | 6 ------
+ 1 file changed, 6 deletions(-)
+
+diff --git a/configure b/configure
+index f37d2fa..eb9f346 100755
+--- a/configure
++++ b/configure
+@@ -6141,12 +6141,6 @@ if test "X${enable_shared}" != "Xyes"; then
+     enable_shared="no"
+ fi
+ 
+-if test "X${enable_shared}" = "Xyes"; then
+-    if test "X${BUILD_FORTRAN}" = "Xyes"; then
+-        as_fn_error $? "Cannot build shared fortran libraries. Please configure with --disable-fortran flag." "$LINENO" 5
+-    fi
+-fi
+-
+ 
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ln -s works" >&5
+ $as_echo_n "checking whether ln -s works... " >&6; }
+-- 
+2.10.0
+
-- 
2.10.0

Re: [PATCH] gnu: Add hdf4

[Ludovic Courtès]

Thomas Danckaert <post@thomasdanckaert.be> skribis:

>> Would “hdf4-minimal” sound appropriate here?  That’s a convention we
>> use
>> in similar cases.  Otherwise “hdf4-alt” is fine with me.
>
> With --disable-netcdf, the library still includes all the same
> features, the netCDF API function names are just mangled to avoid
> conflicts.  So it's not really more “minimal” than the regular HDF4
> :-)  I'd either go for something that explicitly mentions netCDF, or
> hdf4-alt because that's what Debian uses.

Works for me!

> From 3778ec2dfea4e5d7c0712ae1d354b463c8ee6694 Mon Sep 17 00:00:00 2001
> From: Thomas Danckaert <thomas.danckaert@gmail.com>
> Date: Wed, 28 Sep 2016 10:34:58 +0200
> Subject: [PATCH] gnu: Add hdf4
>
> * gnu/packages/maths.scm (hdf4, hdf4-alt): New variables.
> * gnu/packages/patches/hdf4-reproducibility.patch: New file.
> * gnu/packages/patches/hdf4-shared-fortran.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add patches.
>
> Co-Authored-By: Jeremy Robst <jpro@bas.ac.uk>

Applied, thanks!

Ludo’.

Re: [PATCH] gnu: Add hdf4

[Andreas Enge]

Hello,

the packages fail to compile on arm and mips:
   http://hydra.gnu.org/eval/109281#tabs-new
   http://hydra.gnu.org/build/1521682

Could you maybe have a look at them?

Thanks!

Andreas

Re: [PATCH] gnu: Add hdf4

[Thomas Danckaert]

From: Andreas Enge <andreas@enge.fr>
Subject: Re: [PATCH] gnu: Add hdf4
Date: Fri, 14 Oct 2016 10:11:45 +0200

> Hello,
>
> the packages fail to compile on arm and mips:
>    http://hydra.gnu.org/eval/109281#tabs-new
>    http://hydra.gnu.org/build/1521682

Hi,

as far as I understand, the header hdfi.h defines fixed width 
datatypes such as float64, uint8, int32, ... depending on the 
detected architecture, and it's not detecting mips and arm properly.

I've checked, and see that Debian has a number patches for this and 
related issues, but I'm a bit out of my depth here (and don't have 
arm or mips systems to test before submitting any patches).  What are 
the options?  It would take me quite a bit of time to understand the 
Debian patches and see what we need.

Ideally, HDF4 maintainers would fix this in a future release.  I can 
try to contact them.

cheers,

Thomas

Re: [PATCH] gnu: Add hdf4

[Andreas Enge]

Hi Thomas,

On Fri, Oct 14, 2016 at 11:00:29AM +0200, Thomas Danckaert wrote:
> as far as I understand, the header hdfi.h defines fixed width datatypes such
> as float64, uint8, int32, ... depending on the detected architecture, and
> it's not detecting mips and arm properly.
> 
> I've checked, and see that Debian has a number patches for this and related
> issues

thanks for your comments! I picked a (rather large) patch from Debian
modifying hdfi.h, and the package now builds on arm. I pushed this, and
we will see for mips once hydra tries to build it (there is also a mips
specific patch on debian).

> Ideally, HDF4 maintainers would fix this in a future release.  I can try to
> contact them.

That would be nice, assuming that hdf4 is still maintained.

Andreas

[PATCH] gnu: Add hdf4

[Jeremy Robst]

     * gnu/packages/maths.scm (hdf4): New variable
---
  gnu/packages/maths.scm |   35 +++++++++++++++++++++++++++++++++++
  1 file changed, 35 insertions(+)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 19ed44f..570cc32 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -408,6 +408,41 @@ plotting engine by third-party applications like Octave.")
      (license (license:fsf-free
                "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))

+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source 
+     (origin 
+       (method url-fetch)
+       (uri (string-append
+             "http://www.hdfgroup.org/ftp/HDF/HDF_Current/src/hdf-"
+             version
+             ".tar.bz2"))
+       (sha256
+        (base32
+         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments '(#:tests? #f))
+ 
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "At its lowest level, HDF is a physical file format for
+storing scientific data.  At its highest level, HDF is a collection of
+utilities and applications for manipulating, viewing, and analyzing data
+in HDF files.  Between these levels, HDF is a software library that provides
+high-level APIs and a low-level data interface. ")
+    (license (license:x11-style
+              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
  (define-public hdf5
    (package
      (name "hdf5")
-- 
1.7.10.4

Re: [PATCH] gnu: Add hdf4

[Leo Famulari]

On Mon, May 23, 2016 at 06:22:09PM +0100, Jeremy Robst wrote:

Welcome, and thanks for the patch!

Unfortunately, I can't seem to apply it to my source tree. Is anybody
else able to do so? I can't figure *why* it doesn't apply. Jeremy, are
you able to apply it with `patch -p1` or `git am`?

> +(define-public hdf4
> +  (package
> +    (name "hdf4")
> +    (version "4.2.11")
> +    (source +     (origin +       (method url-fetch)

This is strange. It renders as expected once the patch is downloaded,
but in Mutt, it looks like this. I wonder if this has to do with the
patch not applying?

> +    (arguments '(#:tests? #f))

We prefer to add a comment explaining why test are disabled. If there
are no tests, then the comment can be "No test suite."

> +    (license (license:x11-style
> +              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))

To me, this looks more like a variation on the BSD license than the x11
license. Perhaps non-copyleft is a more appropriate choice. Can you look
at some of our packages using non-copyleft and report back?

Can you look into these questions and send an updated patch?

Re: [PATCH] gnu: Add hdf4

[Jeremy Robst]

Hi,

I could apply the previous patch, but I've created a new one (below) that 
handles the tests (I think).

>> +    (license (license:x11-style
>> +              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
>
> To me, this looks more like a variation on the BSD license than the x11
> license. Perhaps non-copyleft is a more appropriate choice. Can you look
> at some of our packages using non-copyleft and report back?

I've changed the license to non-copyleft, which I think is closer. I was 
copying the hdf5 license definition, which is very similar and probably 
should also be changed.

The definition for hdf4 still isn't quite right though, it doesn't 
generate a reproducible build - the hdf4 configure script generates a file 
lib/libhdf4.settings (from libhdf4.settings.in) which includes the config 
time, user, system information, which obviously changes from build to 
build or system to system.

What's the best way to resolve this ?

Thanks

Jeremy
-- 
jpro@bas.ac.uk | (work) 01223 221402 (fax) 01223 362616
Unix System Administrator - British Antarctic Survey
#include <disclaimer.std>

From 857fb05c7dcc8f19dc7d4cf04039743137f4f0ae Mon Sep 17 00:00:00 2001
From: Jeremy Robst <jpro@bas.ac.uk>
Date: Mon, 23 May 2016 17:59:00 +0100
Subject: [PATCH]     gnu: Add hdf4

     * gnu/packages/maths.scm (hdf4): New variable
---
  gnu/packages/maths.scm |   43 +++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 43 insertions(+)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 19ed44f..00c0310 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -408,6 +408,49 @@ plotting engine by third-party applications like Octave.")
      (license (license:fsf-free
                "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))

+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "http://www.hdfgroup.org/ftp/HDF/HDF_Current/src/hdf-"
+             version
+             ".tar.bz2"))
+       (sha256
+        (base32
+         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patch-configure
+          (lambda _
+            (substitute* '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+              (("/bin/rm") "rm")
+              (("/bin/mkdir") "mkdir"))
+            #t)))))
+
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data.")
+    (description "At its lowest level, HDF is a physical file format for
+storing scientific data.  At its highest level, HDF is a collection of
+utilities and applications for manipulating, viewing, and analyzing data
+in HDF files.  Between these levels, HDF is a software library that provides
+high-level APIs and a low-level data interface. ")
+    (license (license:non-copyleft
+              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
  (define-public hdf5
    (package
      (name "hdf5")
-- 
1.7.10.4

Re: [PATCH] gnu: Add hdf4

[Eric Bavier]

On 2016-05-24 08:20, Jeremy Robst wrote:
> Hi,
> 
> I could apply the previous patch, but I've created a new one (below)
> that handles the tests (I think).
> 
>>> +    (license (license:x11-style
>>> +              
>>> "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
>> 
>> To me, this looks more like a variation on the BSD license than the 
>> x11
>> license. Perhaps non-copyleft is a more appropriate choice. Can you 
>> look
>> at some of our packages using non-copyleft and report back?
> 
> I've changed the license to non-copyleft, which I think is closer. I
> was copying the hdf5 license definition, which is very similar and
> probably should also be changed.

Indeed.

> The definition for hdf4 still isn't quite right though, it doesn't
> generate a reproducible build - the hdf4 configure script generates a
> file lib/libhdf4.settings (from libhdf4.settings.in) which includes
> the config time, user, system information, which obviously changes
> from build to build or system to system.
> 
> What's the best way to resolve this ?

Could we apply a variation of the hdf5-config-date.patch?


> --
> jpro@bas.ac.uk | (work) 01223 221402 (fax) 01223 362616
> Unix System Administrator - British Antarctic Survey
> #include <disclaimer.std>
> 
> From 857fb05c7dcc8f19dc7d4cf04039743137f4f0ae Mon Sep 17 00:00:00 2001
> From: Jeremy Robst <jpro@bas.ac.uk>
> Date: Mon, 23 May 2016 17:59:00 +0100
> Subject: [PATCH]     gnu: Add hdf4
> 
>     * gnu/packages/maths.scm (hdf4): New variable
> ---
>  gnu/packages/maths.scm |   43 
> +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
> 
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 19ed44f..00c0310 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -408,6 +408,49 @@ plotting engine by third-party applications like 
> Octave.")
>      (license (license:fsf-free
> 
> "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))
> 
> +(define-public hdf4
> +  (package
> +    (name "hdf4")
> +    (version "4.2.11")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (string-append
> +             "http://www.hdfgroup.org/ftp/HDF/HDF_Current/src/hdf-"
> +             version
> +             ".tar.bz2"))
> +       (sha256
> +        (base32
> +         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))))
> +    (build-system gnu-build-system)
> +    (native-inputs
> +     `(("gfortran" ,gfortran)
> +       ("bison" ,bison)
> +       ("flex" ,flex)))
> +    (inputs
> +     `(("zlib" ,zlib)
> +       ("libjpeg" ,libjpeg)))
> +    (arguments
> +     `(#:phases
> +       (modify-phases %standard-phases
> +         (add-before 'configure 'patch-configure
> +          (lambda _
> +            (substitute* '("mfhdf/hdfimport/testutil.sh.in"
> "hdf/util/testutil.sh.in")
> +              (("/bin/rm") "rm")
> +              (("/bin/mkdir") "mkdir"))
> +            #t)))))
> +
> +    (home-page "https://www.hdfgroup.org/products/hdf4/")

The blank line between these fields is not necessary.

> +    (synopsis
> +     "Library and multi-object file format for storing and managing 
> data.")
> +    (description "At its lowest level, HDF is a physical file format 
> for
> +storing scientific data.  At its highest level, HDF is a collection of
> +utilities and applications for manipulating, viewing, and analyzing 
> data
> +in HDF files.  Between these levels, HDF is a software library that 
> provides
> +high-level APIs and a low-level data interface. ")

This description is noticeably more verbose than that of the hdf5 
package.  It seems like the first two sentences are not necessary here.  
It would also be nice to summarize how hdf4 differs from hdf5.

-- 
`~Eric

Re: [PATCH] gnu: Add hdf4

[Jeremy Robst]

Hi,

On Tue, 24 May 2016, Eric Bavier wrote:

> Could we apply a variation of the hdf5-config-date.patch?

Ok, I've based a patch on this one, and removed the other unreproduciable 
output, so I've not got a reproduciable build.

> This description is noticeably more verbose than that of the hdf5 
> package. It seems like the first two sentences are not necessary here. 
> It would also be nice to summarize how hdf4 differs from hdf5.

I've shortened the description, and added a statement that hdf4 is not 
compatable with hdf5. It's difficult to add more details of the 
differences without becoming verbose again, but let me know if you think 
it needs to be tweaked again.

Jeremy
-- 
jpro@bas.ac.uk | (work) 01223 221402 (fax) 01223 362616
Unix System Administrator - British Antarctic Survey
#include <disclaimer.std>

From 41816d45609674bbc62847960522e49a1646f6d3 Mon Sep 17 00:00:00 2001
From: Jeremy Robst <jpro@bas.ac.uk>
Date: Mon, 23 May 2016 17:59:00 +0100
Subject: [PATCH]     gnu: Add hdf4

     * gnu/packages/maths.scm (hdf4): New variable
---
  gnu/packages/maths.scm                          |   41 +++++++++++++++++
  gnu/packages/patches/hdf4-config-settings.patch |   55 +++++++++++++++++++++++
  2 files changed, 96 insertions(+)
  create mode 100644 gnu/packages/patches/hdf4-config-settings.patch

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 19ed44f..ab4785b 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -408,6 +408,47 @@ plotting engine by third-party applications like Octave.")
      (license (license:fsf-free
                "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))

+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "http://www.hdfgroup.org/ftp/HDF/releases/HDF" version
+             "/src/hdf-" version ".tar.bz2"))
+       (sha256
+        (base32
+         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))
+       (patches (list (search-patch "hdf4-config-settings.patch")))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patch-configure
+          (lambda _
+            (substitute*
+                '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+              (("/bin/rm") "rm")
+              (("/bin/mkdir") "mkdir"))
+            #t)))))
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "HDF4 is a library and multi-object file format for storing
+and managing data between machines.  HDF4 is an older hiearchical data format,
+incompatible with HDF5.")
+    (license (license:non-copyleft
+              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
  (define-public hdf5
    (package
      (name "hdf5")
diff --git a/gnu/packages/patches/hdf4-config-settings.patch b/gnu/packages/patches/hdf4-config-settings.patch
new file mode 100644
index 0000000..e79362e
--- /dev/null
+++ b/gnu/packages/patches/hdf4-config-settings.patch
@@ -0,0 +1,55 @@
+Honor SOURCE_DATE_EPOCH when exporting configuraton date.
+Remove unreproducible configuration data.
+
+--- a/configure
++++ b/configure
+@@ -23169,7 +23169,14 @@
+ 
+ 
+ ## Configuration date
+- CONFIG_DATE="`date`"
++ CONFIG_DATE="`date -u`"
++if test -n "$SOURCE_DATE_EPOCH"; then
++  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
++               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
++  if test -z "$CONFIG_DATE"; then
++    as_fn_error $? "malformed SOURCE_DATE_EPOCH" "$LINENO" 5
++  fi
++fi
+ 
+ ## User doing the configuration
+  CONFIG_USER="`whoami`@`hostname`"
+
+--- a/libhdf4.settings.in
++++ b/libhdf4.settings.in
+@@ -5,17 +5,17 @@
+ -------------------
+ 		   HDF4 Version: @H4_VERSION@
+ 		  Configured on: @CONFIG_DATE@
+-		  Configured by: @CONFIG_USER@
++		  Configured by: guix
+ 		 Configure mode: @CONFIG_MODE@
+-		    Host system: @host_cpu@-@host_vendor@-@host_os@
+-              Uname information: @UNAME_INFO@
++		    Host system: guix
++              Uname information: guix
+ 		      Libraries: @STATIC_SHARED@
+ 	     Installation point: @prefix@
+ 
+ Compiling Options:
+ ------------------
+                Compilation Mode: @CONFIG_MODE@
+-                     C compiler: @CC_VERSION@
++                     C compiler: 
+                          CFLAGS: @CFLAGS@
+                        CPPFLAGS: @CPPFLAGS@
+                Shared Libraries: @enable_shared@
+@@ -29,7 +29,7 @@
+ Languages:
+ ----------
+                         Fortran: @BUILD_FORTRAN@
+-@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: @F77_VERSION@
++@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: 
+ @HDF_BUILD_FORTRAN_TRUE@                         FFLAGS: @FFLAGS@
+ 
+ Features:
-- 
1.7.10.4

Re: [PATCH] gnu: Add hdf4

[Leo Famulari]

On Wed, May 25, 2016 at 06:42:04PM +0100, Jeremy Robst wrote:
> Hi,
> 
> On Tue, 24 May 2016, Eric Bavier wrote:
> 
> > Could we apply a variation of the hdf5-config-date.patch?
> 
> Ok, I've based a patch on this one, and removed the other unreproduciable
> output, so I've not got a reproduciable build.
> 
> > This description is noticeably more verbose than that of the hdf5
> > package. It seems like the first two sentences are not necessary here.
> > It would also be nice to summarize how hdf4 differs from hdf5.
> 
> I've shortened the description, and added a statement that hdf4 is not
> compatable with hdf5. It's difficult to add more details of the differences
> without becoming verbose again, but let me know if you think it needs to be
> tweaked again.

That sounds perfect, but I still can't apply the patch with `git am`,
although I can't figure out why this is. It *looks* like it should
apply.

Are you able to apply it to the current master branch (b5d08d7c28)?

Re: [PATCH] gnu: Add hdf4

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Wed, May 25, 2016 at 06:42:04PM +0100, Jeremy Robst wrote:
>> Hi,
>> 
>> On Tue, 24 May 2016, Eric Bavier wrote:
>> 
>> > Could we apply a variation of the hdf5-config-date.patch?
>> 
>> Ok, I've based a patch on this one, and removed the other unreproduciable
>> output, so I've not got a reproduciable build.
>> 
>> > This description is noticeably more verbose than that of the hdf5
>> > package. It seems like the first two sentences are not necessary here.
>> > It would also be nice to summarize how hdf4 differs from hdf5.
>> 
>> I've shortened the description, and added a statement that hdf4 is not
>> compatable with hdf5. It's difficult to add more details of the differences
>> without becoming verbose again, but let me know if you think it needs to be
>> tweaked again.
>
> That sounds perfect, but I still can't apply the patch with `git am`,
> although I can't figure out why this is. It *looks* like it should
> apply.
>
> Are you able to apply it to the current master branch (b5d08d7c28)?

Same here.  Specifically, ‘git am’ returns:

--8<---------------cut here---------------start------------->8---
Applying: gnu: Add hdf4
.git/rebase-apply/patch:88: space before tab in indent.
 		   HDF4 Version: @H4_VERSION@
.git/rebase-apply/patch:89: space before tab in indent.
 		  Configured on: @CONFIG_DATE@
.git/rebase-apply/patch:92: space before tab in indent.
 		 Configure mode: @CONFIG_MODE@
.git/rebase-apply/patch:97: space before tab in indent.
 		      Libraries: @STATIC_SHARED@
.git/rebase-apply/patch:98: space before tab in indent.
 	     Installation point: @prefix@
fatal: corrupt patch at line 104
Patch failed at 0001 gnu: Add hdf4
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
--8<---------------cut here---------------end--------------->8---

… and then:

--8<---------------cut here---------------start------------->8---
$ patch -p1 < .git/rebase-apply/patch 
patching file gnu/packages/maths.scm
patching file gnu/packages/patches/hdf4-config-settings.patch
patch: **** malformed patch at line 104: @CFLAGS@
--8<---------------cut here---------------end--------------->8---

Jeremy: could you send it as an attachment, or using ‘git send-email’?

Thank you!

Ludo’.

Re: [PATCH] gnu: Add hdf4

[Jeremy Robst]

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

Hi,

> Jeremy: could you send it as an attachment, or using ‘git send-email’?

Here's the patch as an attachment - sorry for the hassle. I will try and 
work out what's going wrong so I can send proper patches in future.

Jeremy
-- 
jpro@bas.ac.uk | (work) 01223 221402 (fax) 01223 362616
Unix System Administrator - British Antarctic Survey
#include <disclaimer.std>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff; name=0001-gnu-Add-hdf4.patch, Size: 4463 bytes --]

From 41816d45609674bbc62847960522e49a1646f6d3 Mon Sep 17 00:00:00 2001
From: Jeremy Robst <jpro@bas.ac.uk>
Date: Mon, 23 May 2016 17:59:00 +0100
Subject: [PATCH]     gnu: Add hdf4

    * gnu/packages/maths.scm (hdf4): New variable
---
 gnu/packages/maths.scm                          |   41 +++++++++++++++++
 gnu/packages/patches/hdf4-config-settings.patch |   55 +++++++++++++++++++++++
 2 files changed, 96 insertions(+)
 create mode 100644 gnu/packages/patches/hdf4-config-settings.patch

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 19ed44f..ab4785b 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -408,6 +408,47 @@ plotting engine by third-party applications like Octave.")
     (license (license:fsf-free
               "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))
 
+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "http://www.hdfgroup.org/ftp/HDF/releases/HDF" version
+             "/src/hdf-" version ".tar.bz2"))
+       (sha256
+        (base32
+         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))
+       (patches (list (search-patch "hdf4-config-settings.patch")))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patch-configure
+          (lambda _
+            (substitute*
+                '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+              (("/bin/rm") "rm")
+              (("/bin/mkdir") "mkdir"))
+            #t)))))
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "HDF4 is a library and multi-object file format for storing
+and managing data between machines.  HDF4 is an older hiearchical data format,
+incompatible with HDF5.")
+    (license (license:non-copyleft
+              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
 (define-public hdf5
   (package
     (name "hdf5")
diff --git a/gnu/packages/patches/hdf4-config-settings.patch b/gnu/packages/patches/hdf4-config-settings.patch
new file mode 100644
index 0000000..e79362e
--- /dev/null
+++ b/gnu/packages/patches/hdf4-config-settings.patch
@@ -0,0 +1,55 @@
+Honor SOURCE_DATE_EPOCH when exporting configuraton date.
+Remove unreproducible configuration data.
+
+--- a/configure
++++ b/configure
+@@ -23169,7 +23169,14 @@
+ 
+ 
+ ## Configuration date
+- CONFIG_DATE="`date`"
++ CONFIG_DATE="`date -u`"
++if test -n "$SOURCE_DATE_EPOCH"; then
++  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
++               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
++  if test -z "$CONFIG_DATE"; then
++    as_fn_error $? "malformed SOURCE_DATE_EPOCH" "$LINENO" 5
++  fi
++fi
+ 
+ ## User doing the configuration
+  CONFIG_USER="`whoami`@`hostname`"
+
+--- a/libhdf4.settings.in
++++ b/libhdf4.settings.in
+@@ -5,17 +5,17 @@
+ -------------------
+ 		   HDF4 Version: @H4_VERSION@
+ 		  Configured on: @CONFIG_DATE@
+-		  Configured by: @CONFIG_USER@
++		  Configured by: guix
+ 		 Configure mode: @CONFIG_MODE@
+-		    Host system: @host_cpu@-@host_vendor@-@host_os@
+-              Uname information: @UNAME_INFO@
++		    Host system: guix
++              Uname information: guix
+ 		      Libraries: @STATIC_SHARED@
+ 	     Installation point: @prefix@
+ 
+ Compiling Options:
+ ------------------
+                Compilation Mode: @CONFIG_MODE@
+-                     C compiler: @CC_VERSION@
++                     C compiler: 
+                          CFLAGS: @CFLAGS@
+                        CPPFLAGS: @CPPFLAGS@
+                Shared Libraries: @enable_shared@
+@@ -29,7 +29,7 @@
+ Languages:
+ ----------
+                         Fortran: @BUILD_FORTRAN@
+-@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: @F77_VERSION@
++@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: 
+ @HDF_BUILD_FORTRAN_TRUE@                         FFLAGS: @FFLAGS@
+ 
+ Features:
-- 
1.7.10.4

Re: [PATCH] gnu: Add hdf4

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 3999 bytes --]

Hello,

Jeremy Robst <jpro@bas.ac.uk> skribis:

> Here's the patch as an attachment - sorry for the hassle. I will try
> and work out what's going wrong so I can send proper patches in
> future.

No problem!

I tried the attached patch, which includes mostly cosmetic changes
compared to the one you posted.  However, I stumble upon test failures
(on x86_64, with Linux-libre 4.6.0):

--8<---------------cut here---------------start------------->8---
----------------------------
hdftest  Test Log
----------------------------
Testing create/read/write compressed datasets (tcomp.c)                PASSED
Testing create/read/write chunked datasets (tchunk.c)                  PASSED
Testing reading of netCDF file using the SDxxx inteface (tnetcdf.c)    PASSED
Testing functionality of dimensions (tdim.c)                           PASSED
Testing functions involving empty data sets (temptySDSs.c)             PASSED
Testing various setting attribute features (tattributes.c)             PASSED
Testing getting data size of special data (tdatasizes.c)               PASSED
Testing getting location info of data (tdatainfo.c)                    PASSED
Testing getting location info of attr and annot data (tattdatainfo.c)  PASSED
Testing a mix of SD, V, and VS functions (tmixed_apis.c)               PASSED
Testing miscellaneous file related functions (tfile.c)                *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
*** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
SDgetfilename: incorrect file being opened - expected <file5>, retrieved <file4>
*** Routine test_maxopenfiles: SDend FAILED at line 305 ***
Testing various SDS' properties (tsdsprops.c)                          PASSED
Testing various coordinate variable features (tcoordvar.c)             PASSED
Testing szip compression for datasets (tszip.c)                        PASSED
*** HDF-SD test fails ***
Makefile:1169: recipe for target 'hdftest.chkexe_' failed
make[5]: *** [hdftest.chkexe_] Error 1
make[5]: *** Waiting for unfinished jobs....
Finished testing testmfhdf.sh 
============================
testmfhdf.sh  Test Log
============================
=============================
Running mfhdf tests
=============================
=============================
HDF-SD C interface tests
=============================
Testing create/read/write compressed datasets (tcomp.c)                PASSED
Testing create/read/write chunked datasets (tchunk.c)                  PASSED
Testing reading of netCDF file using the SDxxx inteface (tnetcdf.c)    PASSED
Testing functionality of dimensions (tdim.c)                           PASSED
Testing functions involving empty data sets (temptySDSs.c)             PASSED
Testing various setting attribute features (tattributes.c)             PASSED
Testing getting data size of special data (tdatasizes.c)               PASSED
Testing getting location info of data (tdatainfo.c)                    PASSED
Testing getting location info of attr and annot data (tattdatainfo.c)  PASSED
Testing a mix of SD, V, and VS functions (tmixed_apis.c)               PASSED
Testing miscellaneous file related functions (tfile.c)                *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
*** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
*** Routine test_maxopenfiles: SDstart FAILED at line 285 ***

[...]

SDgetfilename: incorrect file being opened - expected <file67>, retrieved <file57>
*** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
SDgetfilename: incorrect file being opened - expected <file68>, retrieved <file57>
*** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
SDgetfilename: incorrect file being opened - expected <file69>, retrieved <file57>
*** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
--8<---------------cut here---------------start------------->8---

Any ideas?

Thanks,
Ludo’.


[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 5593 bytes --]

From 77afb922f470ddc97819dd8bb623450585fb2e12 Mon Sep 17 00:00:00 2001
From: Jeremy Robst <jpro@bas.ac.uk>
Date: Mon, 23 May 2016 17:59:00 +0100
Subject: [PATCH] gnu: Add hdf4.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/maths.scm (hdf4): New variable.
* gnu/packages/patches/hdf4-config-settings.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
---
 gnu/local.mk                                    |  1 +
 gnu/packages/maths.scm                          | 42 +++++++++++++++++++
 gnu/packages/patches/hdf4-config-settings.patch | 55 +++++++++++++++++++++++++
 3 files changed, 98 insertions(+)
 create mode 100644 gnu/packages/patches/hdf4-config-settings.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index dade736..9d608e8 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -533,6 +533,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/gtk2-theme-paths.patch			\
   %D%/packages/patches/gtk3-respect-GUIX_GTK3_PATH.patch	\
   %D%/packages/patches/gtkglext-disable-disable-deprecated.patch \
+  %D%/packages/patches/hdf4-config-settings.patch		\
   %D%/packages/patches/hdf5-config-date.patch			\
   %D%/packages/patches/hop-bigloo-4.0b.patch			\
   %D%/packages/patches/hop-linker-flags.patch			\
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index e11208c..b6e15f9 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -13,6 +13,7 @@
 ;;; Copyright © 2016 Roel Janssen <roel@gnu.org>
 ;;; Copyright © 2016 Kei Kebreau <kei@openmailbox.org>
 ;;; Copyright © 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2016 Jeremy Robst <jpro@bas.ac.uk>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -408,6 +409,47 @@ plotting engine by third-party applications like Octave.")
     (license (license:fsf-free
               "http://gnuplot.cvs.sourceforge.net/gnuplot/gnuplot/Copyright"))))
 
+(define-public hdf4
+  (package
+    (name "hdf4")
+    (version "4.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "http://www.hdfgroup.org/ftp/HDF/releases/HDF" version
+             "/src/hdf-" version ".tar.bz2"))
+       (sha256
+        (base32
+         "16yr50j845zlfx20skmw3y75ww77akk9gg0affjqkg66ih5r03mv"))
+       (patches (search-patches "hdf4-config-settings.patch"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("gfortran" ,gfortran)
+       ("bison" ,bison)
+       ("flex" ,flex)))
+    (inputs
+     `(("zlib" ,zlib)
+       ("libjpeg" ,libjpeg)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'patch-configure
+          (lambda _
+            (substitute*
+                '("mfhdf/hdfimport/testutil.sh.in" "hdf/util/testutil.sh.in")
+              (("/bin/rm") "rm")
+              (("/bin/mkdir") "mkdir"))
+            #t)))))
+    (home-page "https://www.hdfgroup.org/products/hdf4/")
+    (synopsis
+     "Library and multi-object file format for storing and managing data")
+    (description "HDF4 is a library and multi-object file format for storing
+and managing data between machines.  HDF4 is an older hiearchical data format,
+incompatible with HDF5.")
+    (license (license:non-copyleft
+              "https://www.hdfgroup.org/ftp/HDF/HDF_Current/src/unpacked/COPYING"))))
+
 (define-public hdf5
   (package
     (name "hdf5")
diff --git a/gnu/packages/patches/hdf4-config-settings.patch b/gnu/packages/patches/hdf4-config-settings.patch
new file mode 100644
index 0000000..e79362e
--- /dev/null
+++ b/gnu/packages/patches/hdf4-config-settings.patch
@@ -0,0 +1,55 @@
+Honor SOURCE_DATE_EPOCH when exporting configuraton date.
+Remove unreproducible configuration data.
+
+--- a/configure
++++ b/configure
+@@ -23169,7 +23169,14 @@
+ 
+ 
+ ## Configuration date
+- CONFIG_DATE="`date`"
++ CONFIG_DATE="`date -u`"
++if test -n "$SOURCE_DATE_EPOCH"; then
++  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
++               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
++  if test -z "$CONFIG_DATE"; then
++    as_fn_error $? "malformed SOURCE_DATE_EPOCH" "$LINENO" 5
++  fi
++fi
+ 
+ ## User doing the configuration
+  CONFIG_USER="`whoami`@`hostname`"
+
+--- a/libhdf4.settings.in
++++ b/libhdf4.settings.in
+@@ -5,17 +5,17 @@
+ -------------------
+ 		   HDF4 Version: @H4_VERSION@
+ 		  Configured on: @CONFIG_DATE@
+-		  Configured by: @CONFIG_USER@
++		  Configured by: guix
+ 		 Configure mode: @CONFIG_MODE@
+-		    Host system: @host_cpu@-@host_vendor@-@host_os@
+-              Uname information: @UNAME_INFO@
++		    Host system: guix
++              Uname information: guix
+ 		      Libraries: @STATIC_SHARED@
+ 	     Installation point: @prefix@
+ 
+ Compiling Options:
+ ------------------
+                Compilation Mode: @CONFIG_MODE@
+-                     C compiler: @CC_VERSION@
++                     C compiler: 
+                          CFLAGS: @CFLAGS@
+                        CPPFLAGS: @CPPFLAGS@
+                Shared Libraries: @enable_shared@
+@@ -29,7 +29,7 @@
+ Languages:
+ ----------
+                         Fortran: @BUILD_FORTRAN@
+-@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: @F77_VERSION@
++@HDF_BUILD_FORTRAN_TRUE@               Fortran Compiler: 
+ @HDF_BUILD_FORTRAN_TRUE@                         FFLAGS: @FFLAGS@
+ 
+ Features:
-- 
2.7.4

Re: [PATCH] gnu: Add hdf4

[Eric Bavier]

On 2016-05-29 17:07, ludo@gnu.org wrote:
> Hello,
> 
> Jeremy Robst <jpro@bas.ac.uk> skribis:
> 
>> Here's the patch as an attachment - sorry for the hassle. I will try
>> and work out what's going wrong so I can send proper patches in
>> future.
> 
> No problem!
> 
> I tried the attached patch, which includes mostly cosmetic changes
> compared to the one you posted.  However, I stumble upon test failures
> (on x86_64, with Linux-libre 4.6.0):
> 
> --8<---------------cut here---------------start------------->8---
> ----------------------------
> hdftest  Test Log
> ----------------------------
> Testing create/read/write compressed datasets (tcomp.c)                
> PASSED
> Testing create/read/write chunked datasets (tchunk.c)                  
> PASSED
> Testing reading of netCDF file using the SDxxx inteface (tnetcdf.c)    
> PASSED
> Testing functionality of dimensions (tdim.c)                           
> PASSED
> Testing functions involving empty data sets (temptySDSs.c)             
> PASSED
> Testing various setting attribute features (tattributes.c)             
> PASSED
> Testing getting data size of special data (tdatasizes.c)               
> PASSED
> Testing getting location info of data (tdatainfo.c)                    
> PASSED
> Testing getting location info of attr and annot data (tattdatainfo.c)  
> PASSED
> Testing a mix of SD, V, and VS functions (tmixed_apis.c)               
> PASSED
> Testing miscellaneous file related functions (tfile.c)
> *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
> *** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
> SDgetfilename: incorrect file being opened - expected <file5>, 
> retrieved <file4>
> *** Routine test_maxopenfiles: SDend FAILED at line 305 ***
> Testing various SDS' properties (tsdsprops.c)                          
> PASSED
> Testing various coordinate variable features (tcoordvar.c)             
> PASSED
> Testing szip compression for datasets (tszip.c)                        
> PASSED
> *** HDF-SD test fails ***
> Makefile:1169: recipe for target 'hdftest.chkexe_' failed
> make[5]: *** [hdftest.chkexe_] Error 1
> make[5]: *** Waiting for unfinished jobs....
> Finished testing testmfhdf.sh
> ============================
> testmfhdf.sh  Test Log
> ============================
> =============================
> Running mfhdf tests
> =============================
> =============================
> HDF-SD C interface tests
> =============================
> Testing create/read/write compressed datasets (tcomp.c)                
> PASSED
> Testing create/read/write chunked datasets (tchunk.c)                  
> PASSED
> Testing reading of netCDF file using the SDxxx inteface (tnetcdf.c)    
> PASSED
> Testing functionality of dimensions (tdim.c)                           
> PASSED
> Testing functions involving empty data sets (temptySDSs.c)             
> PASSED
> Testing various setting attribute features (tattributes.c)             
> PASSED
> Testing getting data size of special data (tdatasizes.c)               
> PASSED
> Testing getting location info of data (tdatainfo.c)                    
> PASSED
> Testing getting location info of attr and annot data (tattdatainfo.c)  
> PASSED
> Testing a mix of SD, V, and VS functions (tmixed_apis.c)               
> PASSED
> Testing miscellaneous file related functions (tfile.c)
> *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
> *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
> *** Routine test_maxopenfiles: SDstart FAILED at line 285 ***
> 
> [...]
> 
> SDgetfilename: incorrect file being opened - expected <file67>,
> retrieved <file57>
> *** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
> SDgetfilename: incorrect file being opened - expected <file68>,
> retrieved <file57>
> *** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
> SDgetfilename: incorrect file being opened - expected <file69>,
> retrieved <file57>
> *** Routine test_maxopenfiles: SDgetfilename FAILED at line 292 ***
> --8<---------------cut here---------------start------------->8---
> 
> Any ideas?

I get the same failures intermittently.  Other times the build passes.

Also, I've verified that the part of the patch that touches 
libhdf4.settings.in is not required for build reproducibility.

-- 
`~Eric

Re: [PATCH] gnu: Add hdf4

[Jeremy Robst]

Hi,

>> Any ideas?
>
> I get the same failures intermittently.  Other times the build passes.

Yes, I've built it many times over the past few days - sometimes I get a 
failure, other times it builds ok, like you. If I do

./pre-inst-env guix gc
./pre-inst-env guix build hdf4

it usually (but not always) fails, if I repeat the build it succeeds, then 
the only way I can get it to fail again is by doing a gc. I haven't 
managed to get the tests to fail building by hand.

Looking at the HDF code it's doing things with the maximum number of open 
files - is there anything guix does when it's creating the restricting 
environment for the build user that would limit the number of open files ?

> Also, I've verified that the part of the patch that touches 
> libhdf4.settings.in is not required for build reproducibility.

libhdf4.settings includes the build user and the details of the build 
host as well as the build time. This file is included in the final build. 
I patched configure to set the build time to the epoch, taken from the 
HDF5 patch.

However I also changed the libhdf4.settings to remove details of the build 
user and build host since I was aiming for reproducibilty across builds 
across machines. E.g I happen to always have built it with the user 
guixbuild00, but if I was building 2 or more things in parallel then it 
might get built with guixbuild01 and I thought this would then show us as 
a different build (since the username would be in libhdf4.settings in the 
file output). Is that right or am I misunderstanding something ?

Jeremy
-- 
jpro@bas.ac.uk | (work) 01223 221402 (fax) 01223 362616
Unix System Administrator - British Antarctic Survey
#include <disclaimer.std>

Re: [PATCH] gnu: Add hdf4

[Ludovic Courtès]

Hello,

Jeremy Robst <jpro@bas.ac.uk> skribis:

>>> Any ideas?
>>
>> I get the same failures intermittently.  Other times the build passes.
>
> Yes, I've built it many times over the past few days - sometimes I get
> a failure, other times it builds ok, like you. If I do
>
> ./pre-inst-env guix gc
> ./pre-inst-env guix build hdf4

Instead, you can run:

  ./pre-inst-env guix build --check hdf4

> Looking at the HDF code it's doing things with the maximum number of
> open files - is there anything guix does when it's creating the
> restricting environment for the build user that would limit the number
> of open files ?

It does not use ‘setrlimit’, which means it inherits the limits of its
parent process.

There was a problem on the maximum number of processes for people
running guix-daemon from systemd, which commit
f489ce3c9343e3ba9927645d11fcb91bb69f0fa7 addressed.

But anyway, I hit the problem on GuixSD, so it’s not systemd-related.

It might be worth trying:

  #:parallel-tests? #f

though I suspect it’s a per-process limit that we’re hitting, not a
global limit.

Next step would be to ask the HDF4 developers.  :-)

> libhdf4.settings includes the build user and the details of the build
> host as well as the build time. This file is included in the final
> build. I patched configure to set the build time to the epoch, taken
> from the HDF5 patch.

Clearing the build time is definitely necessary.

> However I also changed the libhdf4.settings to remove details of the
> build user and build host since I was aiming for reproducibilty across
> builds across machines. E.g I happen to always have built it with the
> user guixbuild00, but if I was building 2 or more things in parallel
> then it might get built with guixbuild01 and I thought this would then
> show us as a different build (since the username would be in
> libhdf4.settings in the file output). Is that right or am I
> misunderstanding something ?

You are right, the user name could differ, so it’s best to patch it if
it gets embedded in the end.

Thanks,
Ludo’.