[PATCH] Add ant-build-system.

[PATCH] Add ant-build-system.

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

Hi Guix,

this patch adds a new build system for Java projects: the
“ant-build-system”.

I’ve used it successfully to package the Java projects hamcrest-core,
junit, osgi-annotation, osgi-core, log4j-api, and javax-mail.

It’s especially useful in cases where a project does *not* use Ant and
would otherwise require building with Maven or recursively calling javac
on all source files (as currently done for swt), because it can generate
a simple Ant build file with all the right targets.  This cuts a lot of
boilerplate from Java packages.

The build.xml it generates contains a target “touch” which is run before
wrapping up the compiled .class files in a jar archive; this target
ensures that the timestamps of all archived files are reset, so the
produced jars can be (and in case of the above-mentioned packages)
deterministic.

Of course the build.xml is generated using sxml.

I’m in no hurry to get this merged (it just isn’t very important to me
at this pont where I’m still trying to figure out how to best deal with
Java libraries) and I welcome comments, bike shedding, and alternative
solutions.

~~ Ricardo


[-- Attachment #2: 0001-build-Add-Ant-build-system.patch --]
[-- Type: text/x-patch, Size: 14784 bytes --]

From 284f79ae6ba584603e58228c7c9ee73ac135912d Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
Date: Tue, 29 Dec 2015 16:56:49 +0100
Subject: [PATCH] build: Add Ant build system.

* guix/build-system/ant.scm: New file.
* guix/build/ant-build-system: New file.
* Makefile.am (MODULES): Add new files.
* doc/guix.texi (Build Systems): Document ant-build-system.
---
 Makefile.am                     |   2 +
 doc/guix.texi                   |  21 ++++++
 guix/build-system/ant.scm       | 148 ++++++++++++++++++++++++++++++++++++++++
 guix/build/ant-build-system.scm | 136 ++++++++++++++++++++++++++++++++++++
 4 files changed, 307 insertions(+)
 create mode 100644 guix/build-system/ant.scm
 create mode 100644 guix/build/ant-build-system.scm

diff --git a/Makefile.am b/Makefile.am
index 760caed..145ac5f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -53,6 +53,7 @@ MODULES =					\
   guix/graph.scm				\
   guix/cve.scm					\
   guix/build-system.scm				\
+  guix/build-system/ant.scm			\
   guix/build-system/cmake.scm			\
   guix/build-system/emacs.scm			\
   guix/build-system/glib-or-gtk.scm		\
@@ -72,6 +73,7 @@ MODULES =					\
   guix/cvs-download.scm				\
   guix/svn-download.scm				\
   guix/ui.scm					\
+  guix/build/ant-build-system.scm		\
   guix/build/download.scm			\
   guix/build/cmake-build-system.scm		\
   guix/build/emacs-build-system.scm		\
diff --git a/doc/guix.texi b/doc/guix.texi
index 33debe6..327f512 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -2593,6 +2593,27 @@ of @var{gnu-build-system}, and differ mainly in the set of inputs
 implicitly added to the build process, and in the list of phases
 executed.  Some of these build systems are listed below.
 
+@defvr {Scheme Variable} ant-build-system
+This variable is exported by @code{(guix build-system ant)}.  It
+implements the build procedure for Java packages that can be built with
+@url{http://ant.apache.org/, Ant build tool}.
+
+It adds both @code{ant} and the @dfn{Java Development Kit} (JDK) as
+provided by the @code{icedtea} package to the set of inputs.  Different
+packages can be specified with the @code{#:ant} and @code{#:jdk}
+parameters, respectively.
+
+When the original package does not provide a suitable Ant build file,
+the parameter @code{#:jar-name} can be used to generate a minimal Ant
+build file @file{build.xml} with tasks to build the specified jar
+archive.
+
+The parameter @code{#:build-target} can be used to specify the Ant task
+that should be run during the @code{build} phase.  By default the
+``jar'' task will be run.
+
+@end defvr
+
 @defvr {Scheme Variable} cmake-build-system
 This variable is exported by @code{(guix build-system cmake)}.  It
 implements the build procedure for packages using the
diff --git a/guix/build-system/ant.scm b/guix/build-system/ant.scm
new file mode 100644
index 0000000..5240ff6
--- /dev/null
+++ b/guix/build-system/ant.scm
@@ -0,0 +1,148 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build-system ant)
+  #:use-module (guix store)
+  #:use-module (guix utils)
+  #:use-module (guix packages)
+  #:use-module (guix derivations)
+  #:use-module (guix search-paths)
+  #:use-module (guix build-system)
+  #:use-module (guix build-system gnu)
+  #:use-module (ice-9 match)
+  #:use-module (srfi srfi-26)
+  #:export (%ant-build-system-modules
+            ant-build
+            ant-build-system))
+
+;; Commentary:
+;;
+;; Standard build procedure for Java packages using Ant.
+;;
+;; Code:
+
+(define %ant-build-system-modules
+  ;; Build-side modules imported by default.
+  `((guix build ant-build-system)
+    ,@%gnu-build-system-modules))
+
+(define (default-jdk)
+  "Return the default JDK package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((jdk-mod (resolve-interface '(gnu packages java))))
+    (module-ref jdk-mod 'icedtea)))
+
+(define (default-ant)
+  "Return the default Ant package."
+  ;; Lazily resolve the binding to avoid a circular dependency.
+  (let ((jdk-mod (resolve-interface '(gnu packages java))))
+    (module-ref jdk-mod 'ant)))
+
+(define* (lower name
+                #:key source inputs native-inputs outputs system target
+                (jdk (default-jdk))
+                (ant (default-ant))
+                #:allow-other-keys
+                #:rest arguments)
+  "Return a bag for NAME."
+  (define private-keywords
+    '(#:source #:target #:inputs #:native-inputs))
+
+  (and (not target)                               ;XXX: no cross-compilation
+       (bag
+         (name name)
+         (system system)
+         (host-inputs `(,@(if source
+                              `(("source" ,source))
+                              '())
+                        ,@inputs
+
+                        ;; Keep the standard inputs of 'gnu-build-system'.
+                        ,@(standard-packages)))
+         (build-inputs `(("jdk" ,jdk "jdk")
+                         ("ant" ,ant)
+                         ,@native-inputs))
+         (outputs outputs)
+         (build ant-build)
+         (arguments (strip-keyword-arguments private-keywords arguments)))))
+
+(define* (ant-build store name inputs
+                    #:key
+                    (tests? #t)
+                    (test-target "tests")
+                    (configure-flags ''())
+                    (make-flags ''())
+                    (build-target "jar")
+                    (jar-name #f)
+                    (phases '(@ (guix build ant-build-system)
+                                %standard-phases))
+                    (outputs '("out"))
+                    (search-paths '())
+                    (system (%current-system))
+                    (guile #f)p
+                    (imported-modules %ant-build-system-modules)
+                    (modules '((guix build ant-build-system)
+                               (guix build utils))))
+  "Build SOURCE with INPUTS."
+  (define builder
+    `(begin
+       (use-modules ,@modules)
+       (ant-build #:name ,name
+                  #:source ,(match (assoc-ref inputs "source")
+                              (((? derivation? source))
+                               (derivation->output-path source))
+                              ((source)
+                               source)
+                              (source
+                               source))
+                  #:make-flags ,make-flags
+                  #:configure-flags ,configure-flags
+                  #:system ,system
+                  #:tests? ,tests?
+                  #:test-target ,test-target
+                  #:build-target ,build-target
+                  #:jar-name ,jar-name
+                  #:phases ,phases
+                  #:outputs %outputs
+                  #:search-paths ',(map search-path-specification->sexp
+                                        search-paths)
+                  #:inputs %build-inputs)))
+
+  (define guile-for-build
+    (match guile
+      ((? package?)
+       (package-derivation store guile system #:graft? #f))
+      (#f                               ; the default
+       (let* ((distro (resolve-interface '(gnu packages commencement)))
+              (guile  (module-ref distro 'guile-final)))
+         (package-derivation store guile system #:graft? #f)))))
+
+  (build-expression->derivation store name builder
+                                #:inputs inputs
+                                #:system system
+                                #:modules imported-modules
+                                #:outputs outputs
+                                #:guile-for-build guile-for-build))
+
+(define ant-build-system
+  (build-system
+    (name 'ant)
+    (description "The standard Ant build system")
+    (lower lower)))
+
+;;; ant.scm ends here
diff --git a/guix/build/ant-build-system.scm b/guix/build/ant-build-system.scm
new file mode 100644
index 0000000..9a7f886
--- /dev/null
+++ b/guix/build/ant-build-system.scm
@@ -0,0 +1,136 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 Ricardo Wurmus <rekado@elephly.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix build ant-build-system)
+  #:use-module ((guix build gnu-build-system) #:prefix gnu:)
+  #:use-module (guix build utils)
+  #:use-module (sxml simple)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 ftw)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:export (%standard-phases
+            ant-build))
+
+;; Commentary:
+;;
+;; Builder-side code of the standard build procedure for Java packages using
+;; Ant.
+;;
+;; Code:
+
+(define (default-build.xml jar-name prefix)
+  "Create a simple build.xml with standard targets for Ant."
+  (let ((file (open-output-file "build.xml")))
+    (sxml->xml
+     `(project (@ (basedir "."))
+               (property (@ (name "classes.dir")
+                            (value "${basedir}/build/classes")))
+               (property (@ (name "jar.dir")
+                            (value "${basedir}/build/jar")))
+               (property (@ (name "dist.dir")
+                            (value ,prefix)))
+
+               ;; respect the CLASSPATH environment variable
+               (property (@ (name "build.sysclasspath")
+                            (value "first")))
+               (property (@ (environment "env")))
+               (path (@ (id "classpath"))
+                     (pathelement (@ (location "${env.CLASSPATH}"))))
+
+               (target (@ (name "compile"))
+                       (mkdir (@ (dir "${classes.dir}")))
+                       (javac (@ (includeantruntime "false")
+                                 (srcdir "src")
+                                 (destdir "${classes.dir}")
+                                 (classpath (@ (refid "classpath"))))))
+
+               ;; Reset the ctime/mtime on all files to ensure that the
+               ;; jar archive for the same class files is bit-identical.
+               (target (@ (name "touch"))
+                       (exec (@ (executable "find"))
+                             (arg (@ (line "${classes.dir} -exec touch -d @0 {} ;")))))
+
+               (target (@ (name "jar")
+                          (depends "compile,touch"))
+                       (mkdir (@ (dir "${jar.dir}")))
+                       ;; We cannot use the simpler "jar" task here, because
+                       ;; there is no way to disable generation of a
+                       ;; manifest.  We do not include a generated manifest
+                       ;; to ensure determinism, because we cannot easily
+                       ;; reset the ctime/mtime before creating the archive.
+                       (exec (@ (executable "jar"))
+                             (arg (@ (line ,(string-append "-Mcf ${jar.dir}/" jar-name
+                                                           " -C ${classes.dir} ."))))))
+
+               (target (@ (name "install"))
+                       (copy (@ (todir "${dist.dir}"))
+                             (fileset (@ (dir "${jar.dir}"))
+                                      (include (@ (name "**/*.jar")))))))
+     file)
+    (close-output-port file)
+    (utime "build.xml" 0 0)
+    #t))
+
+(define (generate-classpath inputs)
+  "Return a colon-separated string of full paths to jar files found among the
+INPUTS."
+  (string-join
+   (apply append (map (match-lambda
+                        ((_ . dir)
+                         (find-files dir "\\.*jar$")))
+                      inputs)) ":"))
+
+(define* (configure #:key inputs outputs (jar-name #f)
+                    #:allow-other-keys)
+  (when jar-name
+    (default-build.xml jar-name
+      (string-append (assoc-ref outputs "out")
+                     "/share/java")))
+  (setenv "JAVA_HOME" (assoc-ref inputs "jdk"))
+  (setenv "CLASSPATH" (generate-classpath inputs)))
+
+(define* (build #:key (make-flags '()) (build-target "jar")
+                #:allow-other-keys)
+  (zero? (apply system* `("ant" ,build-target ,@make-flags))))
+
+(define* (check #:key target (make-flags '()) (tests? (not target))
+                (test-target "check")
+                #:allow-other-keys)
+  (if tests?
+      (zero? (apply system* `("ant" ,test-target ,@make-flags)))
+      (begin
+        (format #t "test suite not run~%")
+        #t)))
+
+(define* (install #:key (make-flags '()) #:allow-other-keys)
+  (zero? (apply system* `("ant" "install" ,@make-flags))))
+
+(define %standard-phases
+  (modify-phases gnu:%standard-phases
+    (replace 'configure configure)
+    (replace 'build build)
+    (replace 'check check)
+    (replace 'install install)))
+
+(define* (ant-build #:key inputs (phases %standard-phases)
+                    #:allow-other-keys #:rest args)
+  "Build the given Java package, applying all of PHASES in order."
+  (apply gnu:gnu-build #:inputs inputs #:phases phases args))
+
+;;; ant-build-system.scm ends here
-- 
2.1.0

Re: [PATCH] Add ant-build-system.

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> I’ve used it successfully to package the Java projects hamcrest-core,
> junit, osgi-annotation, osgi-core, log4j-api, and javax-mail.

Nice.

> It’s especially useful in cases where a project does *not* use Ant and
> would otherwise require building with Maven or recursively calling javac
> on all source files (as currently done for swt), because it can generate
> a simple Ant build file with all the right targets.  This cuts a lot of
> boilerplate from Java packages.

The idea of generating a ‘build.xml’ file is quite unexpected, but I see
that it’s useful.

> The build.xml it generates contains a target “touch” which is run before
> wrapping up the compiled .class files in a jar archive; this target
> ensures that the timestamps of all archived files are reset, so the
> produced jars can be (and in case of the above-mentioned packages)
> deterministic.

Cool.

What should we do about packages that do provide a ‘build.xml’?  I
suppose their jars will most likely include timestamps by default,
right?

If that is the case, maybe we should instead add an additional phase
that would, say, unpack all the installed tarballs, reset timestamps,
and repack them?

> From 284f79ae6ba584603e58228c7c9ee73ac135912d Mon Sep 17 00:00:00 2001
> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
> Date: Tue, 29 Dec 2015 16:56:49 +0100
> Subject: [PATCH] build: Add Ant build system.
>
> * guix/build-system/ant.scm: New file.
> * guix/build/ant-build-system: New file.
> * Makefile.am (MODULES): Add new files.
> * doc/guix.texi (Build Systems): Document ant-build-system.

[...]

> +(define (default-build.xml jar-name prefix)
> +  "Create a simple build.xml with standard targets for Ant."
> +  (let ((file (open-output-file "build.xml")))

Slightly better to use:

  (call-with-output-file "build.xml"
    (lambda (port)
      ;; …
      ))

because it makes sure the port is closed whenever the dynamic extent of
the body is left.

> +(define* (configure #:key inputs outputs (jar-name #f)
> +                    #:allow-other-keys)
> +  (when jar-name
> +    (default-build.xml jar-name
> +      (string-append (assoc-ref outputs "out")

The second ‘default-build.xml’ argument should be aligned under the
first.

> +(define* (install #:key (make-flags '()) #:allow-other-keys)
> +  (zero? (apply system* `("ant" "install" ,@make-flags))))

Should we add a post-install phase or something that makes sure that
.jar files are always installed in the same place, say under lib/java,
similar to the ‘validate-documentation-location’?

Thank you!

Ludo’.

Re: [PATCH] Add ant-build-system.

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

>> The build.xml it generates contains a target “touch” which is run before
>> wrapping up the compiled .class files in a jar archive; this target
>> ensures that the timestamps of all archived files are reset, so the
>> produced jars can be (and in case of the above-mentioned packages)
>> deterministic.
>
> Cool.
>
> What should we do about packages that do provide a ‘build.xml’?  I
> suppose their jars will most likely include timestamps by default,
> right?
>
> If that is the case, maybe we should instead add an additional phase
> that would, say, unpack all the installed tarballs, reset timestamps,
> and repack them?

Yes, I think a generic build phase like that would be better.

>> From 284f79ae6ba584603e58228c7c9ee73ac135912d Mon Sep 17 00:00:00 2001
>> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
>> Date: Tue, 29 Dec 2015 16:56:49 +0100
>> Subject: [PATCH] build: Add Ant build system.
>>
>> * guix/build-system/ant.scm: New file.
>> * guix/build/ant-build-system: New file.
>> * Makefile.am (MODULES): Add new files.
>> * doc/guix.texi (Build Systems): Document ant-build-system.
>
> [...]
>
>> +(define (default-build.xml jar-name prefix)
>> +  "Create a simple build.xml with standard targets for Ant."
>> +  (let ((file (open-output-file "build.xml")))
>
> Slightly better to use:
>
>   (call-with-output-file "build.xml"
>     (lambda (port)
>       ;; …
>       ))
>
> because it makes sure the port is closed whenever the dynamic extent of
> the body is left.

Okay.

>> +(define* (configure #:key inputs outputs (jar-name #f)
>> +                    #:allow-other-keys)
>> +  (when jar-name
>> +    (default-build.xml jar-name
>> +      (string-append (assoc-ref outputs "out")
>
> The second ‘default-build.xml’ argument should be aligned under the
> first.

Oops!

>> +(define* (install #:key (make-flags '()) #:allow-other-keys)
>> +  (zero? (apply system* `("ant" "install" ,@make-flags))))
>
> Should we add a post-install phase or something that makes sure that
> .jar files are always installed in the same place, say under lib/java,
> similar to the ‘validate-documentation-location’?

I don’t really know much about the expectations that Java users may
have.  So far I’ve been installing jars to “$out/share/java/$package/”,
but I don’t know if this is a good idea (I also don’t know if this is a
bad idea).  Ultimately, programmes only care about what jars are in
their classpath.  I don’t know if there are any well-known directories.

Such a phase would probably only make sense from a Guix “quality
assurance” point of view, i.e. to enforce a Guix-internal policy.  I
don’t feel qualified to comment on such policy for Java libraries.

~~ Ricardo

Re: [PATCH] Add ant-build-system.

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]


Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> The build.xml it generates contains a target “touch” which is run before
>>> wrapping up the compiled .class files in a jar archive; this target
>>> ensures that the timestamps of all archived files are reset, so the
>>> produced jars can be (and in case of the above-mentioned packages)
>>> deterministic.
>>
>> Cool.
>>
>> What should we do about packages that do provide a ‘build.xml’?  I
>> suppose their jars will most likely include timestamps by default,
>> right?
>>
>> If that is the case, maybe we should instead add an additional phase
>> that would, say, unpack all the installed tarballs, reset timestamps,
>> and repack them?
>
> Yes, I think a generic build phase like that would be better.

I have addressed the other issues with the build system already, so
here’s just an additional patch that adds a generic “repack” build phase
as discussed.

I think it’s easier to review it this way, so I didn’t squash the
patches.  If these changes are okay I’ll fold them into the (corrected)
patch adding the ant-build-system and push.

~~ Ricardo



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Changes-to-the-ant-build-system.patch --]
[-- Type: text/x-patch, Size: 3833 bytes --]

From ddd1633f12cd53bbe6a8f2ccfbfa02678365f486 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
Date: Thu, 10 Mar 2016 11:50:52 +0100
Subject: [PATCH] Changes to the ant-build-system.

Do this in a new build phase after "install": Unpack jar, reset
timestamps, repack jar.
---
 guix/build-system/ant.scm       |  1 +
 guix/build/ant-build-system.scm | 37 +++++++++++++++++++++++++++++--------
 2 files changed, 30 insertions(+), 8 deletions(-)

diff --git a/guix/build-system/ant.scm b/guix/build-system/ant.scm
index 5240ff6..ac5546c 100644
--- a/guix/build-system/ant.scm
+++ b/guix/build-system/ant.scm
@@ -39,6 +39,7 @@
 (define %ant-build-system-modules
   ;; Build-side modules imported by default.
   `((guix build ant-build-system)
+    (guix build syscalls)
     ,@%gnu-build-system-modules))
 
 (define (default-jdk)
diff --git a/guix/build/ant-build-system.scm b/guix/build/ant-build-system.scm
index 1e3a1ea..7cb620b 100644
--- a/guix/build/ant-build-system.scm
+++ b/guix/build/ant-build-system.scm
@@ -18,6 +18,7 @@
 
 (define-module (guix build ant-build-system)
   #:use-module ((guix build gnu-build-system) #:prefix gnu:)
+  #:use-module (guix build syscalls)
   #:use-module (guix build utils)
   #:use-module (sxml simple)
   #:use-module (ice-9 match)
@@ -61,14 +62,8 @@
                                    (destdir "${classes.dir}")
                                    (classpath (@ (refid "classpath"))))))
 
-                 ;; Reset the ctime/mtime on all files to ensure that the
-                 ;; jar archive for the same class files is bit-identical.
-                 (target (@ (name "touch"))
-                         (exec (@ (executable "find"))
-                               (arg (@ (line "${classes.dir} -exec touch -d @0 {} ;")))))
-
                  (target (@ (name "jar")
-                            (depends "compile,touch"))
+                            (depends "compile"))
                          (mkdir (@ (dir "${jar.dir}")))
                          ;; We cannot use the simpler "jar" task here, because
                          ;; there is no way to disable generation of a
@@ -109,6 +104,31 @@ INPUTS."
                 #:allow-other-keys)
   (zero? (apply system* `("ant" ,build-target ,@make-flags))))
 
+(define* (repack #:key outputs
+                 #:allow-other-keys)
+  "Unpack all jar archives, reset the timestamp of all contained files, and
+repack them.  This is necessary to ensure that archives are reproducible."
+  (define (repack-archive jar)
+    (format #t "repacking ~a\n" jar)
+    (let ((dir (mkdtemp! "jar-contents.XXXXXX")))
+      (and (with-directory-excursion dir
+             (zero? (system* "jar" "xf" jar)))
+           ;; The manifest file contains timestamps
+           (for-each delete-file (find-files dir "MANIFEST.MF"))
+           (delete-file jar)
+           (ftw dir (lambda (file stat flag)
+                      (utime file 0 0)
+                      #t))
+           (format #t "~a\n" (string-join (list "jar" "-Mcf" jar "-C" dir ".")))
+           (zero? (system* "jar" "-Mcf" jar "-C" dir "."))
+           (utime jar 0 0)
+           #t)))
+
+  (every (match-lambda
+           ((output . directory)
+            (every repack-archive (find-files directory "\\.jar$"))))
+         outputs))
+
 (define* (check #:key target (make-flags '()) (tests? (not target))
                 (test-target "check")
                 #:allow-other-keys)
@@ -126,7 +146,8 @@ INPUTS."
     (replace 'configure configure)
     (replace 'build build)
     (replace 'check check)
-    (replace 'install install)))
+    (replace 'install install)
+    (add-after 'install 'repack repack)))
 
 (define* (ant-build #:key inputs (phases %standard-phases)
                     #:allow-other-keys #:rest args)
-- 
2.1.0

Re: [PATCH] Add ant-build-system.

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>>> The build.xml it generates contains a target “touch” which is run before
>>>> wrapping up the compiled .class files in a jar archive; this target
>>>> ensures that the timestamps of all archived files are reset, so the
>>>> produced jars can be (and in case of the above-mentioned packages)
>>>> deterministic.
>>>
>>> Cool.
>>>
>>> What should we do about packages that do provide a ‘build.xml’?  I
>>> suppose their jars will most likely include timestamps by default,
>>> right?
>>>
>>> If that is the case, maybe we should instead add an additional phase
>>> that would, say, unpack all the installed tarballs, reset timestamps,
>>> and repack them?
>>
>> Yes, I think a generic build phase like that would be better.
>
> I have addressed the other issues with the build system already, so
> here’s just an additional patch that adds a generic “repack” build phase
> as discussed.
>
> I think it’s easier to review it this way, so I didn’t squash the
> patches.  If these changes are okay I’ll fold them into the (corrected)
> patch adding the ant-build-system and push.

Thanks, it’s indeed easier this way.

> +(define* (repack #:key outputs
> +                 #:allow-other-keys)
> +  "Unpack all jar archives, reset the timestamp of all contained files, and
> +repack them.  This is necessary to ensure that archives are reproducible."

I would call it ‘strip-jar-timestamps’ for clarify.

(BTW, Debian’s ‘strip-nondeterminism’ tool does this kind of things too;
I’m not suggesting that we should use it here, though.)

> +  (define (repack-archive jar)
> +    (format #t "repacking ~a\n" jar)
> +    (let ((dir (mkdtemp! "jar-contents.XXXXXX")))
> +      (and (with-directory-excursion dir
> +             (zero? (system* "jar" "xf" jar)))
> +           ;; The manifest file contains timestamps
> +           (for-each delete-file (find-files dir "MANIFEST.MF"))
> +           (delete-file jar)
> +           (ftw dir (lambda (file stat flag)
> +                      (utime file 0 0)
> +                      #t))

I’d suggest copying ‘reset-timestamps’ from (gnu build install).
Eventually we’ll move it to (guix build utils).

> +           (format #t "~a\n" (string-join (list "jar" "-Mcf" jar "-C" dir ".")))
> +           (zero? (system* "jar" "-Mcf" jar "-C" dir "."))
> +           (utime jar 0 0)
> +           #t)))

The return value of ‘zero?’ is ignored.  What about making it:

  (unless (zero? …)
    (error "'jar' failed"))

OK with these changes!

Ludo’.

Re: [PATCH] Add ant-build-system.

[Pjotr Prins]

Awesome package. Thanks Ricardo, we will use it.

On Thu, Mar 10, 2016 at 05:17:59PM +0100, Ludovic Courtès wrote:
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
> 
> > Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
> >
> >> Ludovic Courtès <ludo@gnu.org> writes:
> >>
> >>>> The build.xml it generates contains a target “touch” which is run before
> >>>> wrapping up the compiled .class files in a jar archive; this target
> >>>> ensures that the timestamps of all archived files are reset, so the
> >>>> produced jars can be (and in case of the above-mentioned packages)
> >>>> deterministic.
> >>>
> >>> Cool.
> >>>
> >>> What should we do about packages that do provide a ‘build.xml’?  I
> >>> suppose their jars will most likely include timestamps by default,
> >>> right?
> >>>
> >>> If that is the case, maybe we should instead add an additional phase
> >>> that would, say, unpack all the installed tarballs, reset timestamps,
> >>> and repack them?
> >>
> >> Yes, I think a generic build phase like that would be better.
> >
> > I have addressed the other issues with the build system already, so
> > here’s just an additional patch that adds a generic “repack” build phase
> > as discussed.
> >
> > I think it’s easier to review it this way, so I didn’t squash the
> > patches.  If these changes are okay I’ll fold them into the (corrected)
> > patch adding the ant-build-system and push.
> 
> Thanks, it’s indeed easier this way.
> 
> > +(define* (repack #:key outputs
> > +                 #:allow-other-keys)
> > +  "Unpack all jar archives, reset the timestamp of all contained files, and
> > +repack them.  This is necessary to ensure that archives are reproducible."
> 
> I would call it ‘strip-jar-timestamps’ for clarify.
> 
> (BTW, Debian’s ‘strip-nondeterminism’ tool does this kind of things too;
> I’m not suggesting that we should use it here, though.)
> 
> > +  (define (repack-archive jar)
> > +    (format #t "repacking ~a\n" jar)
> > +    (let ((dir (mkdtemp! "jar-contents.XXXXXX")))
> > +      (and (with-directory-excursion dir
> > +             (zero? (system* "jar" "xf" jar)))
> > +           ;; The manifest file contains timestamps
> > +           (for-each delete-file (find-files dir "MANIFEST.MF"))
> > +           (delete-file jar)
> > +           (ftw dir (lambda (file stat flag)
> > +                      (utime file 0 0)
> > +                      #t))
> 
> I’d suggest copying ‘reset-timestamps’ from (gnu build install).
> Eventually we’ll move it to (guix build utils).
> 
> > +           (format #t "~a\n" (string-join (list "jar" "-Mcf" jar "-C" dir ".")))
> > +           (zero? (system* "jar" "-Mcf" jar "-C" dir "."))
> > +           (utime jar 0 0)
> > +           #t)))
> 
> The return value of ‘zero?’ is ignored.  What about making it:
> 
>   (unless (zero? …)
>     (error "'jar' failed"))
> 
> OK with these changes!
> 
> Ludo’.
> 

-- 

Re: [PATCH] Add ant-build-system.

[Ricardo Wurmus]

> OK with these changes!

Thanks.  I performed the requested changes (including a fix to
“reset-timestamps” to include directories).  Pushed as 5f7a1a4.

~~ Ricardo

Re: [PATCH] Add ant-build-system.

[Roel Janssen]

Ricardo Wurmus writes:

>> OK with these changes!
>
> Thanks.  I performed the requested changes (including a fix to
> “reset-timestamps” to include directories).  Pushed as 5f7a1a4.

Great to see this.  Thanks!

Re: [PATCH] Add ant-build-system.

[Alex Vong]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

>> OK with these changes!
>
> Thanks.  I performed the requested changes (including a fix to
> “reset-timestamps” to include directories).  Pushed as 5f7a1a4.
>
> ~~ Ricardo

Thank you. I will re-write the clojure patch using ant-build-system.