* "jar" tool and and reproducible builds - how?
@ 2019-11-04 17:18 Danny Milosavljevic
2019-11-04 21:43 ` Gábor Boskovits
2019-11-05 8:20 ` Hartmut Goebel
0 siblings, 2 replies; 7+ messages in thread
From: Danny Milosavljevic @ 2019-11-04 17:18 UTC (permalink / raw)
To: guix-devel, Gábor Boskovits
[-- Attachment #1.1: Type: text/plain, Size: 748 bytes --]
Hi,
I'm trying to package nesc which has some small tools written in Java.
It compiles them using "javac" and then packs the result using "jar".
If I want
guix build --rounds=2 nesc
to pass, how do I best proceed?
I tried
(substitute* "tools/Makefile.in"
((" jar cf \\.\\./nesc.jar")
" zip -0 -X ../nesc.jar META-INF/MANIFEST.MF "))
but that doesn't seem to be enough.
WIP attached.
Furthermore, I think it would be nice to patch our "jar" tool in icedtea:jdk
so that it provides an option to disable timestamps (not put the timestamp
field into the generated ZIP file in the first place). I've thus unpacked
icedtea but I can't find where the source code to "jar" is.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: nesc.scm --]
[-- Type: text/x-scheme, Size: 1664 bytes --]
(define-module (wip nesc)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system gnu)
#:use-module (guix download)
#:use-module (guix packages)
#:use-module (gnu packages compression)
#:use-module (gnu packages java)
#:use-module (gnu packages perl))
(define-public nesc
(package
(name "nesc")
(version "1.3.4")
(source (origin
(method url-fetch)
(uri (string-append "mirror://sourceforge/nescc/nescc/v" version
"/nesc-" version ".tar.gz"))
(sha256
(base32
"0n4mjnm6q96s8pkdq2n9z6cxf5xpy2386cfn30wm4if9gdwhc3w7"))))
(build-system gnu-build-system)
(native-inputs
`(("icedtea" ,icedtea "jdk") ; for tools/java
("perl" ,perl)
("zip" ,zip)))
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'make-jar-reproducible
(lambda _
;; Make JAR file "lib/ncc/nesc.jar" reproducible.
;(substitute* "tools/Makefile.in"
; ((" jar") " zip -0 -X META-INF/MANIFEST.MF "))
(substitute* "tools/Makefile.in"
((" jar cf \\.\\./nesc.jar")
" zip -0 -X ../nesc.jar META-INF/MANIFEST.MF "))
#t)))))
(synopsis "nesc compiler")
(description "This package provides a C-like programming language with
small extensions to C, notably a module system (including interfaces),
concurrency and platform-independent networking.")
(home-page "http://nescc.sourceforge.net/")
(license license:gpl2)))
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-04 17:18 "jar" tool and and reproducible builds - how? Danny Milosavljevic
@ 2019-11-04 21:43 ` Gábor Boskovits
2019-11-05 9:49 ` Danny Milosavljevic
2019-11-05 8:20 ` Hartmut Goebel
1 sibling, 1 reply; 7+ messages in thread
From: Gábor Boskovits @ 2019-11-04 21:43 UTC (permalink / raw)
To: Danny Milosavljevic; +Cc: Guix-devel
[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]
Hello Danny,
Danny Milosavljevic <dannym@scratchpost.org> ezt írta (időpont: 2019. nov.
4., H, 18:18):
> Hi,
>
> I'm trying to package nesc which has some small tools written in Java.
> It compiles them using "javac" and then packs the result using "jar".
>
> If I want
>
> guix build --rounds=2 nesc
>
> to pass, how do I best proceed?
>
> I tried
>
> (substitute* "tools/Makefile.in"
> ((" jar cf \\.\\./nesc.jar")
> " zip -0 -X ../nesc.jar META-INF/MANIFEST.MF "))
>
> but that doesn't seem to be enough.
>
>
What diff remains there? There might be more jars or jmods inside...
Most of the trickery should be in ant build system.
> WIP attached.
>
> Furthermore, I think it would be nice to patch our "jar" tool in
> icedtea:jdk
> so that it provides an option to disable timestamps (not put the timestamp
> field into the generated ZIP file in the first place). I've thus unpacked
> icedtea but I can't find where the source code to "jar" is.
>
I don't know right now, but have a look later.
Best regards,
g_bor
--
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
[-- Attachment #2: Type: text/html, Size: 1988 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-04 21:43 ` Gábor Boskovits
@ 2019-11-05 9:49 ` Danny Milosavljevic
2019-11-05 11:53 ` Gábor Boskovits
0 siblings, 1 reply; 7+ messages in thread
From: Danny Milosavljevic @ 2019-11-05 9:49 UTC (permalink / raw)
To: Gábor Boskovits; +Cc: Guix-devel
[-- Attachment #1: Type: text/plain, Size: 689 bytes --]
Hi Gábor,
> What diff remains there? There might be more jars or jmods inside...
> Most of the trickery should be in ant build system.
This package does not use ant-build-system or any other java build system since
there are only very few Java tools in nesc (I'm thinking of disabling those,
too).
diffoscope states only the following, via "zipinfo -v" on "lib/ncc/nesc.jar":
- file last modified on (DOS date/time): 2019 Nov 4 17:10:26
+ file last modified on (DOS date/time): 2019 Nov 4 17:10:56
It's repeated a lot of times.
No other files but "lib/ncc/nesc.jar" are affected.
> I don't know right now, but have a look later.
Thanks!
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-05 9:49 ` Danny Milosavljevic
@ 2019-11-05 11:53 ` Gábor Boskovits
2019-11-05 15:09 ` Danny Milosavljevic
0 siblings, 1 reply; 7+ messages in thread
From: Gábor Boskovits @ 2019-11-05 11:53 UTC (permalink / raw)
To: Danny Milosavljevic; +Cc: Guix-devel
[-- Attachment #1: Type: text/plain, Size: 1272 bytes --]
Hello,
Danny Milosavljevic <dannym@scratchpost.org> ezt írta (időpont: 2019. nov.
5., K, 10:49):
> Hi Gábor,
>
> > What diff remains there? There might be more jars or jmods inside...
> > Most of the trickery should be in ant build system.
>
> This package does not use ant-build-system or any other java build system
> since
> there are only very few Java tools in nesc (I'm thinking of disabling
> those,
> too).
>
> diffoscope states only the following, via "zipinfo -v" on
> "lib/ncc/nesc.jar":
>
> - file last modified on (DOS date/time): 2019 Nov 4 17:10:26
> + file last modified on (DOS date/time): 2019 Nov 4 17:10:56
>
This looks like a jar timestamps issue. Adding a phase similar to
strip-jar-timestamps in ant-build-system should work.
It extracts the jar to a temporary directory, resets file timestamps, and
then repacks it, and resets the timestamp on the archive.
Could you check if that works for you?
>
> It's repeated a lot of times.
>
> No other files but "lib/ncc/nesc.jar" are affected.
> apply git send email
> > I don't know right now, but have a look later.
>
> Thanks!
>
Best regards,
g_bor
--
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
[-- Attachment #2: Type: text/html, Size: 1980 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-05 11:53 ` Gábor Boskovits
@ 2019-11-05 15:09 ` Danny Milosavljevic
2019-11-05 17:17 ` Gábor Boskovits
0 siblings, 1 reply; 7+ messages in thread
From: Danny Milosavljevic @ 2019-11-05 15:09 UTC (permalink / raw)
To: Gábor Boskovits; +Cc: Guix-devel
[-- Attachment #1: Type: text/plain, Size: 1381 bytes --]
Hi Gábor,
> This looks like a jar timestamps issue. Adding a phase similar to
> strip-jar-timestamps in ant-build-system should work.
> It extracts the jar to a temporary directory, resets file timestamps, and
> then repacks it, and resets the timestamp on the archive.
> Could you check if that works for you?
Yes, the following works and makes it reproducible:
(arguments
`(#:imported-modules
((guix build utils)
(guix build ant-build-system)
(guix build gremlin)
(guix build syscalls)
(guix elf)
(guix build gnu-build-system))
#:phases
(modify-phases %standard-phases
(add-after 'install 'strip-jar-timestamps
(assoc-ref (@ (guix build ant-build-system) %standard-phases)
'strip-jar-timestamps)))))
But I'd prefer if our "jar" tool did that on its own if SOURCE_DATE_EPOCH
is set--I don't think the code above is maintainable.
The following code does not work (doesn't find ant-build-system):
(arguments
`(#:modules
((guix build gnu-build-system)
((guix build ant-build-system) #:prefix ant:)
(guix build utils))
#:phases
(modify-phases %standard-phases
(add-after 'install 'strip-jar-timestamps
(assoc-ref ant:%standard-phases 'strip-jar-timestamps)))))
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-05 15:09 ` Danny Milosavljevic
@ 2019-11-05 17:17 ` Gábor Boskovits
0 siblings, 0 replies; 7+ messages in thread
From: Gábor Boskovits @ 2019-11-05 17:17 UTC (permalink / raw)
To: Danny Milosavljevic; +Cc: Guix-devel
[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]
Hello,
Danny Milosavljevic <dannym@scratchpost.org> ezt írta (időpont: 2019. nov.
5., Ke 16:09):
> Hi Gábor,
>
> > This looks like a jar timestamps issue. Adding a phase similar to
> > strip-jar-timestamps in ant-build-system should work.
> > It extracts the jar to a temporary directory, resets file timestamps, and
> > then repacks it, and resets the timestamp on the archive.
> > Could you check if that works for you?
>
> Yes, the following works and makes it reproducible:
>
> (arguments
> `(#:imported-modules
> ((guix build utils)
> (guix build ant-build-system)
> (guix build gremlin)
> (guix build syscalls)
> (guix elf)
> (guix build gnu-build-system))
> #:phases
> (modify-phases %standard-phases
> (add-after 'install 'strip-jar-timestamps
> (assoc-ref (@ (guix build ant-build-system) %standard-phases)
> 'strip-jar-timestamps)))))
>
That is great.
>
> But I'd prefer if our "jar" tool did that on its own if SOURCE_DATE_EPOCH
> is set--I don't think the code above is maintainable.
>
Agreed. Could you open a wishlist bug for that, so it is not forgotten.
>
> The following code does not work (doesn't find ant-build-system):
>
> (arguments
> `(#:modules
> ((guix build gnu-build-system)
> ((guix build ant-build-system) #:prefix ant:)
> (guix build utils))
> #:phases
> (modify-phases %standard-phases
> (add-after 'install 'strip-jar-timestamps
> (assoc-ref ant:%standard-phases 'strip-jar-timestamps)))))
>
I believe this strip-jar-timestamps code should be moved to the java utils
module, so that it can be easier to reuse...
Best regards,g_bor
>
[-- Attachment #2: Type: text/html, Size: 2909 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: "jar" tool and and reproducible builds - how?
2019-11-04 17:18 "jar" tool and and reproducible builds - how? Danny Milosavljevic
2019-11-04 21:43 ` Gábor Boskovits
@ 2019-11-05 8:20 ` Hartmut Goebel
1 sibling, 0 replies; 7+ messages in thread
From: Hartmut Goebel @ 2019-11-05 8:20 UTC (permalink / raw)
To: guix-devel
Am 04.11.19 um 18:18 schrieb Danny Milosavljevic:
> I tried
>
> (substitute* "tools/Makefile.in"
> ((" jar cf \\.\\./nesc.jar")
> " zip -0 -X ../nesc.jar META-INF/MANIFEST.MF "))
>
> but that doesn't seem to be enough.
The ant-build-system contains phases dealing with this. Have a look at
guix/buid/ant-build-system.scm, near the bottom. Noteble this runs
"strip-jar-timestamps twice!
--
Regards
Hartmut Goebel
| Hartmut Goebel | h.goebel@crazy-compilers.com |
| www.crazy-compilers.com | compilers which you thought are impossible |
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-11-05 17:17 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-11-04 17:18 "jar" tool and and reproducible builds - how? Danny Milosavljevic
2019-11-04 21:43 ` Gábor Boskovits
2019-11-05 9:49 ` Danny Milosavljevic
2019-11-05 11:53 ` Gábor Boskovits
2019-11-05 15:09 ` Danny Milosavljevic
2019-11-05 17:17 ` Gábor Boskovits
2019-11-05 8:20 ` Hartmut Goebel
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.