all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible.
@ 2021-06-09 12:56 Lars-Dominik Braun
  2021-06-16 20:51 ` Ludovic Courtès
  0 siblings, 1 reply; 5+ messages in thread
From: Lars-Dominik Braun @ 2021-06-09 12:56 UTC (permalink / raw)
  To: 48933; +Cc: jlicht

package.json records two hashes of package.tgz, which change for each
build, resulting in non-reproducible builds.

* guix/build/node-build-system.scm (repack): Add reproducibility options
to tar command.
---
 guix/build/node-build-system.scm | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index a55cab237c..9b3de43e24 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -120,7 +120,15 @@
   #t)
 
 (define* (repack #:key inputs #:allow-other-keys)
-  (invoke "tar" "-czf" "../package.tgz" ".")
+  (invoke "tar"
+          ;; Add options suggested by https://reproducible-builds.org/docs/archives/
+          "--sort=name"
+          (string-append "--mtime=" (getenv "SOURCE_DATE_EPOCH"))
+          "--owner=0"
+          "--group=0"
+          "--numeric-owner"
+          "--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime"
+          "-czf" "../package.tgz" ".")
   #t)
 
 (define* (install #:key outputs inputs #:allow-other-keys)
-- 
2.31.1





^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible.
  2021-06-09 12:56 [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible Lars-Dominik Braun
@ 2021-06-16 20:51 ` Ludovic Courtès
  2021-06-17 12:12   ` Lars-Dominik Braun
  0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2021-06-16 20:51 UTC (permalink / raw)
  To: Lars-Dominik Braun; +Cc: 48933, jlicht

Hi,

Lars-Dominik Braun <lars@6xq.net> skribis:

> package.json records two hashes of package.tgz, which change for each
> build, resulting in non-reproducible builds.
>
> * guix/build/node-build-system.scm (repack): Add reproducibility options
> to tar command.

Yay!

>  (define* (repack #:key inputs #:allow-other-keys)
> -  (invoke "tar" "-czf" "../package.tgz" ".")
> +  (invoke "tar"
> +          ;; Add options suggested by https://reproducible-builds.org/docs/archives/
> +          "--sort=name"
> +          (string-append "--mtime=" (getenv "SOURCE_DATE_EPOCH"))

I think it should be "--mtime=@".

> +          "--owner=0"
> +          "--group=0"
> +          "--numeric-owner"
> +          "--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime"
> +          "-czf" "../package.tgz" ".")

I didn’t know about this ‘--pax-option’ trick; since it’s only useful
when POSIXLY_CORRECT is set, perhaps we can remove it?

(guix docker) does this:

--8<---------------cut here---------------start------------->8---
(define %tar-determinism-options
  ;; GNU tar options to produce archives deterministically.
  '("--sort=name" "--mtime=@1"
    "--owner=root:0" "--group=root:0"

    ;; When 'build-docker-image' is passed store items, the 'nlink' of the
    ;; files therein leads tar to store hard links instead of actual copies.
    ;; However, the 'nlink' count depends on deduplication in the store; it's
    ;; an "implicit input" to the build process.  '--hard-dereference'
    ;; eliminates it.
    "--hard-dereference"))
--8<---------------cut here---------------end--------------->8---

and (guix packages) does something similar.

So ‘--sort=name’ seems to be missing.

HTH,
Ludo’.




^ permalink raw reply	[flat|nested] 5+ messages in thread

* [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible.
  2021-06-16 20:51 ` Ludovic Courtès
@ 2021-06-17 12:12   ` Lars-Dominik Braun
  2021-06-20 20:58     ` Ludovic Courtès
  0 siblings, 1 reply; 5+ messages in thread
From: Lars-Dominik Braun @ 2021-06-17 12:12 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 48933, jlicht

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

Hi Ludo,

> >  (define* (repack #:key inputs #:allow-other-keys)
> > -  (invoke "tar" "-czf" "../package.tgz" ".")
> > +  (invoke "tar"
> > +          ;; Add options suggested by https://reproducible-builds.org/docs/archives/
*
> > +          "--sort=name"
> > +          (string-append "--mtime=" (getenv "SOURCE_DATE_EPOCH"))
> 
> I think it should be "--mtime=@".
you’re right, fixed.

> I didn’t know about this ‘--pax-option’ trick; since it’s only useful
> when POSIXLY_CORRECT is set, perhaps we can remove it?
True, removed.

> (guix docker) does this:
> and (guix packages) does something similar.
Hm, maybe it would make sense to export a set of options, so
build systems/packages can share them? Or create a package that wraps
tar with the proper options?

> So ‘--sort=name’ seems to be missing.
It’s present, see above ↑*

Updated patch attached. I’ll push it if there are no further comments.

Cheers,
Lars


[-- Attachment #2: 0001-build-Make-outputs-of-node-build-system-reproducible.patch --]
[-- Type: text/x-diff, Size: 1242 bytes --]

From 6692ebc8561d4e419c276fcdd01a4088d29c5fd7 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <lars@6xq.net>
Date: Thu, 17 Jun 2021 14:11:19 +0200
Subject: [PATCH] build: Make outputs of node-build-system reproducible.

package.json records two hashes of package.tgz, which change for each
build, resulting in non-reproducible builds.

* guix/build/node-build-system.scm (repack): Add reproducibility options
to tar command.
---
 guix/build/node-build-system.scm | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index a55cab237c..70a367618e 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -120,7 +120,14 @@
   #t)
 
 (define* (repack #:key inputs #:allow-other-keys)
-  (invoke "tar" "-czf" "../package.tgz" ".")
+  (invoke "tar"
+          ;; Add options suggested by https://reproducible-builds.org/docs/archives/
+          "--sort=name"
+          (string-append "--mtime=@" (getenv "SOURCE_DATE_EPOCH"))
+          "--owner=0"
+          "--group=0"
+          "--numeric-owner"
+          "-czf" "../package.tgz" ".")
   #t)
 
 (define* (install #:key outputs inputs #:allow-other-keys)
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible.
  2021-06-17 12:12   ` Lars-Dominik Braun
@ 2021-06-20 20:58     ` Ludovic Courtès
  2021-06-24 12:12       ` bug#48933: " Lars-Dominik Braun
  0 siblings, 1 reply; 5+ messages in thread
From: Ludovic Courtès @ 2021-06-20 20:58 UTC (permalink / raw)
  To: Lars-Dominik Braun; +Cc: 48933, jlicht

Hi,

Lars-Dominik Braun <lars@6xq.net> skribis:

[...]

>> (guix docker) does this:
>> and (guix packages) does something similar.
> Hm, maybe it would make sense to export a set of options, so
> build systems/packages can share them? Or create a package that wraps
> tar with the proper options?

Would be nice.

Even better would be to use a custom tar implementation (similar to
(guix cpio)) that would make it easier and less clunky to ensure
reproducibility.

>> So ‘--sort=name’ seems to be missing.
> It’s present, see above ↑*

Indeed.  :-)

> Updated patch attached. I’ll push it if there are no further comments.

Perfect, thanks!

Ludo’.




^ permalink raw reply	[flat|nested] 5+ messages in thread

* bug#48933: [PATCH] build: Make outputs of node-build-system reproducible.
  2021-06-20 20:58     ` Ludovic Courtès
@ 2021-06-24 12:12       ` Lars-Dominik Braun
  0 siblings, 0 replies; 5+ messages in thread
From: Lars-Dominik Braun @ 2021-06-24 12:12 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 48933-done, jlicht

Hi,

> > Updated patch attached. I’ll push it if there are no further comments.
> 
> Perfect, thanks!
pushed as 9c93573d15e90232de0effb4c28332c454dbc290.

Cheers,
Lars





^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-06-24 12:13 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-09 12:56 [bug#48933] [PATCH] build: Make outputs of node-build-system reproducible Lars-Dominik Braun
2021-06-16 20:51 ` Ludovic Courtès
2021-06-17 12:12   ` Lars-Dominik Braun
2021-06-20 20:58     ` Ludovic Courtès
2021-06-24 12:12       ` bug#48933: " Lars-Dominik Braun

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.