From: "Ludovic Courtès" <ludo@gnu.org>
To: Janneke Nieuwenhuizen <janneke@gnu.org>
Cc: 70169@debbugs.gnu.org
Subject: [bug#70169] [PATCH 0/7] Reproducible `make dist' tarball in defiance of Autotools and Gettext
Date: Wed, 03 Apr 2024 22:57:36 +0200 [thread overview]
Message-ID: <87wmpew4e7.fsf@gnu.org> (raw)
In-Reply-To: <cover.1712165977.git.janneke@gnu.org> (Janneke Nieuwenhuizen's message of "Wed, 3 Apr 2024 21:08:40 +0200")
Hi!
Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
> The recent XZ-utils <https://www.openwall.com/lists/oss-security/2024/03/29/4>
> debacle inspired me to resurrect and finish my patch set for creating a
> reproducible source tarball for Guix, i.e., finally have `make dist' be
> reproducible (when run from Git). I've been using a version of these patches
> in simpler projects for some years now and stole one from Timothy Samplet's
> Gash project.
Yay, kudos to you and Timothy!
> Autotools and Gettext still make it harder than necessary to do reproducible
> (responsible?) computing, which is especially sad given the fact that the
> Reproducible Builds project recently had their 10th birthday
> <https://reproducible-builds.org/_lfs/presentations/2023-05-27-R-B-the-first-10-years/#/>.
>
> Gettext tooling embeds timestamps found in the file-system, fails to respect
> SOURCE_DATE_EPOCH, and lacks options like `--pot-creation-date' so that we
> have to resort to SED to fixup. The caching of all sorts of information, in
> separate build stages, also doesn't help.
Sadness indeed. Hopefully things will improve in the coming weeks, now
that there’s an impetus.
> To create a reproducible source tarball, having a reproducible build
> environment is a prerequitite, so this would have to be recorded too.
> Using this patch set, I created a tarball doing something like
>
> guix pull --commit=1dbe492b993a7629df3b35146ce0272b52913776
> guix shell
> bootstrap && ./configure --localstatedir=/var --sysconfdir=/etc && make dist
> guix hash guix-1.3.0.57425-80a228.tar.gz
> 0mk59ay5k2dxmjni9fx4i8qyfhvlgxbhqzsjpg2pbw381nskkxbj
I applied the whole series on top of
df64d48e6f9f648044aa5279c045b8d6f7bee604 (the ‘base-commit’ at the
bottom of your message). Thus I got the same content as you but with a
different commit ID.
“make dist” gave me guix-1.3.0.57425-9f4a4a.tar.gz. The name indeed
corresponds to the tip of my tree:
--8<---------------cut here---------------start------------->8---
$ guix hash guix-1.3.0.57425-9f4a4a.tar.gz
0z3c4f8g6rsi9n0j8cwzwvw4bc59srg6bl3jj8yi60hbr9vrz5ql
$ git log |head
commit 9f4a4adfa778b281b794b61014e06dad98b6c945
Author: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Wed Apr 3 21:11:09 2024 +0200
maint: Ensure generated file reproducibility for dist.
* doc/local.mk (override $(srcdir)/doc/stamp-vti): New target override.
(doc-clean, man-clean): New targets.
(auto-clean): Depend on it in new target.
(DIST_CONFIGURE_FLAGS): New variable.
--8<---------------cut here---------------end--------------->8---
But as a result, I get a different hash, and since the directory in the
tarball has a different name, ‘diffoscope’ isn’t very helpful.
There’s at least one relevant difference in the gzip metadata:
--8<---------------cut here---------------start------------->8---
--- guix-1.3.0.57425-9f4a4a.tar.gz
+++ /tmp/guix-1.3.0.57425-80a228.tar.gz
├── filetype from file(1)
│ @@ -1 +1 @@
│ -gzip compressed data, from Unix, original size modulo 2^32 208138240 gzip compressed data, rese
rved method, ASCII, extra field, encrypted, from FAT filesystem (MS-DOS, OS/2, NT)
│ +gzip compressed data, from Unix, original size modulo 2^32 222504960 gzip compressed data, rese
rved method, ASCII, has CRC, was "", has comment, encrypted, from FAT filesystem (MS-DOS, OS/2, NT
--8<---------------cut here---------------end--------------->8---
(Your tarball has a CRC and comment, mine doesn’t.)
Maybe we’ll have to iterate once you’ve pushed a first version, so we
can truly build the same thing. Or we should push the branch somewhere
(or use the one from <https://data.qa.guix.gnu.org/> once it’s been
created).
Thanks!
Ludo’.
next prev parent reply other threads:[~2024-04-03 20:58 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 19:08 [bug#70169] [PATCH 0/7] Reproducible `make dist' tarball in defiance of Autotools and Gettext Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 1/7] maint: Cater for running `make dist' from a worktree Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 2/7] maint: Use reproducible timestamps and name for tarball Janneke Nieuwenhuizen
2024-04-03 20:45 ` Ludovic Courtès
2024-04-03 20:49 ` Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 3/7] maint: Generate 'doc/version.texi' reproducibly Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 4/7] maint: Help help2man generate reproducible man-pages Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 5/7] maint: Generate 'doc/version-LANG.texi' reproducibly Janneke Nieuwenhuizen
2024-04-03 20:25 ` Ludovic Courtès
2024-04-03 20:33 ` Janneke Nieuwenhuizen
2024-04-08 9:26 ` Ludovic Courtès
2024-04-03 20:39 ` Ludovic Courtès
2024-04-03 20:43 ` Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 6/7] maint: Use reproducible Git timestamp for POT-Creation-Date Janneke Nieuwenhuizen
2024-04-03 20:26 ` Ludovic Courtès
2024-04-03 20:36 ` Janneke Nieuwenhuizen
2024-04-04 5:12 ` Janneke Nieuwenhuizen
2024-04-03 19:11 ` [bug#70169] [PATCH 7/7] maint: Ensure generated file reproducibility for dist Janneke Nieuwenhuizen
2024-04-03 20:57 ` Ludovic Courtès [this message]
2024-04-03 21:04 ` [bug#70169] [PATCH 0/7] Reproducible `make dist' tarball in defiance of Autotools and Gettext Janneke Nieuwenhuizen
2024-04-03 21:28 ` Ludovic Courtès
2024-04-06 21:16 ` Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 00/12] " Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 01/12] maint: Cater for running `make dist' from a worktree Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 02/12] maint: Use reproducible timestamps and name for tarball Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 03/12] maint: Generate 'doc/version.texi' reproducibly Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 04/12] maint: Help help2man generate reproducible man-pages Janneke Nieuwenhuizen
2024-04-07 13:54 ` pelzflorian (Florian Pelz)
2024-04-07 14:06 ` Janneke Nieuwenhuizen
2024-04-07 15:20 ` pelzflorian (Florian Pelz)
2024-04-08 9:23 ` Ludovic Courtès
2024-04-08 9:40 ` Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 05/12] maint: Generate AUTHORS and ChangeLog reproducibly Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 06/12] maint: Generate 'doc/version-LANG.texi' reproducibly Janneke Nieuwenhuizen
2024-04-06 21:17 ` [bug#70169] [PATCH v2 07/12] maint: Use xgettext.scm wrapper to create .PO files reproducibly Janneke Nieuwenhuizen
2024-04-06 21:18 ` [bug#70169] [PATCH v2 08/12] maint: Use reproducible Git timestamp for POT-Creation-Date Janneke Nieuwenhuizen
2024-04-06 21:18 ` [bug#70169] [PATCH v2 09/12] maint: Remove %%CreationDate from generated EPS files Janneke Nieuwenhuizen
2024-04-08 9:41 ` Ludovic Courtès
2024-04-08 17:12 ` Janneke Nieuwenhuizen
2024-04-06 21:18 ` [bug#70169] [PATCH v2 10/12] maint: Avoid PNG and PDF generation to fail silently Janneke Nieuwenhuizen
2024-04-06 21:18 ` [bug#70169] [PATCH v2 11/12] maint: Reset CreationDate metadata on generated PDFs Janneke Nieuwenhuizen
2024-04-07 17:16 ` pelzflorian (Florian Pelz)
2024-04-07 17:38 ` Janneke Nieuwenhuizen
2024-04-07 18:09 ` Janneke Nieuwenhuizen
2024-04-06 21:18 ` [bug#70169] [PATCH v2 12/12] maint: Ensure generated file reproducibility for dist Janneke Nieuwenhuizen
2024-04-07 8:57 ` [bug#70169] [PATCH v2 13/12] doc: Use "dejavu sans" instead of "Helvetica" or "sans" in dot images Janneke Nieuwenhuizen
2024-04-08 12:15 ` Ludovic Courtès
2024-04-08 17:26 ` Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 00/13] Reproducible `make dist' tarball in defiance of Autotools and Gettext Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 01/13] doc: Use "dejavu sans" instead of "Helvetica" or "sans" in dot images Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 02/13] maint: Cater for running `make dist' from a worktree Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 03/13] maint: Use reproducible timestamps and name for tarball Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 04/13] maint: Generate 'doc/version.texi' reproducibly Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 05/13] maint: Help help2man generate reproducible man-pages Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 06/13] maint: Generate AUTHORS and ChangeLog reproducibly Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 07/13] maint: Generate 'doc/version-LANG.texi' reproducibly Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 08/13] maint: Use xgettext.scm wrapper to create .PO files reproducibly Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 09/13] maint: Use reproducible Git timestamp for POT-Creation-Date Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 10/13] maint: Avoid EPS generation to fail silently, ensure reproducibility Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 11/13] maint: Avoid PNG and PDF generation to fail silently Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 12/13] maint: Reset CreationDate metadata on generated PDFs Janneke Nieuwenhuizen
2024-04-08 18:46 ` [bug#70169] [PATCH v3 13/13] maint: Ensure generated file reproducibility for dist Janneke Nieuwenhuizen
2024-04-10 16:35 ` pelzflorian (Florian Pelz)
2024-04-10 17:25 ` Janneke Nieuwenhuizen
2024-04-11 11:32 ` pelzflorian (Florian Pelz)
2024-04-14 9:24 ` bug#70169: " Janneke Nieuwenhuizen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wmpew4e7.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=70169@debbugs.gnu.org \
--cc=janneke@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.