* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives @ 2021-01-18 17:29 Ludovic Courtès 2021-03-03 13:48 ` Ludovic Courtès 0 siblings, 1 reply; 7+ messages in thread From: Ludovic Courtès @ 2021-01-18 17:29 UTC (permalink / raw) To: 45962 Hi! On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in commencement.scm lacks ‘--enable-deterministic-archives’. So I checked if this had an effect by running: guix build -e '(@@ (gnu packages commencement) gcc-core-mesboot0)' \ --check -K and yes, it does: --8<---------------cut here---------------start------------->8--- $ diff -ru /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3{,-check} Binary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libc.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libc.a differ Binary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libgcc.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libgcc.a differ Binary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/libgcc2.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/libgcc2.a differ Binary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/libiberty.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/libiberty.a differ --8<---------------cut here---------------end--------------->8--- Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’ so we’ll have to patch it. There are a few other Binutils variants in commencement.scm that we’ll have to check. Ludo’. ^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-01-18 17:29 bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives Ludovic Courtès @ 2021-03-03 13:48 ` Ludovic Courtès 2021-03-03 17:54 ` Jan Nieuwenhuizen 2021-03-04 16:14 ` Maxim Cournoyer 0 siblings, 2 replies; 7+ messages in thread From: Ludovic Courtès @ 2021-03-03 13:48 UTC (permalink / raw) To: 45962; +Cc: Maxim Cournoyer [-- Attachment #1: Type: text/plain, Size: 3781 bytes --] Hi, Ludovic Courtès <ludo@gnu.org> skribis: > On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in > commencement.scm lacks ‘--enable-deterministic-archives’. So I checked > if this had an effect by running: [...] > Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’ > so we’ll have to patch it. Sikonas on #bootstrappable provided a patch for that (thanks!) so I went ahead and gave it a try on ‘core-updates’ (Guix patch attached). The binutils source gets patched and repacked, but then decompressing it fails: --8<---------------cut here---------------start------------->8--- building /gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv... starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds starting phase `set-paths' environment variable `PATH' set to `/gnu/store/70w6gnl3vzv2rsjhyjg0wsxl3pbnpjb5-bash-mesboot0-2.05b/bin:/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/bin:/gnu/store/4nxicfzj8a390f4scxcgglhdqgbyhlkw-gzip-mesboot-1.2.4/bin:/gnu/store/6fgjdcl2qrrs7fvdznkx1gcbw6wjfrn0-patch-mesboot-2.5.9/bin:/gnu/store/4lpagdav2gm022v1fgzcs8323xrggz4b-sed-mesboot0-1.18/bin:/gnu/store/y70rava2vqf1nilr8smg54qqxdclvrz6-gash-utils-boot-0.1.0/bin:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/bin:/gnu/store/krnwfrki71dj6zicl2qwv6bdyqvsxgcg-make-mesboot0-3.80/bin:/gnu/store/xxnxdnjdlav4s8v3lfhg7x7amrqcrv57-gash-boot-0.2.0/bin:/gnu/store/aglbpf6bihv35pwpvdy6chxv318hsafq-bootar-1a/bin:/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin' environment variable `BASH_LOADABLES_PATH' unset environment variable `C_INCLUDE_PATH' set to `/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/include:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/include' environment variable `LIBRARY_PATH' set to `/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/lib:/gnu/store/y70rava2vqf1nilr8smg54qqxdclvrz6-gash-utils-boot-0.1.0/lib:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/lib:/gnu/store/xxnxdnjdlav4s8v3lfhg7x7amrqcrv57-gash-boot-0.2.0/lib:/gnu/store/aglbpf6bihv35pwpvdy6chxv318hsafq-bootar-1a/lib:/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/lib' phase `set-paths' succeeded after 0.0 seconds starting phase `install-locale' warning: failed to install 'en_US.utf8' locale: Invalid argument phase `install-locale' succeeded after 0.0 seconds starting phase `unpack' xz: cannot decompress from stdin Backtrace: In ice-9/boot-9.scm: 157: 5 [catch #t #<catch-closure c928e0> ...] In unknown file: ?: 4 [apply-smob/1 #<catch-closure c928e0>] In ice-9/boot-9.scm: 63: 3 [call-with-prompt prompt0 ...] In ice-9/eval.scm: 432: 2 [eval # #] In gash/guix-utils.scm: 138: 1 [call-with-decompressed-port xz ...] In unknown file: ?: 0 [scm-error misc-error #f "~A ~S" ("decompressed-port failure" (7)) #f] ERROR: In procedure scm-error: ERROR: decompressed-port failure (7) error: in phase 'unpack': uncaught exception: srfi-34 #<condition &invoke-error [program: "tar" arguments: ("xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz") exit-status: 1 term-signal: #f stop-signal: #f] 1215400> phase `unpack' failed after 0.3 seconds command "tar" "xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz" failed with status 1 builder for `/gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv' failed with exit code 1 --8<---------------cut here---------------end--------------->8--- Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is less capable, or could it be a Gash-Utils bug? Thanks, Ludo’. [-- Attachment #2: Type: text/x-patch, Size: 4024 bytes --] diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm index 890d57941f..5c523ae7ad 100644 --- a/gnu/packages/commencement.scm +++ b/gnu/packages/commencement.scm @@ -997,13 +997,15 @@ $MES -e '(mescc)' module/mescc.scm -- \"$@\" (inherit binutils) (name "binutils-mesboot0") (version "2.14") - (source (origin - (method url-fetch) - (uri (string-append "mirror://gnu/binutils/binutils-" - version ".tar.gz")) - (sha256 - (base32 - "1w8xp7k44bkijr974x9918i4p1sw4g2fcd5mxvspkjpg38m214ds")))) + (source (bootstrap-origin + (origin + (method url-fetch) + (uri (string-append "mirror://gnu/binutils/binutils-" + version ".tar.gz")) + (sha256 + (base32 + "1w8xp7k44bkijr974x9918i4p1sw4g2fcd5mxvspkjpg38m214ds")) + (patches (search-patches "binutils-2.14-deterministic-ar.patch"))))) (inputs '()) (propagated-inputs '()) (native-inputs (%boot-tcc-inputs)) diff --git a/gnu/packages/patches/binutils-2.14-deterministic-ar.patch b/gnu/packages/patches/binutils-2.14-deterministic-ar.patch new file mode 100644 index 0000000000..7f9310e994 --- /dev/null +++ b/gnu/packages/patches/binutils-2.14-deterministic-ar.patch @@ -0,0 +1,66 @@ +Old binutils do not have support for creating deterministic archives. +Backported from upstream commit 36e4dce69dd23bea9ea2258dea35f034b6d6351c + +Patch by Chris Demetriou <cgd@google.com> (2009), adapted by +Andrius Štikonas <andrius@stikonas.eu> (2021). + +--- a/bfd/archive.c 2021-03-01 00:05:54.888301655 +0000 ++++ b/bfd/archive.c 2021-03-02 21:53:51.001617689 +0000 +@@ -1396,10 +1396,6 @@ + { + /* Assume we just "made" the member, and fake it. */ + struct bfd_in_memory *bim = (struct bfd_in_memory *) member->iostream; +- time (&status.st_mtime); +- status.st_uid = getuid (); +- status.st_gid = getgid (); +- status.st_mode = 0644; + status.st_size = bim->size; + } + else if (stat (filename, &status) != 0) +@@ -1408,6 +1404,11 @@ + return NULL; + } + ++ status.st_mtime = 0; ++ status.st_uid = 0; ++ status.st_gid = 0; ++ status.st_mode = 0644; ++ + amt = sizeof (struct ar_hdr) + sizeof (struct areltdata); + ared = (struct areltdata *) bfd_zalloc (abfd, amt); + if (ared == NULL) +@@ -2003,13 +2004,11 @@ + stat (arch->filename, &statbuf); + memset ((char *) (&hdr), 0, sizeof (struct ar_hdr)); + sprintf (hdr.ar_name, RANLIBMAG); +- /* Remember the timestamp, to keep it holy. But fudge it a little. */ +- bfd_ardata (arch)->armap_timestamp = statbuf.st_mtime + ARMAP_TIME_OFFSET; + bfd_ardata (arch)->armap_datepos = (SARMAG + + offsetof (struct ar_hdr, ar_date[0])); +- sprintf (hdr.ar_date, "%ld", bfd_ardata (arch)->armap_timestamp); +- sprintf (hdr.ar_uid, "%ld", (long) getuid ()); +- sprintf (hdr.ar_gid, "%ld", (long) getgid ()); ++ sprintf (hdr.ar_date, "%ld", 0); ++ sprintf (hdr.ar_uid, "%ld", 0); ++ sprintf (hdr.ar_gid, "%ld", 0); + sprintf (hdr.ar_size, "%-10d", (int) mapsize); + strncpy (hdr.ar_fmag, ARFMAG, 2); + for (i = 0; i < sizeof (struct ar_hdr); i++) +@@ -2082,6 +2081,8 @@ + struct ar_hdr hdr; + unsigned int i; + ++ return TRUE; ++ + /* Flush writes, get last-write timestamp from file, and compare it + to the timestamp IN the file. */ + bfd_flush (arch); +@@ -2169,7 +2170,7 @@ + memset ((char *) (&hdr), 0, sizeof (struct ar_hdr)); + hdr.ar_name[0] = '/'; + sprintf (hdr.ar_size, "%-10d", (int) mapsize); +- sprintf (hdr.ar_date, "%ld", (long) time (NULL)); ++ sprintf (hdr.ar_date, "%ld", 0); + /* This, at least, is what Intel coff sets the values to. */ + sprintf ((hdr.ar_uid), "%d", 0); + sprintf ((hdr.ar_gid), "%d", 0); ^ permalink raw reply related [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-03-03 13:48 ` Ludovic Courtès @ 2021-03-03 17:54 ` Jan Nieuwenhuizen 2021-03-04 16:39 ` Maxim Cournoyer 2021-03-08 14:13 ` Ludovic Courtès 2021-03-04 16:14 ` Maxim Cournoyer 1 sibling, 2 replies; 7+ messages in thread From: Jan Nieuwenhuizen @ 2021-03-03 17:54 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45962, Maxim Cournoyer Ludovic Courtès writes: Hello! > Ludovic Courtès <ludo@gnu.org> skribis: > >> On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in >> commencement.scm lacks ‘--enable-deterministic-archives’. So I checked >> if this had an effect by running: > [...] > >> Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’ >> so we’ll have to patch it. > > Sikonas on #bootstrappable provided a patch for that (thanks!) so I went > ahead and gave it a try on ‘core-updates’ (Guix patch attached). Great! > The binutils source gets patched and repacked, but then decompressing it > fails: [..] > Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is > less capable, or could it be a Gash-Utils bug? Currently, we avoid using non-bootstrapped binaries in the bootstrap, that includes 'xz'; earlier in the bootstrap that includes also 'patch'. See also gcc-core-mesboot0: it applies the patch in a manual phase. So I'm not sure if we want to start depending on 'xz' an this stage? Greetings, Janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-03-03 17:54 ` Jan Nieuwenhuizen @ 2021-03-04 16:39 ` Maxim Cournoyer 2021-03-08 14:13 ` Ludovic Courtès 1 sibling, 0 replies; 7+ messages in thread From: Maxim Cournoyer @ 2021-03-04 16:39 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: 45962 Jan Nieuwenhuizen <janneke@gnu.org> writes: > Ludovic Courtès writes: > > Hello! > >> Ludovic Courtès <ludo@gnu.org> skribis: >> >>> On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in >>> commencement.scm lacks ‘--enable-deterministic-archives’. So I checked >>> if this had an effect by running: > >> [...] >> >>> Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’ >>> so we’ll have to patch it. >> >> Sikonas on #bootstrappable provided a patch for that (thanks!) so I went >> ahead and gave it a try on ‘core-updates’ (Guix patch attached). > > Great! > >> The binutils source gets patched and repacked, but then decompressing it >> fails: > > [..] > >> Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is >> less capable, or could it be a Gash-Utils bug? > > Currently, we avoid using non-bootstrapped binaries in the bootstrap, > that includes 'xz'; earlier in the bootstrap that includes also 'patch'. > > See also gcc-core-mesboot0: it applies the patch in a manual phase. So > I'm not sure if we want to start depending on 'xz' an this stage? I see; so what Ludovic got surprised by is the fact that when adding patches or a snippet to an origin it gets repacked as an xz tarball. That's nothing new (it's how it is on the master branch too); but it can indeed be surprising. Maxim ^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-03-03 17:54 ` Jan Nieuwenhuizen 2021-03-04 16:39 ` Maxim Cournoyer @ 2021-03-08 14:13 ` Ludovic Courtès 2021-03-10 5:52 ` Jan Nieuwenhuizen 1 sibling, 1 reply; 7+ messages in thread From: Ludovic Courtès @ 2021-03-08 14:13 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: 45962, Maxim Cournoyer Howdy Janneke! Jan Nieuwenhuizen <janneke@gnu.org> skribis: > Currently, we avoid using non-bootstrapped binaries in the bootstrap, > that includes 'xz'; earlier in the bootstrap that includes also 'patch'. > > See also gcc-core-mesboot0: it applies the patch in a manual phase. So > I'm not sure if we want to start depending on 'xz' an this stage? Probably not, good point! I can think of two possibilities, then: (1) apply the patch in a phase rather than via the ‘patches’ field, and (2) arrange so that ‘patch-and-repack’ does not compress the patched code or compresses it with the bootstrap gzip. My understanding is that #2 may be doable now thanks to Maxim’s recent changes. I’ll take a look! Thanks, Ludo’. ^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-03-08 14:13 ` Ludovic Courtès @ 2021-03-10 5:52 ` Jan Nieuwenhuizen 0 siblings, 0 replies; 7+ messages in thread From: Jan Nieuwenhuizen @ 2021-03-10 5:52 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45962, Maxim Cournoyer Ludovic Courtès writes: Hey Ludo! > Jan Nieuwenhuizen <janneke@gnu.org> skribis: > > I can think of two possibilities, then: (1) apply the patch in a phase > rather than via the ‘patches’ field, and (2) arrange so that > ‘patch-and-repack’ does not compress the patched code or compresses it > with the bootstrap gzip. Oh, that would be nice: we could remove all these clumsy manual patching stages. Also, we may be able to remove XZ from the bootstrap chain, if no XZ-repackaging happens and only build a final version. > My understanding is that #2 may be doable now thanks to Maxim’s recent > changes. I’ll take a look! Great! Greetings, Janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives 2021-03-03 13:48 ` Ludovic Courtès 2021-03-03 17:54 ` Jan Nieuwenhuizen @ 2021-03-04 16:14 ` Maxim Cournoyer 1 sibling, 0 replies; 7+ messages in thread From: Maxim Cournoyer @ 2021-03-04 16:14 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 45962 Hi, Ludovic Courtès <ludo@gnu.org> writes: [...] > ERROR: In procedure scm-error: > ERROR: decompressed-port failure (7) > error: in phase 'unpack': uncaught exception: > srfi-34 #<condition &invoke-error [program: "tar" arguments: ("xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz") exit-status: 1 term-signal: #f stop-signal: #f] 1215400> > phase `unpack' failed after 0.3 seconds > command "tar" "xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz" failed with status 1 > builder for `/gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv' failed with exit code 1 > > Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is > less capable, or could it be a Gash-Utils bug? From my IRC logs: 2020-09-20 22:32:01 apteryx janneke: haha! the xz-bootstrap supports --memlimit with % after all, my mistake was really silly... forgetting a space between the args passed as XZ_DEFAULTS I recall a similar error I had hit when working on adding multi-core compression support to xz, but it ended up being just a mistake on my part; the xz-bootstrap supported the required arguments just fine after all. HTH, Maxim ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-03-10 5:54 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-18 17:29 bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives Ludovic Courtès 2021-03-03 13:48 ` Ludovic Courtès 2021-03-03 17:54 ` Jan Nieuwenhuizen 2021-03-04 16:39 ` Maxim Cournoyer 2021-03-08 14:13 ` Ludovic Courtès 2021-03-10 5:52 ` Jan Nieuwenhuizen 2021-03-04 16:14 ` Maxim Cournoyer
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.