* Reproducibility of "core" packages in GNU Guix [not found] <87wnfagvnp.fsf@contorta> @ 2022-05-02 13:11 ` Vagrant Cascadian 2022-05-02 13:52 ` zimoun 2022-06-02 3:12 ` Vagrant Cascadian 0 siblings, 2 replies; 4+ messages in thread From: Vagrant Cascadian @ 2022-05-02 13:11 UTC (permalink / raw) To: rb-general; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 5235 bytes --] On 2022-04-27, Vagrant Cascadian wrote: > Lately, I've been trying to get a handle on the status of the really > core packages in Debian ... > I'd also be really curious to hear about the status of similar package > sets in other distros! With my metaphorical guix hoodie[1] on... $ guix describe Generation 73 May 02 2022 05:21:25 (current) guix 9dafaf1 repository URL: /home/vagrant/src/guix branch: master commit: 9dafaf163574edca5cb4eac0f8dc3edbb0ef0a75 $ guix challenge --diff=none $(cat guix-base-set) /gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5 contents differ: no local build for '/gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5' https://ci.guix.gnu.org/nar/zstd/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 19rg55v51wliy9v30sm82f38rxm1lqjpfqs6r63ikb3vklnj0pnw https://bordeaux.guix.gnu.org/nar/lzip/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 14fax6g9sx7qj64z73hrh8ydlbv6kxzhd1hbyqz7v0ra51bprv1k /gnu/store/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3 contents differ: no local build for '/gnu/store/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3' https://ci.guix.gnu.org/nar/lzip/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3: 1sag2bq9kbp5np3fpakyi4xg96kxq5xwbb7ib4hamx2bqh6vscr9 https://bordeaux.guix.gnu.org/nar/lzip/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3: 07ln4fqgvg0ag2d881xhgdw2h3m1lqzs6xlac8p7rz2rgx0wx1yr /gnu/store/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23 contents differ: no local build for '/gnu/store/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23' https://ci.guix.gnu.org/nar/lzip/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23: 03a180af1my7lmsnig01qhrirxa2fp7j052jw9kv5ff4i6ya7fh4 https://bordeaux.guix.gnu.org/nar/lzip/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23: 1j24gc6ysa9d3z4hq6lsxvdik94ddb7nj93krv7cs5lmbmjwmqw7 /gnu/store/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0 contents differ: no local build for '/gnu/store/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0' https://ci.guix.gnu.org/nar/lzip/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0: 0p7lhfxcx7bfjfwlyrp6h5j9fcyzswyj2wkbnhcd3fgxm5swdi6c https://bordeaux.guix.gnu.org/nar/lzip/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0: 0yfpcsmvbnzw0vpjrjwwrjih4ss3yvk7cy4k6ibdpsn7dcx9kw2c /gnu/store/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8 contents differ: no local build for '/gnu/store/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8' https://ci.guix.gnu.org/nar/lzip/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8: 0vppx6fk1a7gvk9ccz9ma992w1h5bhfk535acddrnkhyrk92z5ln https://bordeaux.guix.gnu.org/nar/lzip/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8: 05w5i5zq1k1avqx2gqxnqynn5lmdizis9babk34dkmnazb3h77kb 47 store items were analyzed: - 42 (89.4%) were identical - 5 (10.6%) differed - 0 (0.0%) were inconclusive I love that Guix really has batteries included when it comes to reproducible builds verification! :) At first, I thought I would have to build all this stuff locally, but then I realized guix actually has two independent build farms, so guix challenge can compare the results between them! For more data points, one could build them all locally! The fact that the guix and guile packages do not build reproducibly is a little disappointing as they're both so central to guix itself; I suspect parallelism triggers those reproducibility issues(from experience with Debian), though that may just reveal other issue in guile itself. The linux-libre package *ought* to be reproducible; I hope it is something easy to fix there... $ guix challenge --diff=diffoscope linux-libre /gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5 contents differ: no local build for '/gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5' https://ci.guix.gnu.org/nar/zstd/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 19rg55v51wliy9v30sm82f38rxm1lqjpfqs6r63ikb3vklnj0pnw https://bordeaux.guix.gnu.org/nar/lzip/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 14fax6g9sx7qj64z73hrh8ydlbv6kxzhd1hbyqz7v0ra51bprv1k ... 0% ETA: 4 days, 2:03:47 Ok... well, I guess I won't wait for the results... A better "core" package set for GNU Guix could surely be created. I came up with this list of packages by taking the essential, required and build-essential package sets from Debian, tweaking the package names appropriately, dropping debian-specific stuff, and adding guile and guix to create "guix-base-set": acl attr audit bash binutils bzip2 coreutils diffutils e2fsprogs elogind findutils gawk gcc glibc gmp grep guile guix gzip isl keyutils libcap libcap-ng libnsl libselinux libsigsegv libtirpc libxcrypt linux-pam linux-libre mpfr ncurses openssl patch pcre pcre2 perl readline rpcsvc-proto sed shadow tar tzdata util-linux xz zlib zstd > I would also like to see if there is anything in Debian or other > distros that still needs to be pushed upstream, so we can all benefit! Will dig into some of these issues and see how Debian and Guix are building them to see if there are any patches to share and push upstream. [1] Actually wearing my Aspiration Tech hoodie at the moment, but the Guix hoodie is around here somewhere... live well, vagrant [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Reproducibility of "core" packages in GNU Guix 2022-05-02 13:11 ` Reproducibility of "core" packages in GNU Guix Vagrant Cascadian @ 2022-05-02 13:52 ` zimoun 2022-05-02 14:29 ` Vagrant Cascadian 2022-06-02 3:12 ` Vagrant Cascadian 1 sibling, 1 reply; 4+ messages in thread From: zimoun @ 2022-05-02 13:52 UTC (permalink / raw) To: Vagrant Cascadian, rb-general; +Cc: guix-devel Hi Vagrant, Cool to see these reports. On Mon, 02 May 2022 at 06:11, Vagrant Cascadian <vagrant@reproducible-builds.org> wrote: > $ guix challenge --diff=none $(cat guix-base-set) > > /gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5 contents differ: > no local build for '/gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5' > https://ci.guix.gnu.org/nar/zstd/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 19rg55v51wliy9v30sm82f38rxm1lqjpfqs6r63ikb3vklnj0pnw > https://bordeaux.guix.gnu.org/nar/lzip/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5: 14fax6g9sx7qj64z73hrh8ydlbv6kxzhd1hbyqz7v0ra51bprv1k > /gnu/store/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3 contents differ: > no local build for '/gnu/store/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3' > https://ci.guix.gnu.org/nar/lzip/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3: 1sag2bq9kbp5np3fpakyi4xg96kxq5xwbb7ib4hamx2bqh6vscr9 > https://bordeaux.guix.gnu.org/nar/lzip/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3: 07ln4fqgvg0ag2d881xhgdw2h3m1lqzs6xlac8p7rz2rgx0wx1yr > /gnu/store/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23 contents differ: > no local build for '/gnu/store/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23' > https://ci.guix.gnu.org/nar/lzip/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23: 03a180af1my7lmsnig01qhrirxa2fp7j052jw9kv5ff4i6ya7fh4 > https://bordeaux.guix.gnu.org/nar/lzip/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23: 1j24gc6ysa9d3z4hq6lsxvdik94ddb7nj93krv7cs5lmbmjwmqw7 > /gnu/store/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0 contents differ: > no local build for '/gnu/store/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0' > https://ci.guix.gnu.org/nar/lzip/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0: 0p7lhfxcx7bfjfwlyrp6h5j9fcyzswyj2wkbnhcd3fgxm5swdi6c > https://bordeaux.guix.gnu.org/nar/lzip/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0: 0yfpcsmvbnzw0vpjrjwwrjih4ss3yvk7cy4k6ibdpsn7dcx9kw2c > /gnu/store/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8 contents differ: > no local build for '/gnu/store/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8' > https://ci.guix.gnu.org/nar/lzip/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8: 0vppx6fk1a7gvk9ccz9ma992w1h5bhfk535acddrnkhyrk92z5ln > https://bordeaux.guix.gnu.org/nar/lzip/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8: 05w5i5zq1k1avqx2gqxnqynn5lmdizis9babk34dkmnazb3h77kb > > 47 store items were analyzed: > - 42 (89.4%) were identical > - 5 (10.6%) differed > - 0 (0.0%) were inconclusive [...] > The fact that the guix and guile packages do not build reproducibly is a > little disappointing as they're both so central to guix itself; I > suspect parallelism triggers those reproducibility issues(from > experience with Debian), though that may just reveal other issue in > guile itself. About Guix, probably bug#44835 [1] for one, I guess. And note this old Guile bug#20272 [2] for two, which implies unreproducible Guix. 1: <http://issues.guix.gnu.org/issue/44835> 2: <https://issues.guix.gnu.org/issue/20272> Cheers, simon ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Reproducibility of "core" packages in GNU Guix 2022-05-02 13:52 ` zimoun @ 2022-05-02 14:29 ` Vagrant Cascadian 0 siblings, 0 replies; 4+ messages in thread From: Vagrant Cascadian @ 2022-05-02 14:29 UTC (permalink / raw) To: zimoun, rb-general; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1047 bytes --] On 2022-05-02, zimoun wrote: > On Mon, 02 May 2022 at 06:11, Vagrant Cascadian <vagrant@reproducible-builds.org> wrote: >> $ guix challenge --diff=none $(cat guix-base-set) ... >> The fact that the guix and guile packages do not build reproducibly is a >> little disappointing as they're both so central to guix itself; I >> suspect parallelism triggers those reproducibility issues(from >> experience with Debian), though that may just reveal other issue in >> guile itself. > > About Guix, probably bug#44835 [1] for one, I guess. And note this old > Guile bug#20272 [2] for two, which implies unreproducible Guix. > > 1: <http://issues.guix.gnu.org/issue/44835> This one is regarding build paths, and since guix normalizes the build path in the build environment, it should not affect builds in guix... > 2: <https://issues.guix.gnu.org/issue/20272> But this one definitely touches on the parallelism issue! Thanks for the links! I remembered commenting on them... just didn't find the links with a quick search... live well, vagrant [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Reproducibility of "core" packages in GNU Guix 2022-05-02 13:11 ` Reproducibility of "core" packages in GNU Guix Vagrant Cascadian 2022-05-02 13:52 ` zimoun @ 2022-06-02 3:12 ` Vagrant Cascadian 1 sibling, 0 replies; 4+ messages in thread From: Vagrant Cascadian @ 2022-06-02 3:12 UTC (permalink / raw) To: rb-general; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1645 bytes --] On 2022-05-02, Vagrant Cascadian wrote: > $ guix challenge --diff=none $(cat guix-base-set) > > /gnu/store/8gmqvwf0ccqfyimficcnhxvrykwx6y8g-linux-libre-5.17.5 contents differ: Proving more difficult than I'd hoped for, smallish diffs in the .ko files and in the bzImage and System.map, but nothing obvious leaping out at me. The corresponding files are reproducible in Debian bookworm... Working on this lead me to notice a bug in diffoscope at least: https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/305 > /gnu/store/7qz2jlghm4gc87jww5j24c5mcip0whzy-keyutils-1.6.3 contents differ: Patch: https://issues.guix.gnu.org/55758 There was already a patch in debian to set the date using an environment variable. Might be worth working up a patch to support SOURCE_DATE_EPOCH and push it upstream... or nudging upstream to drop the timestamp entirely. :) > /gnu/store/ajw8nnrnd6hr183skwqdgc8c7mazg97h-isl-0.23 contents differ: Patch: https://issues.guix.gnu.org/55757 Disabling parallel building in guix fixes it for me consistently, although Debian's "isl" package is reproducible but... builds with parallelism. (well, not reproducible on i386, but who's really counting?) What about other distros? Do you do anything to make "isl" reproducible? > /gnu/store/45b6181w68a3lprx9m6riwgyinw3y145-guix-1.3.0-25.c1719a0 contents differ: > /gnu/store/1jgcbdzx2ss6xv59w55g3kr3x4935dfb-guile-3.0.8 contents differ: Both of these were not *just* due to parallelism as I'd hoped... inscrutible guile... So, 2 out of the 5 remaining packages have plausible fixes (out of 47 total)... not too bad! live well, vagrant [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 227 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-06-02 3:13 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <87wnfagvnp.fsf@contorta> 2022-05-02 13:11 ` Reproducibility of "core" packages in GNU Guix Vagrant Cascadian 2022-05-02 13:52 ` zimoun 2022-05-02 14:29 ` Vagrant Cascadian 2022-06-02 3:12 ` Vagrant Cascadian
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).