* bug#50015: Rust packages are not reproducible
@ 2021-08-11 21:15 Ludovic Courtès
2021-08-12 6:44 ` Efraim Flashner
0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2021-08-11 21:15 UTC (permalink / raw)
To: 50015
Hello!
Rust packages, which are essentially empty, are not bit-reproducible:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix challenge rust-rocket-codegen --substitute-urls='https://ci.guix.gnu.org https://bordeaux.guix.gnu.org'
/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7 contents differ:
no local build for '/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7'
https://ci.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 0j6zf2afvc49jnp7i6z7yvbxm0bmw8yc65hz3lncgvw5lc6z1bc1
https://bordeaux.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 015kb637b56mqcsg3f6x1qggm2bybiszji2069gb913wxbj6rs7w
differing file:
/share/cargo/registry/rocket_codegen-0.4.7.crate
1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive
ludo@ribbon ~/src/guix$ ./pre-inst-env guix challenge rust-rocket-codegen
/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7 contents differ:
no local build for '/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7'
https://ci.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 0j6zf2afvc49jnp7i6z7yvbxm0bmw8yc65hz3lncgvw5lc6z1bc1
https://bordeaux.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 015kb637b56mqcsg3f6x1qggm2bybiszji2069gb913wxbj6rs7w
differing file:
/share/cargo/registry/rocket_codegen-0.4.7.crate
1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive
$ git log |head -1
commit 973842acbc2d0dc1ab41738a534d4abda6d9efa7
--8<---------------cut here---------------end--------------->8---
The diffoscope output suggests it’s about timestamps on one file in the
.crate archive:
--8<---------------cut here---------------start------------->8---
│ │ │ │ --- /tmp/guix-directory.ii5wmv/share/cargo/registry/rocket_codegen-0.4.7.crate
│ │ │ ├── +++ /tmp/guix-directory.uTTKSw/share/cargo/registry/rocket_codegen-0.4.7.crate
│ │ │ │ ├── rocket_codegen-0.4.7.crate-content
│ │ │ │ │ ├── file list
│ │ │ │ │ │ @@ -1,67 +1,67 @@
│ │ │ │ │ │ --rw-r--r-- 0 0 0 1293 2021-07-27 15:22:18.000000 rocket_codegen-0.4.7/Cargo.toml
[…]
│ │ │ │ │ │ +-rw-r--r-- 0 0 0 1293 2021-07-27 22:01:49.000000 rocket_codegen-0.4.7/Cargo.toml
--8<---------------cut here---------------end--------------->8---
Does that ring a bell?
Thanks,
Ludo’.
PS: I noticed this via
<http://data.guix.gnu.org/revision/973842acbc2d0dc1ab41738a534d4abda6d9efa7/package-reproducibility>
with help from Chris. Fixing this could noticeably improve our
stats. :-)
^ permalink raw reply [flat|nested] 6+ messages in thread* bug#50015: Rust packages are not reproducible
2021-08-11 21:15 bug#50015: Rust packages are not reproducible Ludovic Courtès
@ 2021-08-12 6:44 ` Efraim Flashner
2021-08-12 8:06 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Efraim Flashner @ 2021-08-12 6:44 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 50015
[-- Attachment #1: Type: text/plain, Size: 3604 bytes --]
On Wed, Aug 11, 2021 at 11:15:09PM +0200, Ludovic Courtès wrote:
> Hello!
>
> Rust packages, which are essentially empty, are not bit-reproducible:
>
> --8<---------------cut here---------------start------------->8---
> $ ./pre-inst-env guix challenge rust-rocket-codegen --substitute-urls='https://ci.guix.gnu.org https://bordeaux.guix.gnu.org'
> /gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7 contents differ:
> no local build for '/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7'
> https://ci.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 0j6zf2afvc49jnp7i6z7yvbxm0bmw8yc65hz3lncgvw5lc6z1bc1
> https://bordeaux.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 015kb637b56mqcsg3f6x1qggm2bybiszji2069gb913wxbj6rs7w
> differing file:
> /share/cargo/registry/rocket_codegen-0.4.7.crate
>
> 1 store items were analyzed:
> - 0 (0.0%) were identical
> - 1 (100.0%) differed
> - 0 (0.0%) were inconclusive
> ludo@ribbon ~/src/guix$ ./pre-inst-env guix challenge rust-rocket-codegen
> /gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7 contents differ:
> no local build for '/gnu/store/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7'
> https://ci.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 0j6zf2afvc49jnp7i6z7yvbxm0bmw8yc65hz3lncgvw5lc6z1bc1
> https://bordeaux.guix.gnu.org/nar/lzip/09hlac18bwl1kcnhdig7z1v2j8ly1czw-rust-rocket-codegen-0.4.7: 015kb637b56mqcsg3f6x1qggm2bybiszji2069gb913wxbj6rs7w
> differing file:
> /share/cargo/registry/rocket_codegen-0.4.7.crate
>
> 1 store items were analyzed:
> - 0 (0.0%) were identical
> - 1 (100.0%) differed
> - 0 (0.0%) were inconclusive
> $ git log |head -1
> commit 973842acbc2d0dc1ab41738a534d4abda6d9efa7
> --8<---------------cut here---------------end--------------->8---
>
> The diffoscope output suggests it’s about timestamps on one file in the
> .crate archive:
>
> --8<---------------cut here---------------start------------->8---
> │ │ │ │ --- /tmp/guix-directory.ii5wmv/share/cargo/registry/rocket_codegen-0.4.7.crate
> │ │ │ ├── +++ /tmp/guix-directory.uTTKSw/share/cargo/registry/rocket_codegen-0.4.7.crate
> │ │ │ │ ├── rocket_codegen-0.4.7.crate-content
> │ │ │ │ │ ├── file list
> │ │ │ │ │ │ @@ -1,67 +1,67 @@
> │ │ │ │ │ │ --rw-r--r-- 0 0 0 1293 2021-07-27 15:22:18.000000 rocket_codegen-0.4.7/Cargo.toml
> […]
> │ │ │ │ │ │ +-rw-r--r-- 0 0 0 1293 2021-07-27 22:01:49.000000 rocket_codegen-0.4.7/Cargo.toml
> --8<---------------cut here---------------end--------------->8---
>
> Does that ring a bell?
>
> Thanks,
> Ludo’.
>
> PS: I noticed this via
> <http://data.guix.gnu.org/revision/973842acbc2d0dc1ab41738a534d4abda6d9efa7/package-reproducibility>
> with help from Chris. Fixing this could noticeably improve our
> stats. :-)
>
I tried patching this a couple of ways, but it looks like the best
option is going to be a 'patch-and-repack phase after 'install. the
.crate file is really a gzip tarball, and I suspect that each time we
run 'cargo <something>' the timestamp gets updated.
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#50015: Rust packages are not reproducible
2021-08-12 6:44 ` Efraim Flashner
@ 2021-08-12 8:06 ` Ludovic Courtès
2023-10-04 3:30 ` Maxim Cournoyer
0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2021-08-12 8:06 UTC (permalink / raw)
To: Efraim Flashner; +Cc: 50015
Hello!
Efraim Flashner <efraim@flashner.co.il> skribis:
> I tried patching this a couple of ways, but it looks like the best
> option is going to be a 'patch-and-repack phase after 'install. the
> .crate file is really a gzip tarball, and I suspect that each time we
> run 'cargo <something>' the timestamp gets updated.
So that ‘Cargo.toml’ file is not something taken from the build tree?
In that case we could reset the timestamp before the tarball is
created. But otherwise yeah, patch’n’repack.
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#50015: Rust packages are not reproducible
2021-08-12 8:06 ` Ludovic Courtès
@ 2023-10-04 3:30 ` Maxim Cournoyer
2023-10-04 12:06 ` Efraim Flashner
0 siblings, 1 reply; 6+ messages in thread
From: Maxim Cournoyer @ 2023-10-04 3:30 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 50015, Efraim Flashner
Hello,
Ludovic Courtès <ludo@gnu.org> writes:
> Hello!
>
> Efraim Flashner <efraim@flashner.co.il> skribis:
>
>> I tried patching this a couple of ways, but it looks like the best
>> option is going to be a 'patch-and-repack phase after 'install. the
>> .crate file is really a gzip tarball, and I suspect that each time we
>> run 'cargo <something>' the timestamp gets updated.
>
> So that ‘Cargo.toml’ file is not something taken from the build tree?
> In that case we could reset the timestamp before the tarball is
> created. But otherwise yeah, patch’n’repack.
A better solution would be to have cargo honor SOURCE_DATE_EPOCH,
perhaps? They'd probably accept such an improvement upstream.
--
Thanks,
Maxim
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#50015: Rust packages are not reproducible
2023-10-04 3:30 ` Maxim Cournoyer
@ 2023-10-04 12:06 ` Efraim Flashner
2023-10-04 15:05 ` Maxim Cournoyer
0 siblings, 1 reply; 6+ messages in thread
From: Efraim Flashner @ 2023-10-04 12:06 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 50015, Ludovic Courtès
[-- Attachment #1: Type: text/plain, Size: 1767 bytes --]
On Tue, Oct 03, 2023 at 11:30:15PM -0400, Maxim Cournoyer wrote:
> Hello,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
> > Hello!
> >
> > Efraim Flashner <efraim@flashner.co.il> skribis:
> >
> >> I tried patching this a couple of ways, but it looks like the best
> >> option is going to be a 'patch-and-repack phase after 'install. the
> >> .crate file is really a gzip tarball, and I suspect that each time we
> >> run 'cargo <something>' the timestamp gets updated.
> >
> > So that ‘Cargo.toml’ file is not something taken from the build tree?
> > In that case we could reset the timestamp before the tarball is
> > created. But otherwise yeah, patch’n’repack.
>
> A better solution would be to have cargo honor SOURCE_DATE_EPOCH,
> perhaps? They'd probably accept such an improvement upstream.
That'd be an interesting idea, having 'cargo package' set the timestamp
of all the files to SOURCE_DATE_EPOCH. I guess I can look into how
feasible that would be and if they'd be likely to accept a change like
that.
I have a local patch which unpacks, resets the timestamp and repacks the
crate. I'll definitely push it to the rust-team branch before the next
merge.
With it I introduced an issue where the 'package phase would repack all
the crates, not just the current one, and ran into our
underscore-to-dash naming convention causing issues with how I'm reusing
the filename to work on the crate. I'll fix that, probably by only
repacking the current crate instead of all crates in the environment.
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#50015: Rust packages are not reproducible
2023-10-04 12:06 ` Efraim Flashner
@ 2023-10-04 15:05 ` Maxim Cournoyer
0 siblings, 0 replies; 6+ messages in thread
From: Maxim Cournoyer @ 2023-10-04 15:05 UTC (permalink / raw)
To: Efraim Flashner; +Cc: 50015, Ludovic Courtès
Hi Efraim,
Efraim Flashner <efraim@flashner.co.il> writes:
> On Tue, Oct 03, 2023 at 11:30:15PM -0400, Maxim Cournoyer wrote:
>> Hello,
>>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>> > Hello!
>> >
>> > Efraim Flashner <efraim@flashner.co.il> skribis:
>> >
>> >> I tried patching this a couple of ways, but it looks like the best
>> >> option is going to be a 'patch-and-repack phase after 'install. the
>> >> .crate file is really a gzip tarball, and I suspect that each time we
>> >> run 'cargo <something>' the timestamp gets updated.
>> >
>> > So that ‘Cargo.toml’ file is not something taken from the build tree?
>> > In that case we could reset the timestamp before the tarball is
>> > created. But otherwise yeah, patch’n’repack.
>>
>> A better solution would be to have cargo honor SOURCE_DATE_EPOCH,
>> perhaps? They'd probably accept such an improvement upstream.
>
> That'd be an interesting idea, having 'cargo package' set the timestamp
> of all the files to SOURCE_DATE_EPOCH. I guess I can look into how
> feasible that would be and if they'd be likely to accept a change like
> that.
>
> I have a local patch which unpacks, resets the timestamp and repacks the
> crate. I'll definitely push it to the rust-team branch before the next
> merge.
>
> With it I introduced an issue where the 'package phase would repack all
> the crates, not just the current one, and ran into our
> underscore-to-dash naming convention causing issues with how I'm reusing
> the filename to work on the crate. I'll fix that, probably by only
> repacking the current crate instead of all crates in the environment.
From past interactions with Rust developers (via their web-based chat
system), I could get some change merged rather easily. If you are
motivated to fix it cleanly I encourage you to pursue that way!
--
Thanks,
Maxim
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-10-04 15:06 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-11 21:15 bug#50015: Rust packages are not reproducible Ludovic Courtès
2021-08-12 6:44 ` Efraim Flashner
2021-08-12 8:06 ` Ludovic Courtès
2023-10-04 3:30 ` Maxim Cournoyer
2023-10-04 12:06 ` Efraim Flashner
2023-10-04 15:05 ` Maxim Cournoyer
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).