Re: Rust reprodubility -- .rmeta and shadow-rs

["Ludovic Courtès" <ludo@gnu.org>]

Hello!

Maxime Devos <maximedevos@telenet.be> skribis:

> There was some mail about irreproducibility in Rust, but I couldn't
> find it anymore.  Anyway, I found a potential cause: rust-shadow-rs
> embeds timestamps (even though it nominally respects
> SOURCE_DATE_EPOCH???) and the ordering of definitions it generates is
> based on a hash map (and hence, irreproducible).

I found these:

  https://issues.guix.gnu.org/50015
  https://issues.guix.gnu.org/55928

> The crate id is based on a hash over the source code, so this
> irreproducibility can cause build failures if substitutes are used.
>
> By removing the time stamp and sorting the definitions, 'nushell'
> successfully built on ci.guix.gnu.org whereas it previously failed to
> build on ci.guix.gnu.org but built successfully locally (with
> antioxidant), and IIRC the (antioxidated) 'rust-nu-command' is now
> reproducible:
>
> Anyway, here the patch I used:
>
>     ("rust-shadow-rs"
>      ,#~((add-after 'unpack 'fixup-source-date-epoch
> 	   (lambda _
> 	     ;; TODO: it nominally supports SOURCE_DATE_EPOCH, yet something things go wrong,
> 	     ;; as the shadow.rs still contains the unnormalised time stamp ...
> 	     ;; For now, do a work-around.
> 	     (substitute* '("src/lib.rs" "src/env.rs")
> 	       (("BuildTime::Local\\(Local::now\\(\\)\\)\\.human_format\\(\\)")
> 		(object->string "[timestamp expunged for reproducibility]"))
> 	       (("time\\.human_format\\(\\)")
> 		"\"[timestamp expunged for reproducibility]\".to_string()")
> 	       (("time\\.to_rfc3339_opts\\(SecondsFormat::Secs, true)")
> 		"\"[timestamp expunged for reproducibility]\".to_string()")
> 	       (("time\\.to_rfc2822\\(\\)")
> 		"\"[timestamp expunged for reproducibility]\".to_string()"))))
> 	 (add-after 'unpack 'more-reproducibility ;; by default, it uses a hashmap, leading to an irreproducible ordering in shadow.rs and hence an irreproducible .rmeta (TODO: upstream?)
> 	   (lambda _
> 	     (substitute* "src/lib.rs" ; sort
> 	       (("\\(k, v\\) in self\\.map\\.clone\\(\\)")
> 		"(k, v) in std::collections::BTreeMap::from_iter(self.map.clone().iter())")
> 	       (("self\\.write_const\\(k, v\\)") "self.write_const(k, v.clone())")
> 	       (("self\\.map\\.keys\\(\\)") "std::collections::BTreeSet::from_iter(self.map.keys())"))))))
>
> Maybe that was the cause?

You mean this issue you identified could have been the cause of
reproducibility issues found in other Rust packages?

Anyway, it looks like the snippet above should be applied to
‘rust-shadow-rs’ in current ‘master’, no?

Thanks,
Ludo’.