many similar builds

many similar builds

[Marco van Hulten]

$ md5sum /gnu/store/*-xeyes-*/bin/xeyes
e17c8aab7cdeedb763601288b6eb0c43  /gnu/store/3syyc8yvyymnwsada0kgm1jmc83232c8-xeyes-1.1.2/bin/xeyes
f44bf885477a110f7d0ee7b3524b978a  /gnu/store/j0ycra947a0y3d3c9afms7sjlykis629-xeyes-1.1.2/bin/xeyes
$ ls -l /gnu/store/*-atril-*/bin/atril
-r-xr-xr-x 2 root root 1226 Jan  1
1970 /gnu/store/01j2hg31cv41a0fb19h1wvmn7dzxlv8s-atril-1.18.1/bin/atril
-r-xr-xr-x 2 root root 1226 Jan  1
1970 /gnu/store/1qsm3qmiqc84p8yakbxvyccjbg64ak7z-atril-1.18.1/bin/atril
-r-xr-xr-x 2 root root 1226 Jan  1
1970 /gnu/store/30syxb0vqbcc0zap47mhxlswmhq0w2vw-atril-1.18.1/bin/atril
-r-xr-xr-x 2 root root 1226 Jan  1
1970 /gnu/store/3b95dhdrkyhbw3bijzy9phd34bq32nnq-atril-1.18.1/bin/atril
...

and many more, and 14 different «same-version» installations for
brasero.  Does this mean that it is just very difficult to get
reproducible builds?

—Marco

-- 
☎	+31 30 711 0341 (VoIP)
💬 	hulten@jabber.fsfe.org (using OMEMO encryption)
🕸	http://marcovan.hulten.org/

Freedom to read — https://www.nature.com/articles/d41586-018-06178-7
Finally, punishment of researchers not open accessing is long overdue!

Re: many similar builds

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

The hash is computed over everything: the package itself but also all its dependencies.

When a dependency is updated, the package gets rebuilt to a new store item.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: many similar builds

[Tobias Geerinckx-Rice]

Marco,

Pierre Neidhardt wrote:
> The hash is computed over everything: the package itself but 
> also all its dependencies.

Pierre's right, but note that 'the package itself' here means the 
package 'expression' (minus mere metadata like home-page ond 
descriptions), not the output produced by building 
it. Reproducibility is important but cannot affect the hash.

Kind regards,

T G-R

Re: many similar builds

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 2553 bytes --]

On Tue, 9 Oct 2018 20:34:28 +0200
Marco van Hulten <marco@hulten.org> wrote:

> $ md5sum /gnu/store/*-xeyes-*/bin/xeyes
> e17c8aab7cdeedb763601288b6eb0c43  /gnu/store/3syyc8yvyymnwsada0kgm1jmc83232c8-xeyes-1.1.2/bin/xeyes
> f44bf885477a110f7d0ee7b3524b978a  /gnu/store/j0ycra947a0y3d3c9afms7sjlykis629-xeyes-1.1.2/bin/xeyes
> $ ls -l /gnu/store/*-atril-*/bin/atril
> -r-xr-xr-x 2 root root 1226 Jan  1
> 1970 /gnu/store/01j2hg31cv41a0fb19h1wvmn7dzxlv8s-atril-1.18.1/bin/atril
> -r-xr-xr-x 2 root root 1226 Jan  1
> 1970 /gnu/store/1qsm3qmiqc84p8yakbxvyccjbg64ak7z-atril-1.18.1/bin/atril
> -r-xr-xr-x 2 root root 1226 Jan  1
> 1970 /gnu/store/30syxb0vqbcc0zap47mhxlswmhq0w2vw-atril-1.18.1/bin/atril
> -r-xr-xr-x 2 root root 1226 Jan  1
> 1970 /gnu/store/3b95dhdrkyhbw3bijzy9phd34bq32nnq-atril-1.18.1/bin/atril
> ...
> 
> and many more, and 14 different «same-version» installations for
> brasero.  Does this mean that it is just very difficult to get
> reproducible builds?


To add to the already given answers: The store is deduplicated (if that
feature is not explicitly disabled): If two files share the same
content, they are stored only once on hard disk and a hard link points
to that same inode. See here:

I have 21 copies of icedtea-3.7.0 in my store:

$ ls /gnu/store/*icedtea-3.7.0/bin/java -d | wc
     21      21    1407

But if I look at the java command in there, I see it has only two different
ones, with inode count 17 and 24 (first column after permissions):

ls -l /gnu/store/*icedtea-3.7.0/bin/java -d


-r-xr-xr-x 17 root root  8344 Jan  1  1970 /gnu/store/5737dyajx477kl9h0bi393ax0qb4vixc-icedtea-3.7.0/bin/java
-r-xr-xr-x 24 root root  8328 Jan  1  1970 /gnu/store/9lg4136fwprf0fm5jjwd818zqsn2bqim-icedtea-3.7.0/bin/java
[...]

(Of cause 17 + 24 > 21, so there must be some bitwise-identical java
around in some other package)

And `du` tells me:

$ du -shc /gnu/store/*icedtea-3.7.0
114M    /gnu/store/0s9z0prad24i9l6bjvzjf6agwgfmh1l8-icedtea-3.7.0
15M     /gnu/store/2k3ixx4hfkb2nf8d9530phympalxb7bs-icedtea-3.7.0
[..]
186M    total

# With '-l' option to ignore hard links:
$ du -shcl /gnu/store/*icedtea-3.7.0
114M    /gnu/store/0s9z0prad24i9l6bjvzjf6agwgfmh1l8-icedtea-3.7.0
[..]
2.4G    total

Wow, that's a space-saver! You don't have to worry to have all those
(nearly similar/totally similar) store entries around. I prefer to keep
them if I can, that saves me CPU-time whenever I build older commits
out of the Guix git repository.

Björn







[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]