Re: persistent reproducibility ?

[Alex Sassmannshausen <alex.sassmannshausen@gmail.com>]

Hi Simon,

zimoun writes:

> Hi Guixers,
>
>
> Disclaimer: my background is about numerical analysis (PDE, Krylov
> solver, etc.) and I am currently working in some Core Facility about
> biology and bioinformatic (NGS pipelines, flow cytometry clustering,
> etc.). Orthogonal communities but same issues. :-)

Welcome! Sounds very exciting :-)

> The typical research workflow is:
>
> - Alice proposes new method and/or algorithm, publishes a paper and
> illustrates that by the software `foo'. Let the best case: Alice
> provides a Guix "recipe", and all the material is stored in Github
> (let say). This software `foo' depends on both `bar' and `baz', one
> also in Github and the other one included in the Guix package tree.
>
> - It is easy for Bob to check out and experiment. Guix allows him to
> straightforwardly build the bit identical `foo' (all dependencies
> included). Nice!! Repeatability is there for free.
>
> - New features are added to `foo', `bar' and `baz'. All the codes
> evolve, especially the research ones.
>
> - Now, Joe is implementing the Alice's method; science means
> reproducible. And Joe would like to compare his implementation to the
> Alice one provided by `foo'. However, how ? The `foo' "ecosystem" has
> changed with the new features. Therefore, Joe has to navigate in the
> Git tree of the Guix "recipe" of `foo', `bar', `baz' to be able to
> produce the bit-identical `foo' used in the initial paper. I mean, it
> is what I understand to do, and it does not seem reasonable.

There are others on here who may have more in-depth experience of
precisely this issue, but here's my impression of how this would work.

Alice's package recipe would specify the exact version of itself, and
also of `bar` and `baz` that that exactly version required.  As a
result, when software, and also package recipe versions evolve, all Joe
would have to do is locate the historic package recipe of `foo` that Joe
is trying to reproduce.  That package recipe would then point to the
historic versions of `bar` & `baz`, so Joe would not have to worry about
that.

> My question is: does Guix provide any mechanism to build reproducible
> software over the time ?

Guix just parses the recipe. The recipe details the exact versions of
software required.

If the exact recipe of a given time is not available then Guix cannot
help that (though if the recipe is added to Guix then it will be
available in Guix's repository).
If the repository hosting the exact version of the software that the
recipe specifies is no longer available then Guix cannot help.

> Last, `foo' and `bar' are stored in two Github repositories. And they
> should disappear.
> ( I am not talking if it is good or not to use github, right now, it
> just is used by many teams of researchers )
>
> Could we used the Software Heritage initiative to maintain a kind of
> persistency ?
> https://www.softwareheritage.org

There was some conversation about this at some point.  Again others may
know more about this. It certainly seems a nice means of ensuring
software remains available…

> I do not know if my wishes make any sense.

I think they definitely do :-)

HTH,

Alex