reproducibility of numerical experiments

reproducibility of numerical experiments

[Paul Garlick]

Hi Guix-Science,

I gave a talk to the FEniCS finite element community last week at the
FEniCS 2021 conference.  The title was 'On the reproducibility of
numerical experiments with FEniCS'. 

I introduced GNU Guix, described the reproducible build process and
gave an example of a user profile defined by a manifest and channel
specification.

The conference was online this year.

The slides can be found at:

 https://mscroggs.github.io/fenics2021/talks/garlick.html

Best regards,

Paul.

Re: reproducibility of numerical experiments

[Ludovic Courtès]

Hi Paul,

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

> I introduced GNU Guix, described the reproducible build process and
> gave an example of a user profile defined by a manifest and channel
> specification.
>
> The conference was online this year.
>
> The slides can be found at:
>
>  https://mscroggs.github.io/fenics2021/talks/garlick.html

Nice!  What kind of feedback did you get?  Is the FenICS community
generally aware of reproducible deployment issues?

It’s great to reach out to the numerical simulation community.
Reproducible deployment and numerical simulation are two links in the
long chain of reproducible science that we have to connect.

Cheers,
Ludo’.

Re: reproducibility of numerical experiments

[Paul Garlick]

Hi Ludo,

On Fri, 2021-04-02 at 17:11 +0200, Ludovic Courtès wrote:

> Nice!  What kind of feedback did you get?  

The feedback was very positive.  Owing to the complexity of the
software dependencies. many people  in the finite element community
would like a more dependable method of managing their installations.

There were two questions on the day:

i) Q: what happens if a repository disappears?  Is the environment
still reproducible?
   A: Yes.  The Software Heritage Project provides a backup of all Guix
packages.  This is automatically used as a fallback, if needed.

ii) Q: Suppose I am an end user who wishes to include some third-party
python package, do I require it to exist as a Guix package to be able
to have the "exact" reproducibility you described?
     A: There is a useful feature that allows a local channel to be
defined.  One can keep extra packages and one’s own solvers in the
local channel.  These are not visible in the main Guix repository but
have all the same features regarding reproducibility.

> Is the FenICS community generally aware of reproducible deployment
> issues?

Yes, reproducible deployment is seen as desirable, though difficult to
achieve with the currently-used tools.  Typically model development
starts on a local workstation and is scaled-up to an HPC system when
needed.  Tools such as singularity are often used.  However, the local
and remote installations may differ and as a consequence applications
that run on the local system may not run on the remote system.

> It’s great to reach out to the numerical simulation community.
> Reproducible deployment and numerical simulation are two links in the
> long chain of reproducible science that we have to connect.

Sure.  My view is that Guix provides the machinery to transform what
can be a stop-start mode of numerical model development to something
that is more sustainable over time.

Best regards,

Paul.

Re: reproducibility of numerical experiments

[Ludovic Courtès]

Hi Paul,

Paul Garlick <pgarlick@tourbillion-technology.com> skribis:

> There were two questions on the day:
>
> i) Q: what happens if a repository disappears?  Is the environment
> still reproducible?
>    A: Yes.  The Software Heritage Project provides a backup of all Guix
> packages.  This is automatically used as a fallback, if needed.
>
> ii) Q: Suppose I am an end user who wishes to include some third-party
> python package, do I require it to exist as a Guix package to be able
> to have the "exact" reproducibility you described?
>      A: There is a useful feature that allows a local channel to be
> defined.  One can keep extra packages and one’s own solvers in the
> local channel.  These are not visible in the main Guix repository but
> have all the same features regarding reproducibility.

OK.

>> Is the FenICS community generally aware of reproducible deployment
>> issues?
>
> Yes, reproducible deployment is seen as desirable, though difficult to
> achieve with the currently-used tools.  Typically model development
> starts on a local workstation and is scaled-up to an HPC system when
> needed.  Tools such as singularity are often used.  However, the local
> and remote installations may differ and as a consequence applications
> that run on the local system may not run on the remote system.
>
>> It’s great to reach out to the numerical simulation community.
>> Reproducible deployment and numerical simulation are two links in the
>> long chain of reproducible science that we have to connect.
>
> Sure.  My view is that Guix provides the machinery to transform what
> can be a stop-start mode of numerical model development to something
> that is more sustainable over time.

Interesting, thank you!

Ludo’.