From: Bengt Richter <bokr@bokr.com>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: Guix Devel <guix-devel@gnu.org>, zimoun <zimon.toutoune@gmail.com>
Subject: Re: Investigating a reproducibility failure
Date: Tue, 15 Feb 2022 15:10:32 +0100 [thread overview]
Message-ID: <20220215141031.GA13837@LionPure> (raw)
In-Reply-To: <87tuddh06r.fsf@gnu.org>
Hi,
On +2022-02-05 15:12:28 +0100, Ludovic Courtès wrote:
> Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
>
> > There is obviously a trade-off between reproducibility and performance
> > here.
>
I suspect what you really want to reproduce is not verbatim
code, but the abstract computation that it implements,
typically a digitally simulated experiment?
Thus far, "show me the code" is the usual way to ask someone
what they did, and guix makes is possible to answer in great
detail.
But what is really relevant if you are helping a colleague
reproduce e.g. a monte-carlo simulation experiment computing
pi by throwing random darts at a square, to draw a graph
showing convergence of statistically-computed pi on y-axis
vs number of darts thrown on x-axis?
(IIRC pi should be hits within inscribed circle / hits in
1x1 square)
Well, ISTM you can reproduce this experiment in any language
and method that does the abtract job.
The details of Fortran version or Julia/Clang or guile
pedigree only really come into play for forensics looking
for where the abstract was implemented differently.
E.g., if results were different, were the x and y random
numbers displacing the darts within the square really
uniform and independent, and seeded with constants to ensure
bit-for-bit equivalent computations?
How fast the computations happened is not relevant,
though of course nice for getting work done :)
> I tried hard to dispel that belief: you do not have to trade one for the other.
>
> Yes, in some cases scientific software might lack the engineering work
> that allows for portable performance; but in those cases, there’s
> ‘--tune’.
>
> https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/
>
> We should keep repeating that message: reproducibility and performance
> are not antithetic. And I really mean it, otherwise fellow HPC
> practitioners will keep producing unverifiable results on the grounds
> that they cannot possibly compromise on performance!
>
Maybe the above pi computation could be a start on some kind
of abstract model validation test? It's simple, but it pulls
on a lot of simulation tool chains. WDYT?
> Thanks,
> Ludo’.
>
--
Regards,
Bengt Richter
next prev parent reply other threads:[~2022-02-15 14:13 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-02 20:35 Investigating a reproducibility failure zimoun
2022-02-02 23:43 ` zimoun
2022-02-03 9:16 ` Konrad Hinsen
2022-02-03 11:41 ` Ricardo Wurmus
2022-02-03 17:05 ` Konrad Hinsen
2022-02-03 12:07 ` zimoun
2022-02-05 14:12 ` Ludovic Courtès
2022-02-15 14:10 ` Bengt Richter [this message]
2022-02-16 12:03 ` zimoun
2022-02-16 13:04 ` Konrad Hinsen
2022-02-17 11:21 ` zimoun
2022-02-17 16:55 ` Konrad Hinsen
-- strict thread matches above, loose matches on Subject: below --
2022-02-01 14:05 Konrad Hinsen
2022-02-01 14:30 ` Konrad Hinsen
2022-02-02 23:19 ` Ricardo Wurmus
2022-02-02 23:36 ` Ricardo Wurmus
2022-02-05 14:05 ` Ludovic Courtès
2022-02-08 5:57 ` Konrad Hinsen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220215141031.GA13837@LionPure \
--to=bokr@bokr.com \
--cc=guix-devel@gnu.org \
--cc=ludo@gnu.org \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).