Guix-HPC activity report

Guix-HPC activity report

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

Hello Guix!

We have just published an activity report of Guix-HPC for the past 18
months:

  https://guix-hpc.bordeaux.inria.fr/blog/2019/02/guix-hpc-activity-report-2018/

As you may know, some of us are involved at work with Guix in a
scientific context.  Guix-HPC is about bringing reproducible software
deployment to high-performance computing (HPC) and scientific workflows.

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Guix-HPC activity report

[Laura Lazzati]

Congratulations!

Guess who is interested in contributing to this after Outreachy
finishes if you don't mind? :)

Regards!
Laura

Re: Guix-HPC activity report

[zimoun]

Hi,

Nice blog post !


And I discovered the (gnu packages simulation) module. :-)

How the split between maths.scm and simulation.scm is done?
I mean, e.g., why deal.ii is in maths.scm and not in simulation.scm?


Thank you!

All the best,
simon

Re: Guix-HPC activity report

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

> How the split between maths.scm and simulation.scm is done?
> I mean, e.g., why deal.ii is in maths.scm and not in simulation.scm?

I think the only reason is that maths.scm is older than simulation.scm.
We’ve got the same situation for bioinformatics.scm, bioconductor.scm,
cran.scm, and statistics.scm.

-- 
Ricardo

Re: Guix-HPC activity report

[Pjotr Prins]

Hi Ludo,

What is required to get institutes on the list? With UTHSC, we have
the intention of running Guix on HPC. Provisioning Singularity
containers may be key. Not quite there yet.

Obviously we are running Guix on GeneNetwork.

Pj.

On Tue, Feb 12, 2019 at 02:51:27PM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> We have just published an activity report of Guix-HPC for the past 18
> months:
> 
>   https://guix-hpc.bordeaux.inria.fr/blog/2019/02/guix-hpc-activity-report-2018/
> 
> As you may know, some of us are involved at work with Guix in a
> scientific context.  Guix-HPC is about bringing reproducible software
> deployment to high-performance computing (HPC) and scientific workflows.
> 
> Ludo’.

Re: Guix-HPC activity report

[Marco van Hulten]

Ludovic—

On 12 Feb 14:51 Ludovic Courtès wrote:
> We have just published an activity report of Guix-HPC for the past 18
> months:
> 
>   https://guix-hpc.bordeaux.inria.fr/blog/2019/02/guix-hpc-activity-report-2018/
> 
> As you may know, some of us are involved at work with Guix in a
> scientific context.  Guix-HPC is about bringing reproducible software
> deployment to high-performance computing (HPC) and scientific workflows.

This is awesome!  I read the report with great attention.


Some typos (from the pdf I downloaded two days ago):

+ footnote 8:
    https://softwareheritage.org → https://www.softwareheritage.org

+ page 18:
    becomming → becoming

+ page 17 (bottom):
    worflows → workflows.

—Marco

Re: Guix-HPC activity report

[Ludovic Courtès]

Hello,

Marco van Hulten <marco@hulten.org> skribis:

> On 12 Feb 14:51 Ludovic Courtès wrote:
>> We have just published an activity report of Guix-HPC for the past 18
>> months:
>> 
>>   https://guix-hpc.bordeaux.inria.fr/blog/2019/02/guix-hpc-activity-report-2018/
>> 
>> As you may know, some of us are involved at work with Guix in a
>> scientific context.  Guix-HPC is about bringing reproducible software
>> deployment to high-performance computing (HPC) and scientific workflows.
>
> This is awesome!  I read the report with great attention.

Thank you.

> Some typos (from the pdf I downloaded two days ago):

Oops, fixed!

Ludo’.

Re: Guix-HPC activity report

[zimoun]

Hi,

I have quickly given a look at some pointer. For example, some
presentations at the EasyBuild meetup.
There are slides with not so nice figures, e.g., p45 of [1] about
FFTW. I remember similar number in other easybuild presentation.

In the Guix FFTW package definition, there is a comment about AVX.

Well, is it still accurate ?
Is Guix binary still slower than other ?

[1] https://users.ugent.be/~kehoste/eum18/eum18_easybuild_past_present_future_20180130.pdf


Thank you in advance for any pointer.


All the best,
simon

Re: Guix-HPC activity report

[Ludovic Courtès]

Hi Simon,

zimoun <zimon.toutoune@gmail.com> skribis:

> In the Guix FFTW package definition, there is a comment about AVX.
>
> Well, is it still accurate ?
> Is Guix binary still slower than other ?

The comment links to
<http://fftw.org/fftw3_doc/Installation-on-Unix.html>, which reads:

  Enable various SIMD instruction sets. […] FFTW will try to detect at
  runtime whether the CPU supports these extensions.  That is, you can
  compile with --enable-avx and the code will still run on a CPU without
  AVX support.

So I believe the numbers one gets with the ‘fftw’ package in Guix are
the best one can get because FFTW does the right thing of using the
right version of its hot functions at run time.  (That said, you can
redo the benchs, and if this is not the case, it’s a bug! :-))

See also this article, which addresses this topic:

  https://guix-hpc.bordeaux.inria.fr/blog/2018/01/pre-built-binaries-vs-performance/

Thanks,
Ludo’.

Re: Guix-HPC activity report

[Ludovic Courtès]

Hi Pjotr,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> What is required to get institutes on the list? With UTHSC, we have
> the intention of running Guix on HPC. Provisioning Singularity
> containers may be key. Not quite there yet.

The logos there are those of the institutes that initiated the effort,
and those with the most involvement (one person more or less full-time
at each of these.)

But as you know this is quite informal and the goal is to let others
join, as Cray and Tourbillion did.  So of course, we’ll be happy to add
UTHSC or any other organization that supports the effort there!  Perhaps
we should actually drop the logo footer and/or move them elsewhere;
ideas welcome.  :-)

Thanks,
Ludo’.

Re: Guix-HPC activity report

[zimoun]

Hi Ludo,

Thank you for your precision.

On Fri, 15 Feb 2019 at 18:09, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > Well, is it still accurate ?
> > Is Guix binary still slower than other ?
>
> The comment links to
> <http://fftw.org/fftw3_doc/Installation-on-Unix.html>, which reads:
>
>   Enable various SIMD instruction sets. […] FFTW will try to detect at
>   runtime whether the CPU supports these extensions.  That is, you can
>   compile with --enable-avx and the code will still run on a CPU without
>   AVX support.

If I understand well, the claims p.45 of the EasyBuild presentation
[1] should not true any more.
Because there is a factor 2+ with their benchmarks.

[1] https://users.ugent.be/~kehoste/eum18/eum18_easybuild_past_present_future_20180130.pdf


> So I believe the numbers one gets with the ‘fftw’ package in Guix are
> the best one can get because FFTW does the right thing of using the
> right version of its hot functions at run time.  (That said, you can
> redo the benchs, and if this is not the case, it’s a bug! :-))

A benchmark from "our side" should be informative. :-)
But I am not sure that I would like to dive in EasyBuild stuff... ;-)


Thank you again !
All the best,
simon