Re: Guix on clusters and in HPC

[ludo@gnu.org (Ludovic Courtès)]

Hi Ben,

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> I hope the proposal is/was working out.

Making progress; it’s due for the end of the year.  Until then you’re
welcome to make suggestions.  :-)

> On 03/11/16 23:44, Ludovic Courtès wrote:
>> Hi!
>>
>> Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:
>>
>>> I'm a little late here, but please do all of the things on that list :)
>> :-)
>>
>>> With this suggestion:
>>>
>>>      + for [[https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00005.html][CPU-specific optimizations]]
>>>      + somehow support -mtune=native (and even profile-guided
>>>        optimizations?)
>>>
>>> I'm not sure if you already thought of this, but an important use case is that of pipelines, where we may want to optimise not just the package being built, but instead one (or more) of its dependencies. So your suggestion of this syntax:
>>>
>>>    guix package --tune=haswell -i diamond
>>>
>>> requires some extensions, maybe something like this, where bamm can be used as a pipeline that uses bwa and samtools:
>>>
>>>    guix package -i bamm --tune=haswell bwa samtools
>>>
>>> and to optimise the C in bamm itself too:
>>>
>>>    guix package -i bamm --tune=haswell bwa samtools bamm
>> So you’re saying that --tune should apply recursively, right?
> Sort of. The difficulty with applying it fully recursively is that it
> might greatly increase the maintenance burden of using --tune for
> little performance improvement, since all inputs would have to be
> tuned, not just those that substantively affect performance. This is
> all conjecture though, I'm not sure how many packages will fail to
> build when tuned.

Good question.  It’s also unclear whether tuning should be attempted
recursively on all packages (it wouldn’t make sense to rebuild glibc or
GCC, for instance), or if packages should somehow declare that they are
candidates.

Thanks,
Ludo’.