Making evaluations faster

Making evaluations faster

[Ludovic Courtès]

Hello Guix,

I’ve been looking for ways to make evaluations (computing the derivation
of every package for every supported architecture, like Hydra and
Cuirass do) faster.

I measured evaluations with:

  rm -rf /tmp/cache hydra-jobs.scm
  mkdir /tmp/cache
  XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm

Currently build-aux/hydra/gnu-system.scm, which performs the evaluation
work, turns on auto-compile such that every Guix module gets compiled.

I measured with 2.2.3 the impact of turning off auto-compilation for
everything but the core modules (meaning that package modules get
interpreted instead) and surprisingly, this is slower than compiling
everything:

  * 2.2.3, auto-compile everything minus (system base compile) & co.

  1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata 1201444maxresident)k
  0inputs+203560outputs (0major+285018minor)pagefaults 0swaps

  * 2.2.3, auto-compile (guix packages) only

  2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata 2121532maxresident)k
  128inputs+36568outputs (0major+242281minor)pagefaults 0swaps

  * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix download))

  2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata 2061496maxresident)k
  256inputs+41800outputs (0major+236514minor)pagefaults 0swaps

I guess the extra source properties that are retained when evaluating
account for part the space and time overhead, but I’m not sure this
explains everything.

Andy, what do you think of this?

Thanks,
Ludo’.

Re: Making evaluations faster

[Andy Wingo]

On Tue 19 Dec 2017 10:05, ludo@gnu.org (Ludovic Courtès) writes:

> I’ve been looking for ways to make evaluations (computing the derivation
> of every package for every supported architecture, like Hydra and
> Cuirass do) faster.
>
> I measured evaluations with:
>
>   rm -rf /tmp/cache hydra-jobs.scm
>   mkdir /tmp/cache
>   XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm
>
> Currently build-aux/hydra/gnu-system.scm, which performs the evaluation
> work, turns on auto-compile such that every Guix module gets compiled.
>
> I measured with 2.2.3 the impact of turning off auto-compilation for
> everything but the core modules (meaning that package modules get
> interpreted instead) and surprisingly, this is slower than compiling
> everything:
>
>   * 2.2.3, auto-compile everything minus (system base compile) & co.
>
>   1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata 1201444maxresident)k
>   0inputs+203560outputs (0major+285018minor)pagefaults 0swaps
>
>   * 2.2.3, auto-compile (guix packages) only
>
>   2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata 2121532maxresident)k
>   128inputs+36568outputs (0major+242281minor)pagefaults 0swaps
>
>   * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix download))
>
>   2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata 2061496maxresident)k
>   256inputs+41800outputs (0major+236514minor)pagefaults 0swaps
>
> I guess the extra source properties that are retained when evaluating
> account for part the space and time overhead, but I’m not sure this
> explains everything.
>
> Andy, what do you think of this?

I have literally no idea :)  This isn't really enough information for me
to understand things.  I guess my biggest question would be, why are you
using the auto-compile infrastructure?  Why wouldn't you compile the
things you want to compile and otherwise turn off auto-compilation?

Andy

Re: Making evaluations faster

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> On Tue 19 Dec 2017 10:05, ludo@gnu.org (Ludovic Courtès) writes:
>
>> I’ve been looking for ways to make evaluations (computing the derivation
>> of every package for every supported architecture, like Hydra and
>> Cuirass do) faster.
>>
>> I measured evaluations with:
>>
>>   rm -rf /tmp/cache hydra-jobs.scm
>>   mkdir /tmp/cache
>>   XDG_CACHE_HOME=/tmp/cache make hydra-jobs.scm
>>
>> Currently build-aux/hydra/gnu-system.scm, which performs the evaluation
>> work, turns on auto-compile such that every Guix module gets compiled.
>>
>> I measured with 2.2.3 the impact of turning off auto-compilation for
>> everything but the core modules (meaning that package modules get
>> interpreted instead) and surprisingly, this is slower than compiling
>> everything:
>>
>>   * 2.2.3, auto-compile everything minus (system base compile) & co.
>>
>>   1362.79user 3.35system 20:59.40elapsed 108%CPU (0avgtext+0avgdata 1201444maxresident)k
>>   0inputs+203560outputs (0major+285018minor)pagefaults 0swaps
>>
>>   * 2.2.3, auto-compile (guix packages) only
>>
>>   2462.05user 3.69system 39:26.05elapsed 104%CPU (0avgtext+0avgdata 2121532maxresident)k
>>   128inputs+36568outputs (0major+242281minor)pagefaults 0swaps
>>
>>   * 2.2.3, auto-compile ((guix packages) (guix build-system gnu) (guix download))
>>
>>   2364.22user 3.21system 37:44.45elapsed 104%CPU (0avgtext+0avgdata 2061496maxresident)k
>>   256inputs+41800outputs (0major+236514minor)pagefaults 0swaps
>>
>> I guess the extra source properties that are retained when evaluating
>> account for part the space and time overhead, but I’m not sure this
>> explains everything.
>>
>> Andy, what do you think of this?
>
> I have literally no idea :)  This isn't really enough information for me
> to understand things.  I guess my biggest question would be, why are you
> using the auto-compile infrastructure?  Why wouldn't you compile the
> things you want to compile and otherwise turn off auto-compilation?

Evaluations run code from (guix …) as well as everything in (gnu
packages …).

At every evaluation, we have to start afresh: we cannot just reuse .go
files from a previous evaluation, for instance because the ABI might
have changed.  So, to be on the safe side, we re-compile (or
re-evaluate) all these files.

Auto-compilation is the simplest way to do that.  I thought we could
improve performance by compiling only the hot code and interpreting (gnu
packages …), but the measurements above suggest it’s not the case.

I hope that clarifies the situation.  :-)


As an aside, I also feel (though I haven’t tried) that if our packages
were inert sexps, reading them and turning them into <package> records
would be more efficient than it is to compile or interpret our package
modules.  This is understandable in a way (the sexps would be a dumb
language, so no wonder it can be implemented efficiently), but still a
bit puzzling.

Ludo’.