[bug#50384] [PATCH v2] Optimise search-patch (reducing I/O)

[Maxime Devos <maximedevos@telenet.be>]

[-- Attachment #1: Type: text/plain, Size: 3055 bytes --]

zimoun schreef op ma 06-09-2021 om 10:39 [+0200]:
> Hi Maxime,
> 
> Thanks for looking at optimising stuff. :-)
> 
> On Sun, 05 Sep 2021 at 21:48, Maxime Devos <maximedevos@telenet.be> wrote:
> 
> > Let's compare the numbers again!  This time, I've run
> > 
> > echo powersave |sudo tee /sys/devices/system/cpu/cpufreq/policy{0,1,2,3}/scaling_governor
> > 
> > to make sure the CPU frequency doesn't change.  On a hot (disk) cache:
> > 
> > # After the patch series
> > time GUIX_PROFILING="rpc gc" ./the-optimised-guix/bin/guix build -d --no-grafts pigx
> 
> [...]
> 
> > So on a hot disk cache, there doesn't appear to be any improvement
> > (except for ‘time spent in GC’ -- presumably that's due to the optimisations
> > to guix/base32.scm).
> 
> Which kind of disk do you have?  SSD, spinning HDD, other?

A spinning disk, presumably a HDD.  FWIW, it's a ‘TOSHIBA MQ01ABD100 (AX1P2C)’
that has been ‘on’ for 10 months and 10 days, according to ‘SMART-data and selftests’
I just noticed it has ‘2304 bad sectors’.  Maybe I should make backups and run file
system checks?

> 
> > What about a cold cache?
> > 
> > # After the patch series
> > 
> > sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
> > ./the-optimised-guix/bin/guix --help
> > time GUIX_PROFILING="rpc gc" ./the-optimised-guix/bin/guix build -d --no-grafts pigx
> > /gnu/store/fq6x8d2vcm6sbjkimg7g8kcgb4c5xv1b-pigx-0.0.3.drv
> > Remote procedure call summary: 5949 RPCs
> >   built-in-builders              ...     1
> >   add-to-store                   ...     3
> >   add-to-store/tree              ...    26
> >   add-temp-root                  ...   195
> >   valid-path?                    ...   195
> >   add-text-to-store              ...  5529
> > Garbage collection statistics:
> >   heap size:        93.85 MiB
> >   allocated:        312.03 MiB
> >   GC times:         17
> >   time spent in GC: 3.37 seconds (23% of user time)
> > 
> > real	1m39,178s
> > user	0m14,557s
> > sys	0m0,990s
> 
> How the average (against 3 examples) looks like?

I'll try to optimise more things and report the average for the v3. 

> > It seems that if the disk cache is cold, the time-to-derivation decreases
> > a little by this patch series.  Much less than I had hoped for though; I'll
> > have to look into other areas for interesting performance gains ...
> 
> Please update the number using your diff showed in [1]. :-)
> 
> 
> 1: <http://issues.guix.gnu.org/50384#4>

Are you referring to:

> Oops, I forgot to include the following change to build-aux/compile-all.scm:
>       (or (not (file-exists? go))
>           (file-mtime<? go file)
> -          (any (cut file-mtime<? go <>) extra-dependencies))))
> +          (any (lambda (dependency)
> +               (or (not (file-exists? dependency))
> +                   (file-mtime<? go dependency))) extra-dependencies))))

That's compilation-time only, so that cannot affect the timing of "guix build -d ...".

Geetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]