unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Will 2021 be the year of build systems on gexps?
@ 2021-02-19 16:01 Ludovic Courtès
  2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-02-19 16:01 UTC (permalink / raw)
  To: Guix-devel

Hello Guix!

Over the last few days I’ve been head-down working on
‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
systems and packages, so we can say goodbye to
‘build-expression->derivation’.  And… it’s quite a ride!

I rebased it on ‘core-updates’, which was a bit tedious (it hadn’t been
touched in 3 years or so).  There are still test failures and build
systems not yet converted, but that’s the easy part, although both are
time-consuming.

The more difficult part is performance.  On current ‘core-updates’ I get:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/z27l6plrxr5wm7818xhj9mdll99jcqz3-qemu-5.1.0.drv
Garbage collection statistics:
  heap size:        80.52 MiB
  allocated:        191.14 MiB
  GC times:         15
  time spent in GC: 0.66 seconds (32% of user time)
2.05user 0.12system 0:01.93elapsed 112%CPU (0avgtext+0avgdata 242872maxresident)k
0inputs+0outputs (0major+34228minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---

but on ‘wip-build-systems-gexp’ I get:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/5n44l8cmrmkr747nsqbxpm4764jdsl3l-qemu-5.1.0.drv
Garbage collection statistics:
  heap size:        80.52 MiB
  allocated:        249.89 MiB
  GC times:         16
  time spent in GC: 0.75 seconds (34% of user time)
2.22user 0.13system 0:02.06elapsed 114%CPU (0avgtext+0avgdata 243532maxresident)k
0inputs+0outputs (0major+34304minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---

In short, 30% more garbage allocated and an 8% slowdown.

I’ve made progress identifying and mitigating what seems to be the main
cause of this (the fact that it’s possible to ungexp a list and that
list will be scanned in its entirety in search of file-like
objects—convenient but expensive) but as you can see, there’s still a
lot to do.

Statprof and gcprof are not super helpful here; I feel a need for better
tools here, or better metrics.

Anyway, that’s the situation.  If you have ideas or if you’re in a
profiling & optimization mood, now’s the time to unleash your
creativity.  :-)

Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2021-03-30 22:32 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-19 16:01 Will 2021 be the year of build systems on gexps? Ludovic Courtès
2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
2021-02-23 14:28 ` Will 2021 be the year of build systems on gexps? Ludovic Courtès
2021-03-10 11:09 ` Ludovic Courtès
2021-03-10 11:15   ` Julien Lepiller
2021-03-10 14:12   ` zimoun
2021-03-10 17:00     ` Leo Famulari
2021-03-23 13:38   ` Ludovic Courtès
2021-03-23 16:08     ` Ricardo Wurmus
2021-03-30  2:16     ` Maxim Cournoyer
2021-03-30 21:38       ` Ludovic Courtès
2021-03-30 22:32         ` Timothy Sample

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).