From: "Ludovic Courtès" <ludo@gnu.org>
To: Guix-devel <guix-devel@gnu.org>
Subject: Will 2021 be the year of build systems on gexps?
Date: Fri, 19 Feb 2021 17:01:38 +0100 [thread overview]
Message-ID: <87im6ojbbx.fsf@gnu.org> (raw)
Hello Guix!
Over the last few days I’ve been head-down working on
‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
systems and packages, so we can say goodbye to
‘build-expression->derivation’. And… it’s quite a ride!
I rebased it on ‘core-updates’, which was a bit tedious (it hadn’t been
touched in 3 years or so). There are still test failures and build
systems not yet converted, but that’s the easy part, although both are
time-consuming.
The more difficult part is performance. On current ‘core-updates’ I get:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/z27l6plrxr5wm7818xhj9mdll99jcqz3-qemu-5.1.0.drv
Garbage collection statistics:
heap size: 80.52 MiB
allocated: 191.14 MiB
GC times: 15
time spent in GC: 0.66 seconds (32% of user time)
2.05user 0.12system 0:01.93elapsed 112%CPU (0avgtext+0avgdata 242872maxresident)k
0inputs+0outputs (0major+34228minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
but on ‘wip-build-systems-gexp’ I get:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/5n44l8cmrmkr747nsqbxpm4764jdsl3l-qemu-5.1.0.drv
Garbage collection statistics:
heap size: 80.52 MiB
allocated: 249.89 MiB
GC times: 16
time spent in GC: 0.75 seconds (34% of user time)
2.22user 0.13system 0:02.06elapsed 114%CPU (0avgtext+0avgdata 243532maxresident)k
0inputs+0outputs (0major+34304minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
In short, 30% more garbage allocated and an 8% slowdown.
I’ve made progress identifying and mitigating what seems to be the main
cause of this (the fact that it’s possible to ungexp a list and that
list will be scanned in its entirety in search of file-like
objects—convenient but expensive) but as you can see, there’s still a
lot to do.
Statprof and gcprof are not super helpful here; I feel a need for better
tools here, or better metrics.
Anyway, that’s the situation. If you have ideas or if you’re in a
profiling & optimization mood, now’s the time to unleash your
creativity. :-)
Ludo’.
next reply other threads:[~2021-02-19 16:03 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-19 16:01 Ludovic Courtès [this message]
2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
2021-02-23 14:28 ` Will 2021 be the year of build systems on gexps? Ludovic Courtès
2021-03-10 11:09 ` Ludovic Courtès
2021-03-10 11:15 ` Julien Lepiller
2021-03-10 14:12 ` zimoun
2021-03-10 17:00 ` Leo Famulari
2021-03-23 13:38 ` Ludovic Courtès
2021-03-23 16:08 ` Ricardo Wurmus
2021-03-30 2:16 ` Maxim Cournoyer
2021-03-30 21:38 ` Ludovic Courtès
2021-03-30 22:32 ` Timothy Sample
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87im6ojbbx.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.