Re: Will 2021 be the year of build systems on gexps?

["Ludovic Courtès" <ludo@gnu.org>]

Hello!

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Over the last few days I’ve been head-down working on
>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
>> systems and packages, so we can say goodbye to
>> ‘build-expression->derivation’.  And… it’s quite a ride!
>
> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
> it can build ‘guix’ and cross-build things like ‘sed’:
>
>   https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>
>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>   currently has unrelated problems)

It’s building and well!

> In terms of performance, there’s still a ~10% slowdown when computing
> derivations compared to the ‘core-updates’ revision the branch is based
> on.

I made some improvements yesterday (reducing object cache lookups and
the number of entries therein), but we’re still in the 10% ballpark.
WIP branch:

--8<---------------cut here---------------start------------->8---
$ git log |head -5
commit 082df93be3472e0f38970634260af8c432420b35
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Mon Mar 8 13:59:23 2021 +0100

    gnu: docbook-xsl: Move 'use-modules' form to the top level.
$ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
/gnu/store/fsrbbi8vfrwwdz2dlyzpfvvnky03nczz-libreoffice-6.4.7.2.drv
Garbage collection statistics:
  heap size:        87.18 MiB
  allocated:        254.25 MiB
  GC times:         16
  time spent in GC: 0.74 seconds (31% of user time)

real	0m2.225s
user	0m2.415s
sys	0m0.087s
--8<---------------cut here---------------end--------------->8---

Compared to ‘core-updates’:

--8<---------------cut here---------------start------------->8---
$ git log |head -5
commit b35581bd63d929e83d18f42b067f63efc867353c
Author: Efraim Flashner <efraim@flashner.co.il>
Date:   Sun Mar 21 09:42:06 2021 +0200

    gnu: openjpeg: Update to 2.4.0.
$ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
/gnu/store/irdhm6jx30bgdxvgb0an1mn223rzshkg-libreoffice-6.4.7.2.drv
Garbage collection statistics:
  heap size:        79.18 MiB
  allocated:        216.51 MiB
  GC times:         16
  time spent in GC: 0.74 seconds (33% of user time)

real	0m2.094s
user	0m2.277s
sys	0m0.106s
--8<---------------cut here---------------end--------------->8---

> Here’s what I’d like to do in the coming days, if that doesn’t interfere
> with what others have in mind for the upcoming release:
>
>   • Monitor build failures due to typos/thinkos made while adjusting
>     build systems;
>
>   • Merge on ‘core-updates’.

I’ll go ahead with that if there are no objections.

> Then there are optimizations to work on, but that can take a bit longer.
> In particular, in ‘gexp->derivation’, allow file-like objects to be
> specified as environment variable values.  In turn, use that so that,
> say, ‘gnu-build-system’ has a single builder for all its packages and
> just calls ‘getenv’ to get the value of its various parameters, similar
> to what (guix git-download) does.

I’m also starting work in this area.

Ludo’.