all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Will 2021 be the year of build systems on gexps?
@ 2021-02-19 16:01 Ludovic Courtès
  2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-02-19 16:01 UTC (permalink / raw)
  To: Guix-devel

Hello Guix!

Over the last few days I’ve been head-down working on
‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
systems and packages, so we can say goodbye to
‘build-expression->derivation’.  And… it’s quite a ride!

I rebased it on ‘core-updates’, which was a bit tedious (it hadn’t been
touched in 3 years or so).  There are still test failures and build
systems not yet converted, but that’s the easy part, although both are
time-consuming.

The more difficult part is performance.  On current ‘core-updates’ I get:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/z27l6plrxr5wm7818xhj9mdll99jcqz3-qemu-5.1.0.drv
Garbage collection statistics:
  heap size:        80.52 MiB
  allocated:        191.14 MiB
  GC times:         15
  time spent in GC: 0.66 seconds (32% of user time)
2.05user 0.12system 0:01.93elapsed 112%CPU (0avgtext+0avgdata 242872maxresident)k
0inputs+0outputs (0major+34228minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---

but on ‘wip-build-systems-gexp’ I get:

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
/gnu/store/5n44l8cmrmkr747nsqbxpm4764jdsl3l-qemu-5.1.0.drv
Garbage collection statistics:
  heap size:        80.52 MiB
  allocated:        249.89 MiB
  GC times:         16
  time spent in GC: 0.75 seconds (34% of user time)
2.22user 0.13system 0:02.06elapsed 114%CPU (0avgtext+0avgdata 243532maxresident)k
0inputs+0outputs (0major+34304minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---

In short, 30% more garbage allocated and an 8% slowdown.

I’ve made progress identifying and mitigating what seems to be the main
cause of this (the fact that it’s possible to ungexp a list and that
list will be scanned in its entirety in search of file-like
objects—convenient but expensive) but as you can see, there’s still a
lot to do.

Statprof and gcprof are not super helpful here; I feel a need for better
tools here, or better metrics.

Anyway, that’s the situation.  If you have ideas or if you’re in a
profiling & optimization mood, now’s the time to unleash your
creativity.  :-)

Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Rust packages and performance impact on Guix
  2021-02-19 16:01 Will 2021 be the year of build systems on gexps? Ludovic Courtès
@ 2021-02-23 10:31 ` Ludovic Courtès
  2021-02-23 14:28 ` Will 2021 be the year of build systems on gexps? Ludovic Courtès
  2021-03-10 11:09 ` Ludovic Courtès
  2 siblings, 0 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-02-23 10:31 UTC (permalink / raw)
  To: Guix-devel

Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

> but on ‘wip-build-systems-gexp’ I get:
>
> $ GUIX_PROFILING=gc time ./pre-inst-env guix build qemu -d --no-grafts
> /gnu/store/5n44l8cmrmkr747nsqbxpm4764jdsl3l-qemu-5.1.0.drv
> Garbage collection statistics:
>   heap size:        80.52 MiB
>   allocated:        249.89 MiB
>   GC times:         16
>   time spent in GC: 0.75 seconds (34% of user time)
> 2.22user 0.13system 0:02.06elapsed 114%CPU (0avgtext+0avgdata 243532maxresident)k
> 0inputs+0outputs (0major+34304minor)pagefaults 0swaps
>
> In short, 30% more garbage allocated and an 8% slowdown.

In the meantime, librsvg on core-updates switched to the version that
depends on Rust.  As a result, the above command, on core-updates, went
to ~2.25s on my laptop (+10–20%!).

Would be nice to investigate.  One issue is that ‘cargo-build-system’
traverses packages.  The ‘crate-closure’ call in there certainly adds a
fair bit of overhead.  Could we avoid it?

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-02-19 16:01 Will 2021 be the year of build systems on gexps? Ludovic Courtès
  2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
@ 2021-02-23 14:28 ` Ludovic Courtès
  2021-03-10 11:09 ` Ludovic Courtès
  2 siblings, 0 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-02-23 14:28 UTC (permalink / raw)
  To: Guix-devel

Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

> I’ve made progress identifying and mitigating what seems to be the main
> cause of this (the fact that it’s possible to ungexp a list and that
> list will be scanned in its entirety in search of file-like
> objects—convenient but expensive) but as you can see, there’s still a
> lot to do.

I cherry-picked to ‘master’ the gexp optimizations I made on
‘wip-build-systems-gexp’.  The good news is that it pays off on system
compilations, going from (commit 3c548c3e0eade12d3e86b1201dbd95863ca64ea7):

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING="gc object-cache" ./pre-inst-env guix system build -d --no-grafts gnu/system/examples/desktop.tmpl
/gnu/store/g6kd2blajq6yja80516m85zv07d4ysb3-system.drv
Store object cache:
  fresh caches:     2
  lookups:      119282
  hits:         116225 (97.4%)
Garbage collection statistics:
  heap size:        119.43 MiB
  allocated:        649.07 MiB
  GC times:         26
  time spent in GC: 2.15 seconds (37% of user time)
--8<---------------cut here---------------end--------------->8---

to (commit 0a912b3a434c9ff9515f9036feae4d3a7075fb2f):

--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING="gc object-cache" ./pre-inst-env guix system build -d --no-grafts gnu/system/examples/desktop.tmpl
/gnu/store/f63c8rk8rwrfr5cl4s74hnx25l3z1hrp-system.drv
Store object cache:
  fresh caches:     2
  lookups:      119285
  hits:         116227 (97.4%)
  cache size:    3058 entries
Garbage collection statistics:
  heap size:        111.43 MiB
  allocated:        608.01 MiB
  GC times:         25
  time spent in GC: 2.00 seconds (37% of user time)
--8<---------------cut here---------------end--------------->8---

Reduced heap allocations (-6%), reduce heap size (-7%), and reduced run
time (ditto).  Getting better!

Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-02-19 16:01 Will 2021 be the year of build systems on gexps? Ludovic Courtès
  2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
  2021-02-23 14:28 ` Will 2021 be the year of build systems on gexps? Ludovic Courtès
@ 2021-03-10 11:09 ` Ludovic Courtès
  2021-03-10 11:15   ` Julien Lepiller
                     ` (2 more replies)
  2 siblings, 3 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-03-10 11:09 UTC (permalink / raw)
  To: Guix-devel

Hello!

Ludovic Courtès <ludo@gnu.org> skribis:

> Over the last few days I’ve been head-down working on
> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
> systems and packages, so we can say goodbye to
> ‘build-expression->derivation’.  And… it’s quite a ride!

The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
it can build ‘guix’ and cross-build things like ‘sed’:

  https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp

  https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
  currently has unrelated problems)

In terms of performance, there’s still a ~10% slowdown when computing
derivations compared to the ‘core-updates’ revision the branch is based
on.

Here’s what I’d like to do in the coming days, if that doesn’t interfere
with what others have in mind for the upcoming release:

  • Monitor build failures due to typos/thinkos made while adjusting
    build systems;

  • Merge on ‘core-updates’.

Then there are optimizations to work on, but that can take a bit longer.
In particular, in ‘gexp->derivation’, allow file-like objects to be
specified as environment variable values.  In turn, use that so that,
say, ‘gnu-build-system’ has a single builder for all its packages and
just calls ‘getenv’ to get the value of its various parameters, similar
to what (guix git-download) does.

That said, if people think it’s too late in the cycle, we could just as
well keep it for the next cycle.

Thoughts?

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-10 11:09 ` Ludovic Courtès
@ 2021-03-10 11:15   ` Julien Lepiller
  2021-03-10 14:12   ` zimoun
  2021-03-23 13:38   ` Ludovic Courtès
  2 siblings, 0 replies; 12+ messages in thread
From: Julien Lepiller @ 2021-03-10 11:15 UTC (permalink / raw)
  To: guix-devel, Ludovic Courtès

[-- Attachment #1: Type: text/plain, Size: 2092 bytes --]

If we aim at merging c-u before next release, we should probably wait next cycle, as this introduces quite a lot of changes, and packages might break in subtle ways.

10% increase for computing derivations is not great :/ it already takes a long time to do that on my arm system ^^". I wonder how we could improve that?

Le 10 mars 2021 06:09:23 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hello!
>
>Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Over the last few days I’ve been head-down working on
>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to
>build
>> systems and packages, so we can say goodbye to
>> ‘build-expression->derivation’.  And… it’s quite a ride!
>
>The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being
>built,
>it can build ‘guix’ and cross-build things like ‘sed’:
>
>https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>
>  https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>  currently has unrelated problems)
>
>In terms of performance, there’s still a ~10% slowdown when computing
>derivations compared to the ‘core-updates’ revision the branch is based
>on.
>
>Here’s what I’d like to do in the coming days, if that doesn’t
>interfere
>with what others have in mind for the upcoming release:
>
>  • Monitor build failures due to typos/thinkos made while adjusting
>    build systems;
>
>  • Merge on ‘core-updates’.
>
>Then there are optimizations to work on, but that can take a bit
>longer.
>In particular, in ‘gexp->derivation’, allow file-like objects to be
>specified as environment variable values.  In turn, use that so that,
>say, ‘gnu-build-system’ has a single builder for all its packages and
>just calls ‘getenv’ to get the value of its various parameters, similar
>to what (guix git-download) does.
>
>That said, if people think it’s too late in the cycle, we could just as
>well keep it for the next cycle.
>
>Thoughts?
>
>Thanks,
>Ludo’.

[-- Attachment #2: Type: text/html, Size: 2683 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-10 11:09 ` Ludovic Courtès
  2021-03-10 11:15   ` Julien Lepiller
@ 2021-03-10 14:12   ` zimoun
  2021-03-10 17:00     ` Leo Famulari
  2021-03-23 13:38   ` Ludovic Courtès
  2 siblings, 1 reply; 12+ messages in thread
From: zimoun @ 2021-03-10 14:12 UTC (permalink / raw)
  To: Ludovic Courtès, Guix-devel

Hi Ludo,

On Wed, 10 Mar 2021 at 12:09, Ludovic Courtès <ludo@gnu.org> wrote:

> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
> it can build ‘guix’ and cross-build things like ‘sed’:
>
>   https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>
>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>   currently has unrelated problems)

Cool!

> In terms of performance, there’s still a ~10% slowdown when computing
> derivations compared to the ‘core-updates’ revision the branch is based
> on.

Less cool! :-)


> Here’s what I’d like to do in the coming days, if that doesn’t interfere
> with what others have in mind for the upcoming release:
>
>   • Monitor build failures due to typos/thinkos made while adjusting
>     build systems;
>
>   • Merge on ‘core-updates’.

[...]

> That said, if people think it’s too late in the cycle, we could just as
> well keep it for the next cycle.

I do not think it interferes with the release since for now and except a
big change, the plan is to release without the core-updates merge.
Well, that’s my understanding of the previous discussion.


Cheers,
simon


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-10 14:12   ` zimoun
@ 2021-03-10 17:00     ` Leo Famulari
  0 siblings, 0 replies; 12+ messages in thread
From: Leo Famulari @ 2021-03-10 17:00 UTC (permalink / raw)
  To: zimoun; +Cc: Guix-devel

On Wed, Mar 10, 2021 at 03:12:42PM +0100, zimoun wrote:
> I do not think it interferes with the release since for now and except a
> big change, the plan is to release without the core-updates merge.
> Well, that’s my understanding of the previous discussion.

That's my understand as well.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-10 11:09 ` Ludovic Courtès
  2021-03-10 11:15   ` Julien Lepiller
  2021-03-10 14:12   ` zimoun
@ 2021-03-23 13:38   ` Ludovic Courtès
  2021-03-23 16:08     ` Ricardo Wurmus
  2021-03-30  2:16     ` Maxim Cournoyer
  2 siblings, 2 replies; 12+ messages in thread
From: Ludovic Courtès @ 2021-03-23 13:38 UTC (permalink / raw)
  To: Guix-devel

Hello!

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Over the last few days I’ve been head-down working on
>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
>> systems and packages, so we can say goodbye to
>> ‘build-expression->derivation’.  And… it’s quite a ride!
>
> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
> it can build ‘guix’ and cross-build things like ‘sed’:
>
>   https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>
>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>   currently has unrelated problems)

It’s building and well!

> In terms of performance, there’s still a ~10% slowdown when computing
> derivations compared to the ‘core-updates’ revision the branch is based
> on.

I made some improvements yesterday (reducing object cache lookups and
the number of entries therein), but we’re still in the 10% ballpark.
WIP branch:

--8<---------------cut here---------------start------------->8---
$ git log |head -5
commit 082df93be3472e0f38970634260af8c432420b35
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Mon Mar 8 13:59:23 2021 +0100

    gnu: docbook-xsl: Move 'use-modules' form to the top level.
$ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
/gnu/store/fsrbbi8vfrwwdz2dlyzpfvvnky03nczz-libreoffice-6.4.7.2.drv
Garbage collection statistics:
  heap size:        87.18 MiB
  allocated:        254.25 MiB
  GC times:         16
  time spent in GC: 0.74 seconds (31% of user time)

real	0m2.225s
user	0m2.415s
sys	0m0.087s
--8<---------------cut here---------------end--------------->8---

Compared to ‘core-updates’:

--8<---------------cut here---------------start------------->8---
$ git log |head -5
commit b35581bd63d929e83d18f42b067f63efc867353c
Author: Efraim Flashner <efraim@flashner.co.il>
Date:   Sun Mar 21 09:42:06 2021 +0200

    gnu: openjpeg: Update to 2.4.0.
$ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
/gnu/store/irdhm6jx30bgdxvgb0an1mn223rzshkg-libreoffice-6.4.7.2.drv
Garbage collection statistics:
  heap size:        79.18 MiB
  allocated:        216.51 MiB
  GC times:         16
  time spent in GC: 0.74 seconds (33% of user time)

real	0m2.094s
user	0m2.277s
sys	0m0.106s
--8<---------------cut here---------------end--------------->8---

> Here’s what I’d like to do in the coming days, if that doesn’t interfere
> with what others have in mind for the upcoming release:
>
>   • Monitor build failures due to typos/thinkos made while adjusting
>     build systems;
>
>   • Merge on ‘core-updates’.

I’ll go ahead with that if there are no objections.

> Then there are optimizations to work on, but that can take a bit longer.
> In particular, in ‘gexp->derivation’, allow file-like objects to be
> specified as environment variable values.  In turn, use that so that,
> say, ‘gnu-build-system’ has a single builder for all its packages and
> just calls ‘getenv’ to get the value of its various parameters, similar
> to what (guix git-download) does.

I’m also starting work in this area.

Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-23 13:38   ` Ludovic Courtès
@ 2021-03-23 16:08     ` Ricardo Wurmus
  2021-03-30  2:16     ` Maxim Cournoyer
  1 sibling, 0 replies; 12+ messages in thread
From: Ricardo Wurmus @ 2021-03-23 16:08 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> skribis:
>>
>>> Over the last few days I’ve been head-down working on
>>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
>>> systems and packages, so we can say goodbye to
>>> ‘build-expression->derivation’.  And… it’s quite a ride!
>>
>> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
>> it can build ‘guix’ and cross-build things like ‘sed’:
>>
>>   https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>>
>>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>>   currently has unrelated problems)
>
> It’s building and well!

Excellent!  It’s great to see that you brought this idea across all
the hurdles you encountered in previous attempts.  Inspiring!

>> Here’s what I’d like to do in the coming days, if that doesn’t interfere
>> with what others have in mind for the upcoming release:
>>
>>   • Monitor build failures due to typos/thinkos made while adjusting
>>     build systems;
>>
>>   • Merge on ‘core-updates’.
>
> I’ll go ahead with that if there are no objections.

This sounds reasonable to me.  “core-updates” will need some work to
push it into shape for the next merge, but I don’t think merging a
well-monitored branch into “core-updates” would make the job
considerably harder.

Thank you!

-- 
Ricardo


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-23 13:38   ` Ludovic Courtès
  2021-03-23 16:08     ` Ricardo Wurmus
@ 2021-03-30  2:16     ` Maxim Cournoyer
  2021-03-30 21:38       ` Ludovic Courtès
  1 sibling, 1 reply; 12+ messages in thread
From: Maxim Cournoyer @ 2021-03-30  2:16 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix-devel

Hi Ludo!

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> skribis:
>>
>>> Over the last few days I’ve been head-down working on
>>> ‘wip-build-systems-gexp’, the mythical branch that brings gexps to build
>>> systems and packages, so we can say goodbye to
>>> ‘build-expression->derivation’.  And… it’s quite a ride!
>>
>> The current tip of ‘wip-build-systems-gexp’ Just Works; it’s being built,
>> it can build ‘guix’ and cross-build things like ‘sed’:
>>
>>   https://data.guix-patches.cbaines.net/repository/2/branch/wip-build-systems-gexp
>>
>>   https://ci.guix.gnu.org/jobset/wip-build-systems-gexp (though Cuirass
>>   currently has unrelated problems)
>
> It’s building and well!
>
>> In terms of performance, there’s still a ~10% slowdown when computing
>> derivations compared to the ‘core-updates’ revision the branch is based
>> on.
>
> I made some improvements yesterday (reducing object cache lookups and
> the number of entries therein), but we’re still in the 10% ballpark.
> WIP branch:
>
> $ git log |head -5
> commit 082df93be3472e0f38970634260af8c432420b35
> Author: Ludovic Courtès <ludo@gnu.org>
> Date:   Mon Mar 8 13:59:23 2021 +0100
>
>     gnu: docbook-xsl: Move 'use-modules' form to the top level.
> $ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
> /gnu/store/fsrbbi8vfrwwdz2dlyzpfvvnky03nczz-libreoffice-6.4.7.2.drv
> Garbage collection statistics:
>   heap size:        87.18 MiB
>   allocated:        254.25 MiB
>   GC times:         16
>   time spent in GC: 0.74 seconds (31% of user time)
>
> real	0m2.225s
> user	0m2.415s
> sys	0m0.087s
>
>
> Compared to ‘core-updates’:
>
> $ git log |head -5
> commit b35581bd63d929e83d18f42b067f63efc867353c
> Author: Efraim Flashner <efraim@flashner.co.il>
> Date:   Sun Mar 21 09:42:06 2021 +0200
>
>     gnu: openjpeg: Update to 2.4.0.
> $ time GUIX_PROFILING=gc ./pre-inst-env guix build libreoffice --no-grafts -d
> /gnu/store/irdhm6jx30bgdxvgb0an1mn223rzshkg-libreoffice-6.4.7.2.drv
> Garbage collection statistics:
>   heap size:        79.18 MiB
>   allocated:        216.51 MiB
>   GC times:         16
>   time spent in GC: 0.74 seconds (33% of user time)
>
> real	0m2.094s
> user	0m2.277s
> sys	0m0.106s

This looks reasonable!

>> Here’s what I’d like to do in the coming days, if that doesn’t interfere
>> with what others have in mind for the upcoming release:
>>
>>   • Monitor build failures due to typos/thinkos made while adjusting
>>     build systems;
>>
>>   • Merge on ‘core-updates’.
>
> I’ll go ahead with that if there are no objections.

Sounds good!  Thanks for picking up this work! :-)

Maxim


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-30  2:16     ` Maxim Cournoyer
@ 2021-03-30 21:38       ` Ludovic Courtès
  2021-03-30 22:32         ` Timothy Sample
  0 siblings, 1 reply; 12+ messages in thread
From: Ludovic Courtès @ 2021-03-30 21:38 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: Guix-devel

Hi Maxim and all,

So this is it, 2021 *is* the year of build systems on gexps!

I’ve rebased it and pushed on ‘core-updates’ with
2eafeb2f3d661061bc412c3f27c90202e4532532 as the tip.

The build farm has a bit of work to do now it seems.  Hmm actually the
evaluation at <https://ci.guix.gnu.org/jobset/core-updates> just failed
with a yellow cross, not sure what that means.

Anyway, I’ll be monitoring in the coming days, but do ping me when you
encounter problems!

It’s crazy that it took 6 years (six!) to get past the finish line.  The
initial reason for not merging it was that I was worried about
performance regressions; there’s still work to do in this area, as I
wrote earlier, but it’s looking good.  Another reason is that I was
afraid of unlocking features that would break assumptions we make about
packages—for instance, that all the inputs are specified in the various
inputs fields, and that these fields are all we need to care about when
doing graph rewriting.  (I now think this fear was unwarranted.)

And then, rebasing the branch became harder every time, there were more
and more build systems that needed to be converted… and there were
always more pressing issues to work on.

What’s the lesson here?  I don’t know!  Had it been done six years ago,
it would have had to go through a number of iterations:
‘with-imported-modules’ and ‘with-extensions’ didn’t exist, the template
mechanism in (guix monads) didn’t exist, self-referential records
(‘this-package’, etc.) weren’t there.  It would have worked though, and
perhaps we’d have explored other parts of the design space, who knows.

Ludo’.


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Will 2021 be the year of build systems on gexps?
  2021-03-30 21:38       ` Ludovic Courtès
@ 2021-03-30 22:32         ` Timothy Sample
  0 siblings, 0 replies; 12+ messages in thread
From: Timothy Sample @ 2021-03-30 22:32 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix-devel, Maxim Cournoyer

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> So this is it, 2021 *is* the year of build systems on gexps!

Wow!  Congratulations!  I am very happy to see this finally come
together.

> It’s crazy that it took 6 years (six!) to get past the finish line.

I think I’ve only known what a gexp is for three years... and that’s
about how long I’ve been excited by this feature.

Thanks for sticking with it for six years!


-- Tim


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2021-03-30 22:32 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-19 16:01 Will 2021 be the year of build systems on gexps? Ludovic Courtès
2021-02-23 10:31 ` Rust packages and performance impact on Guix Ludovic Courtès
2021-02-23 14:28 ` Will 2021 be the year of build systems on gexps? Ludovic Courtès
2021-03-10 11:09 ` Ludovic Courtès
2021-03-10 11:15   ` Julien Lepiller
2021-03-10 14:12   ` zimoun
2021-03-10 17:00     ` Leo Famulari
2021-03-23 13:38   ` Ludovic Courtès
2021-03-23 16:08     ` Ricardo Wurmus
2021-03-30  2:16     ` Maxim Cournoyer
2021-03-30 21:38       ` Ludovic Courtès
2021-03-30 22:32         ` Timothy Sample

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.