Hi! As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix are no exception since Guile builds are non-reproducible, despite work done a few years ago: https://issues.guix.gnu.org/20272 Until it’s fixed in Guile proper, what do you think of building Guile 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in ‘core-updates’ now. That would work around the problem for Guile itself. It would increase build times, but probably not that much since the most expensive part (compiling the first few files) is sequential anyway. IIRC this is what Vagrant did for the Debian packages. We could also disable parallel builds in ‘guile-build-system’. It’s only used for small packages so the extra build time is probably OK. Thoughts? Ludo’.
Hi Ludo, On Wed, 20 Jan 2021 at 09:41, Ludovic Courtès <ludo@gnu.org> wrote: > Until it’s fixed in Guile proper, what do you think of building Guile > 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in > ‘core-updates’ now. [...] > We could also disable parallel builds in ‘guile-build-system’. It’s > only used for small packages so the extra build time is probably OK. Good idea. One question: will “guix pull” be slowed? (if it makes sense) Cheers, simon
Hello, Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2021. jan. 20., Sze, 9:42): > > Hi! > > As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix > are no exception since Guile builds are non-reproducible, despite work > done a few years ago: > > https://issues.guix.gnu.org/20272 > > Until it’s fixed in Guile proper, what do you think of building Guile > 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in > ‘core-updates’ now. > > That would work around the problem for Guile itself. It would increase > build times, but probably not that much since the most expensive part > (compiling the first few files) is sequential anyway. IIRC this is what > Vagrant did for the Debian packages. > > We could also disable parallel builds in ‘guile-build-system’. It’s > only used for small packages so the extra build time is probably OK. > > Thoughts? Let's go for it. > > Ludo’. > Regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
Ludovic Courtès writes: Hello, > As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix > are no exception since Guile builds are non-reproducible, despite work > done a few years ago: > > https://issues.guix.gnu.org/20272 Ugh. Thanks for bringing this up again. Would you consider disabling parallel builds in Guile by default, adding something like a "--enable-somewhat-faster-non-reproducible-build" configure option until this is fixed? > Until it’s fixed in Guile proper, what do you think of building Guile > 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in > ‘core-updates’ now. > > That would work around the problem for Guile itself. It would increase > build times, but probably not that much since the most expensive part > (compiling the first few files) is sequential anyway. IIRC this is what > Vagrant did for the Debian packages. Yes, I would support at least doing this. > We could also disable parallel builds in ‘guile-build-system’. It’s > only used for small packages so the extra build time is probably OK. Are packages using guile-build-system expected to build non-reproducible too? In that case, I would certainly support doing it there. Greetings, Janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
Hi Ludovic,
Ludovic Courtès <ludo@gnu.org> writes:
> As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix
> are no exception since Guile builds are non-reproducible, despite work
> done a few years ago:
>
> https://issues.guix.gnu.org/20272
>
> Until it’s fixed in Guile proper, what do you think of building Guile
> 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in
> ‘core-updates’ now.
Sounds good to me. Given the nature of Guile's module system, it's
still not clear to me how to ensure reproducible builds of Guile modules
unless the order of compilation is deterministic. I haven't thought
about it in a few years, though.
Mark
Hi,
Mark H Weaver <mhw@netris.org> writes:
> Hi Ludovic,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix
>> are no exception since Guile builds are non-reproducible, despite work
>> done a few years ago:
>>
>> https://issues.guix.gnu.org/20272
>>
>> Until it’s fixed in Guile proper, what do you think of building Guile
>> 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in
>> ‘core-updates’ now.
I'm not too found of it. It'll make the already slow Guile build much
slower, making the lower strata of core-updates packages more painfully
slow to build and test. It'll also enable us to overlook the issue for
years to come (similarly to the fact that the testsuite of the Guix
package itself hasn't been run in parallel for the last 6 years or so
:-)).
It also won't fix the issue of Guix modules compiled in parallel
suffering from the same problem.
So I'd keep it as it is for now, as a reminder that this is a serious
problem in need of a fix.
Thanks,
Maxim
Hi Maxim, Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > Mark H Weaver <mhw@netris.org> writes: > >> Hi Ludovic, >> >> Ludovic Courtès <ludo@gnu.org> writes: >> >>> As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix >>> are no exception since Guile builds are non-reproducible, despite work >>> done a few years ago: >>> >>> https://issues.guix.gnu.org/20272 >>> >>> Until it’s fixed in Guile proper, what do you think of building Guile >>> 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in >>> ‘core-updates’ now. I pushed that as 2ea52f90141974fb5d88b8c6b1785817c4203da4. > I'm not too found of it. It'll make the already slow Guile build much > slower, making the lower strata of core-updates packages more painfully > slow to build and test. Yes, it’s suboptimal. OTOH, as I wrote, I think the slowdown is not as important as one might think: a large fraction of the Guile build time goes into building ice-9/eval.scm and the first few files, which is already sequential (enforced by Guile’s makefile). > It'll also enable us to overlook the issue for years to come > (similarly to the fact that the testsuite of the Guix package itself > hasn't been run in parallel for the last 6 years or so :-)). It remains and bug to fix in Guile, no doubt. But we also have to be pragmatic IMO. > It also won't fix the issue of Guix modules compiled in parallel > suffering from the same problem. Definitely. > So I'd keep it as it is for now, as a reminder that this is a serious > problem in need of a fix. There’s also the problem that a fix in Guile takes time to deploy (basically a ‘core-updates’ cycle). It’s not the first time we work around a Guile bug until it’s fixed in Guile proper. Thanks, Ludo’.
Hi, Ludovic Courtès <ludo@gnu.org> writes: > Hi Maxim, > > Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > >> Mark H Weaver <mhw@netris.org> writes: >> >>> Hi Ludovic, >>> >>> Ludovic Courtès <ludo@gnu.org> writes: >>> >>>> As the saying goes, “the cobbler’s children go barefoot”. Guile/Guix >>>> are no exception since Guile builds are non-reproducible, despite work >>>> done a few years ago: >>>> >>>> https://issues.guix.gnu.org/20272 >>>> >>>> Until it’s fixed in Guile proper, what do you think of building Guile >>>> 2.0/2.2/3.0 with #:parallel-build? #f ? We could do that in >>>> ‘core-updates’ now. > > I pushed that as 2ea52f90141974fb5d88b8c6b1785817c4203da4. > >> I'm not too found of it. It'll make the already slow Guile build much >> slower, making the lower strata of core-updates packages more painfully >> slow to build and test. > > Yes, it’s suboptimal. OTOH, as I wrote, I think the slowdown is not as > important as one might think: a large fraction of the Guile build time > goes into building ice-9/eval.scm and the first few files, which is > already sequential (enforced by Guile’s makefile). I've built Guile many times on core-updates since and it seems to be true, that the build time didn't change noticeably. >> It'll also enable us to overlook the issue for years to come >> (similarly to the fact that the testsuite of the Guix package itself >> hasn't been run in parallel for the last 6 years or so :-)). > > It remains and bug to fix in Guile, no doubt. But we also have to be > pragmatic IMO. I had the feeling this partial fix would cause build the build time to explode, but given it didn't, I'm happy with it. Thanks, Maxim