Proposed change for the disruptive changes process (staging/core-updates)

Proposed change for the disruptive changes process (staging/core-updates)

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2790 bytes --]

Hey,

So, the process as I currently understand it, commits that cause lots of
derivation outputs to change should be pushed to either the core-updates
or staging branches, rather than master. Additionally, these branches
should be periodically frozen (no one pushing new changes other than
fixes), and then once ci.guix.gnu.org has caught up, the branch and
associated changes can be merged to master.

From both the personal perspective as someone continuously building Guix
things to provide substitutes, and from my view on the operation of
ci.guix.gnu.org, I think with a small change to the process, could
benefit both ci.guix.gnu.org and others building Guix packages.

The issue that I'm thinking about with respect to the current process is
that ci.guix.gnu.org doesn't just build staging/core-updates when it's
frozen, it builds it all the time. This means that on top of the cost of
building everything you need to merge the branch, you're potentially
spending a lot of time building and storing outputs that no one is ever
going to request. Now there is some value in building
staging/core-updates all the time, for those who are working on pushing
changes/fixing things on those branches, but I think it would be good to
allow these things to be separated.

I think this issue is clearer for the substitute server I try and
operate. I could track and build things off of staging/core-updates, but
at the moment it's not worth the cost, because of the amount of
redundant building/storing that this would involve. I could also just
enable building staging/core-updates when I see that the relevant branch
is frozen, but that requires paying attention.

A simple process change that I think would help to address this is as
follows (I'll use core-updates as the example, but this applies for
staging as well):

 - core-updates is effectively renamed to core-updates-next

 - When you want to merge core-updates-next in to master, you create
   core-updates pointing at the same commit as core-updates-next. This
   begins the freeze.

 - Once a sufficient amount of time has past for the things on
   core-updates to have been built, you merge in to master

 - Shortly after the merge to master, you then delete the core-updates
   branch

This would mean that a build server can track core-updates, and it'll
only build things when they're relevant for substitutes. For
ci.guix.gnu.org, maybe it could build both branches initially, to
replicate the current setup, but I think in the long run, it would be
helpful to separate out the behaviour so that ci.guix.gnu.org
concentrates on builds for substitutes, and there's another thing for
actually testing out potential core-updates changes.

Thoughts?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Proposed change for the disruptive changes process (staging/core-updates)

[zimoun]

Hi,

On Fri, 23 Oct 2020 at 18:20, Christopher Baines <mail@cbaines.net> wrote:

> A simple process change that I think would help to address this is as
> follows (I'll use core-updates as the example, but this applies for
> staging as well):
>
>  - core-updates is effectively renamed to core-updates-next
>
>  - When you want to merge core-updates-next in to master, you create
>    core-updates pointing at the same commit as core-updates-next. This
>    begins the freeze.
>
>  - Once a sufficient amount of time has past for the things on
>    core-updates to have been built, you merge in to master
>
>  - Shortly after the merge to master, you then delete the core-updates
>    branch
>
> This would mean that a build server can track core-updates, and it'll
> only build things when they're relevant for substitutes. For
> ci.guix.gnu.org, maybe it could build both branches initially, to
> replicate the current setup, but I think in the long run, it would be
> helpful to separate out the behaviour so that ci.guix.gnu.org
> concentrates on builds for substitutes, and there's another thing for
> actually testing out potential core-updates changes.

Based on the current CI issues, and orthogonal with the Chris’s and
Mathieu’s effort (Build Coordinator and Cuirass)––thanks a lot for all
the tough work––I agree with this proposal.

And it would help to reduce the load on Berlin and so increase the
throughput of substitutes.

BTW, I agree that it seems better to separate what is “test” and what is
“production”, i.e., build on separate machines.  All the wip-* branches
could be built on Bayfront.  This implies a rebuild once merged but
somehow this rebuild already happens more than often.

All the best,
simon