bug#63413: Stop and restart builds in cuirass

bug#63413: Stop and restart builds in cuirass

[Andreas Enge]

Stopping and starting builds in cuirass is not very comfortable.

When working on core-updates, I stopped builds for some old evaluations
on aarch64, where our build power would not be enough, assuming that many
of the old builds were made obsolete by newer changes. (For instance,
these could be builds in branches that were already merged to master.)

However, this also stopped the builds that were not obsolete. And if the
derivation was unchanged in a later evaluation (or a different branch),
it would be kept as failed and not be restarted. I would argue that the
desirable behaviour would be to try all derivations in a new evaluation,
regardless of whether they were stopped in a previous evaluation.

A use case are feature branches:
Assume x-team and y-team are branches that are developed simultaneously;
now x-team is ready first and gets merged to master. All builds of y-team
are stopped (since there is no point in continuing with builds made obsolete
by changes in x-team), the branch is rebased on master, and now all builds
need to be restarted on the rebuilt branch. There is a button for this,
but it also restarts genuinely failed builds, if I remember correctly.
(In any case it did not solve the problem, but I have forgotten the
details.)

Restarting builds manually requires to take the dependency order into
account. If x->y->z is a dependency chain, x has been stopped, and y or z
are restarted, they will immediately fail since x is not available;
otherwise said, restarting manually requires a manual traversal of the
dependency graph, which should be automated.

It would also be a useful feature to restart all builds of dependent
packages: If x is "repaired" and manually started, it would be nice to
be able to start all its dependents (ordered automatically by cuirass)
to see whether they now succeed, instead of forcing us to manually
traverse the graph again.
For instance, we recently got a report on guix-devel of "no space left
on device" for icedtea@3. This would be easy to solve by a "guix gc"
on the build machines, but we currently have no way of restarting the
many builds dependent on icedtea@3 other than restarting all packages.

Andreas

bug#63413: Stop and restart builds in cuirass

[Ludovic Courtès]

Hi Andreas,

Andreas Enge <andreas@enge.fr> skribis:

> Stopping and starting builds in cuirass is not very comfortable.
>
> When working on core-updates, I stopped builds for some old evaluations
> on aarch64, where our build power would not be enough, assuming that many
> of the old builds were made obsolete by newer changes. (For instance,
> these could be builds in branches that were already merged to master.)
>
> However, this also stopped the builds that were not obsolete. And if the
> derivation was unchanged in a later evaluation (or a different branch),
> it would be kept as failed and not be restarted. I would argue that the
> desirable behaviour would be to try all derivations in a new evaluation,
> regardless of whether they were stopped in a previous evaluation.

(For clarity: s/stop/cancel/ above.)

I’m happy to report that this bug should be fixed in the snapshot
recently deployed on ci.guix.gnu.org.  The fix is here:

  https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=12d79854ee7685b356a72ca374f2a90b8efe5903

Lemme know if it turns out to be a broken fix!

Ludo’.

bug#63413: Stop and restart builds in cuirass

[Andreas Enge]

Hello,

Am Tue, Jun 04, 2024 at 03:59:53PM +0200 schrieb Ludovic Courtès:
> I’m happy to report that this bug should be fixed in the snapshot
> recently deployed on ci.guix.gnu.org.  The fix is here:
>   https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git/commit/?id=12d79854ee7685b356a72ca374f2a90b8efe5903

this looks good, thanks a lot for fixing this long standing annoyance!

Andreas