bug#52250: [cuirass] Feature request: Compare evaluations in the web interface

bug#52250: [cuirass] Feature request: Compare evaluations in the web interface

[Leo Famulari]

As Mark mentioned recently in #33362 [0], the Cuirass web interface is
still missing an important feature compared to Hyrda:

"[...] the ability to compare the results of two evaluations
and show the *differences* between those results, i.e. to enumerate the
newly failing jobs, the newly succeeding jobs, and the newly aborted
jobs."

I remember using this capability in Hydra and it made the branching
workflow much more efficient than is currently possible.

[0]
https://issues.guix.gnu.org/33362#11

bug#52250: [cuirass] Feature request: Compare evaluations in the web interface

[Mathieu Othacehe]

Hello Leo,

> "[...] the ability to compare the results of two evaluations
> and show the *differences* between those results, i.e. to enumerate the
> newly failing jobs, the newly succeeding jobs, and the newly aborted
> jobs."

Yes that's something we should definitely add to Cuirass. On the
database side, the Jobs table gives the list of all the builds related
to an evaluation.

Implementing an evaluation diff (new derivations, newly successful
derivations, newly failing derivations, removed derivations) should just
be a matter of displaying data nicely.

The not so trivial part is how to make it somehow user friendly in the
Web UI. The rest of the interface isn't super easy to understand and I
fear that it will add an extra layer of complexity if not integrated
properly.

A first implementation could be a completely separate page, with two
text inputs to enter evaluation A and evaluation B ids. Then, when
clicking on a "GO" button the diff would be displayed under an array
with different tabulations.

WDYT?

Thanks,

Mathieu

bug#52250: [cuirass] Feature request: Compare evaluations in the web interface

[Leo Famulari]

On Fri, Dec 03, 2021 at 11:04:07AM +0100, Mathieu Othacehe wrote:
> The not so trivial part is how to make it somehow user friendly in the
> Web UI. The rest of the interface isn't super easy to understand and I
> fear that it will add an extra layer of complexity if not integrated
> properly.

Take a look at how Hydra does it:

https://hydra.nixos.org/eval/1727596

See the "Compare to..." button in the upper right? I think we should
just copy what they do. That interface was convenient and efficient for
us in the past.