Re: [GSoC] Development of Cuirass.

[ludo@gnu.org (Ludovic Courtès)]

Hello Mathieu!

Mathieu Lirzin <mthl@gnu.org> skribis:

>   The project is the continuation of last year work, and is separated in
>   two parts.  The first part is to improve the testing infrastructure.
>   The second one is about providing a more complete HTTP API.

Sounds cool!

> 2.1 The testing infrastructure
> ──────────────────────────────
>
>   Currently Cuirass is providing a small test suite consisting only of
>   unit tests.  The integration tests are done manually with basic
>   examples consisting in building small packages such as GNU Hello.
>   This is not good, because actual packages take a consequent time to
>   compile, and testing only basic examples leaves the interesting
>   complex case out of the scope of testing.  The worst is that the
>   result depend on the phase of the moon since every commit in Guix can
>   potentially break the build.
>
>   The idea is to take inspiration from the Guix test suite by providing
>   tiny mocked packages that will help having lightweight build
>   specifications.  Those tests require launching another instance of the
>   `guix-daemon' with a local temporary store.

Sounds like a good idea.

It would be nice to refine what it is we’re trying to test and that’s
not addressed by unit tests.  At one end of the spectrum, I imagine we
could be spawning a couple of GuixSD VMs connected together, one serving
a Git repo, and the other one running Cuirass pulling from that repo (a
system test).

At the middle of the spectrum is something like you describe, I think:
run Cuirass directly upon “make check”, though there are complications:
what repo do we pull from, do we really want to build packages and if
not how to we mock builds, etc.

There are several components we’d like to test more closely IMO: SCM
polling, evaluation, and job scheduling.  Perhaps some of that can also
be achieved through unit tests?

> 2.2 HTTP API
> ────────────
>
>   Since commit [12d71ee098c3eb84a9a5dc2419bd37a3b9e55dd2] Cuirass has a
>   web server which is running in a separate thread of the `cuirass'
>   process.  However the resources it provides are pretty limited since
>   you can only query the lists of specifications currently watched.
>   This is done with the `/specifications' or `/jobsets' resource names.
>   "Jobsets" is the terminology used by Hydra and is used here for
>   compatibility with its HTTP API.

For a start, I think we should implement all or a subset of the Hydra
API, as-is (it’s currently used by Emacs-Guix):

  https://github.com/NixOS/hydra/blob/master/src/lib/Hydra/Controller/API.pm

A large part of it is about monitoring the status of Hydra: queued
builds, recently-completed builds, jobs, etc.  That is the most pressing
need, I think.

>   The principal objective of this part is to provide a more complete API
>   which allows users to query the build results and to add their
>   specifications to the current set.  The design of this API will apply
>   the principles of the Representational State Transfert (REST)
>   architectural pattern[2].  Sensitive requests should be done with an
>   authentification mechanism which is not determined yet.  I currently
>   have no experience with any and lack the knowledge to properly choose
>   one.

pelzflorian already gave good ideas, so that’s good.  :-)

>   This roadmap lacks details about the concrete tasks and its associated
>   timeline.  However I prefer having some feedback about this proposal
>   first, before providing a complete roadmap.
>
>   • From May 30 to June 26, I will work on the testing infrastructure,.
>   • From June 27 to August 29, I will work on the HTTP API.

You may find that you’ll want to interleave tasks.  I don’t know about
you, but sometimes I get bored if I keep working on the same task or
domain for too long, so switching to something else for a while can be a
relief.  :-)

There are a couple of isolated tasks that come to mind, which do not
really fit in the proposal but are worth looking into as time permits:

  • improved error reporting upon evaluation failures and so on;

  • improved error recovery;

  • use of Guile-Git instead of Git.

> 4 Feedback
> ══════════
>
>   Some people have expressed their interest in having a Web /frontend/
>   for Cuirass.  To avoid possible confusions I have no plan in working
>   on it this summer.  While this would be a very valuable effort, I
>   don't feel up to the task.

Understood.  That said, once the JSON API is in place, Web-savvy people
could certainly come with a JSish frontend.  It might be that
<https://github.com/hercules-ci/hercules> would work.

Thanks for posting this proposal!

Ludo’.