Re: [GSoC] Continuous integration tool à la Hydra.

[Mathieu Lirzin <mthl@gnu.org>]

Hello Guix!

Here is a second update on my GSoC project after the first month.

As a reminder, Hydra (https://nixos.org/hydra/) is a Nix-based
continuous build system which is used by Guix to compile packages on
different platforms and to distribute packages substitutes.  The aim of
this project is to replace Hydra with a more integrated software written
in Guile, named “Cuirass”.

After providing a first basic evaluation loop described in my previous
mail, I have started thinking about the architecture of the global job
evaluation/compilation process in order to identify what type of data
structures would make sense.  It was not easy for me to reason about the
different layers of current Hydra usage in Guix, so it took me 2 weeks
to analyse it and provide a first decomposition of logic steps.  For now
this decomposition is:

  job-spec > job > build-result

where:
  
  - 'job-spec' defines all the information required by Cuirass to get
    the actual job definitions.  These information contains the
    repository type and url, the file and procedure name which yields a
    list of job, and the list of arguments passed to that procedure.

  - 'job' contains the derivation file name which describe all the
    dependencies.

  - 'build-result' contains the output obtained when realizing/building
    the derivation from a job + some logs.

'job-spec' and 'job' are already implemented in Cuirass. However
'build-result' will require a database to be useful.  Since I have no
experience with databases at all, this last week has been dedicated to
learn more about them, play with SQL queries, and use Guile-dbi and
Guile-sqlite3 bindings.  My plan is to use Sqlite first and eventually
switch to Postgresql later when concurrent writers would be critical.

The next step is to apply my newly acquired knowledge by allowing
'build-result' entries to be added to the database.  I will be AFK for 5
days so I will start working on that, next Sunday.

For those willing to follow my work, a Git repository is available here:

  https://notabug.org/mthl/cuirass

Everyone is of course welcome to provide any feedback.

Thanks.

-- 
Mathieu Lirzin