[GSoC] Continuous integration tool à la Hydra.

[Mathieu Lirzin <mthl@gnu.org>]

Hello Guix!

Here is a first update on my GSoC project after the first week.

Hydra (https://nixos.org/hydra/) is a Nix-based continuous build system
which is used by Guix to compile packages on different platforms and to
distribute packages substitutes.  The aim of this project is to replace
Hydra with a more integrated software written in Guile.

Since it would be cumbersome to speak about something without giving it
a name, I have decided to call this software “Cuirass”.

While being intended as a general purpose continuous integration tool,
the first iteration of Cuirass is focusing only on building Guix
packages.  The logic of the program is simple since 'cuirass' sits in a
loop:

 - Fetching new changes from Guix master branch
 - Compiling Guix code
 - Evaluating packages derivations
 - Building those derivations

The Guix repository is kept in a directory set by 'CUIRASS_CACHEDIR'
environment variable to avoid cloning it every time.  Even if the
algorithm is trivial, the implementation has required some thought in
order to allow using guile code fetched from the Guix git repository.
Hopefully the solution was already in 'hydra-eval-guile-jobs' script
from Hydra, so I have used the same technique.

Currently the build phase is failing with the following error:

--8<---------------cut here---------------start------------->8---
Starting download of /gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz
From http://www.zsh.org/pub/zsh-5.1.1.tar.gz...
ERROR: In procedure connect*: Network is unreachable

Starting download of /gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz
From http://www.zsh.org/pub/old/zsh-5.1.1.tar.gz...
ERROR: In procedure connect*: Network is unreachable

Starting download of /gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz
From http://tarballs.nixos.org/sha256/11shllzhq53fg8ngy3bgbmpf09fn2czifg7hsb41nxi3410mpvcl...
ERROR: download failed "http://tarballs.nixos.org/sha256/11shllzhq53fg8ngy3bgbmpf09fn2czifg7hsb41nxi3410mpvcl" 404 "Not Found"
failed to download "/gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz" from ("http://www.zsh.org/pub/zsh-5.1.1.tar.gz" "http://www.zsh.org/pub/old/zsh-5.1.1.tar.gz")
builder for `/gnu/store/srsm569hhqiz56abp90dksknn6mqb03j-zsh-5.1.1.tar.gz.drv' failed to produce output path `/gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz'
@ build-failed /gnu/store/srsm569hhqiz56abp90dksknn6mqb03j-zsh-5.1.1.tar.gz.drv - 1 builder for `/gnu/store/srsm569hhqiz56abp90dksknn6mqb03j-zsh-5.1.1.tar.gz.drv' failed to produce output path `/gnu/store/y17blrd1x80nqik4ss4djbm92a5i5ivl-zsh-5.1.1.tar.gz'
cannot build derivation `/gnu/store/ri86p3f5w84r6pxyz54icyb0r00lmqaj-zsh-5.1.1.drv': 1 dependencies couldn't be built
Backtrace:
In ice-9/boot-9.scm:
 157: 9 [catch #t #<catch-closure 1ffeba0> ...]
In unknown file:
   ?: 8 [apply-smob/1 #<catch-closure 1ffeba0>]
In ice-9/boot-9.scm:
  63: 7 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 6 [eval # #]
In ice-9/boot-9.scm:
  63: 5 [call-with-prompt break486 ...]
  63: 4 [call-with-prompt continue487 ...]
In ice-9/eval.scm:
 432: 3 [eval # #]
In ice-9/r4rs.scm:
  90: 2 [dynamic-wind #<procedure 4c92620 at ice-9/boot-9.scm:1133:2 _> ...]
In ice-9/boot-9.scm:
 702: 1 [map #<procedure 3b489120 at ice-9/eval.scm:416:20 (a)> (# # # # ...)]
In guix/store.scm:
 684: 0 [build # # 0]

guix/store.scm:684:15: In procedure build:
guix/store.scm:684:15: Throw to key `srfi-34' with args `(#<condition &nix-protocol-error [message: "build of `/gnu/store/ri86p3f5w84r6pxyz54icyb0r00lmqaj-zsh-5.1.1.drv' failed" status: 1] 3b50d5a0>)'.
--8<---------------cut here---------------end--------------->8---

which is probably related to the fact that https is not handled.

The next step will be to fix this issue, to provide a better abstraction
for the 'build jobs', and to find a way to make some test scripts using
dummy jobs.

For those willing to see more, a Git repository is available here:

  https://notabug.org/mthl/cuirass

Everyone is of course welcome to provide any feedback.

Thanks.

-- 
Mathieu Lirzin