From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [GSoC] Continuous integration tool =?utf-8?Q?=C3=A0?= la Hydra. Date: Mon, 25 Jul 2016 23:36:16 +0200 Message-ID: <87lh0pe6y7.fsf@gnu.org> References: <871t4ksk0n.fsf@gnu.org> <87vazueezx.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50071) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bRnY4-0007CX-02 for guix-devel@gnu.org; Mon, 25 Jul 2016 17:36:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bRnY1-0002Dc-Pf for guix-devel@gnu.org; Mon, 25 Jul 2016 17:36:27 -0400 In-Reply-To: <87vazueezx.fsf@gnu.org> (Mathieu Lirzin's message of "Mon, 25 Jul 2016 02:30:10 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Mathieu Lirzin Cc: guix-devel Hello! Mathieu Lirzin skribis: > Since my second update I have first fixed a major bug. When building > different branches of Guix and evaluating package derivations the > results were always the same. The issue was that the evaluations were > happening in the same Guile process which does not play well with module > changes. To fix that I have used a separate process + pipe to get the > evaluation results. Sounds good. Using a separate process makes sure what=E2=80=99s evaluated doesn=E2=80=99t erroneously ends up using modules already loaded by the evaluator, and vice versa. > In the last update, I have introduced usage of SRFI-9 records for > specifications, jobs, and builds. While they are nice to organize data, > they have major drawbacks: > > - not flexible when you want to informally create a container. > - require serialization when passing them throught pipes. > > For those reasons I have switched to good ol' alists, which are flexible, > persistant, directly readable and don't require messing with load paths. > The downside is of course that there is no compile time checks when > manipulating data. OK. > As stated in my last update. I have been working on storing data in a > database. For that I have decided to use Guile-sqlite3. The principal > efforts have consist of using an external schema file and design it. > > I have come up with this, but this will likely evolve in the future: > > CREATE TABLE Specifications ( > id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, > repo_name TEXT NOT NULL, > url TEXT NOT NULL, > load_path TEXT NOT NULL, > file TEXT NOT NULL, > proc TEXT NOT NULL, > arguments TEXT NOT NULL, > -- The following columns are optional. > branch TEXT, > tag TEXT, > revision TEXT > ); > > CREATE TABLE Evaluations ( > derivation TEXT NOT NULL PRIMARY KEY, > job_name TEXT NOT NULL, > specification INTEGER NOT NULL, > FOREIGN KEY (specification) REFERENCES Specifications (id) > ); An evaluation leads to several derivations (for Guix, roughly one derivation per package and per system type), but the table above seems to suggest that each evaluation is mapped to only one derivation? > CREATE TABLE Builds ( > id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, > derivation TEXT NOT NULL, > log TEXT NOT NULL, > output TEXT, -- NULL if build failed > FOREIGN KEY (derivation) REFERENCES Evaluations (derivation) > ); > > The next step will be to continue improving the database communication, > and start looking how to implement the HTTP API. Cool! It seems that the database already has or is about to have the info we typically look for at : which evaluations were performed, what commit (specification) they correspond to, which derivations they correspond to and whether they succeeded/failed/aborted. If the HTTP API exposes this info, possibly using the same API as Hydra (see guix-hydra-jobset.el), that will cover an important part of our daily needs. > For those willing to follow my work, a Git repository is available here: > > https://notabug.org/mthl/cuirass =E2=80=A6 and the README has instructions on how to run it. If anyone has = spare CPU cycles, run Cuirass, =E2=80=98guix publish=E2=80=99, and share. :-) I=E2=80=99m happy with the progress that=E2=80=99s been made, thank you! Ludo=E2=80=99.