From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [GSoC] Development of Cuirass. Date: Mon, 13 Mar 2017 09:49:53 +0100 Message-ID: <87inndblxq.fsf@gnu.org> References: <87tw6yim7o.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54904) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cnLg9-0002Nv-Uz for guix-devel@gnu.org; Mon, 13 Mar 2017 04:50:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cnLg3-0001OK-RT for guix-devel@gnu.org; Mon, 13 Mar 2017 04:50:09 -0400 In-Reply-To: <87tw6yim7o.fsf@gnu.org> (Mathieu Lirzin's message of "Sun, 12 Mar 2017 15:49:47 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Mathieu Lirzin Cc: guix-devel@gnu.org Hello Mathieu! Mathieu Lirzin skribis: > The project is the continuation of last year work, and is separated in > two parts. The first part is to improve the testing infrastructure. > The second one is about providing a more complete HTTP API. Sounds cool! > 2.1 The testing infrastructure > =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 > > Currently Cuirass is providing a small test suite consisting only of > unit tests. The integration tests are done manually with basic > examples consisting in building small packages such as GNU Hello. > This is not good, because actual packages take a consequent time to > compile, and testing only basic examples leaves the interesting > complex case out of the scope of testing. The worst is that the > result depend on the phase of the moon since every commit in Guix can > potentially break the build. > > The idea is to take inspiration from the Guix test suite by providing > tiny mocked packages that will help having lightweight build > specifications. Those tests require launching another instance of the > `guix-daemon' with a local temporary store. Sounds like a good idea. It would be nice to refine what it is we=E2=80=99re trying to test and that= =E2=80=99s not addressed by unit tests. At one end of the spectrum, I imagine we could be spawning a couple of GuixSD VMs connected together, one serving a Git repo, and the other one running Cuirass pulling from that repo (a system test). At the middle of the spectrum is something like you describe, I think: run Cuirass directly upon =E2=80=9Cmake check=E2=80=9D, though there are co= mplications: what repo do we pull from, do we really want to build packages and if not how to we mock builds, etc. There are several components we=E2=80=99d like to test more closely IMO: SCM polling, evaluation, and job scheduling. Perhaps some of that can also be achieved through unit tests? > 2.2 HTTP API > =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80 > > Since commit [12d71ee098c3eb84a9a5dc2419bd37a3b9e55dd2] Cuirass has a > web server which is running in a separate thread of the `cuirass' > process. However the resources it provides are pretty limited since > you can only query the lists of specifications currently watched. > This is done with the `/specifications' or `/jobsets' resource names. > "Jobsets" is the terminology used by Hydra and is used here for > compatibility with its HTTP API. For a start, I think we should implement all or a subset of the Hydra API, as-is (it=E2=80=99s currently used by Emacs-Guix): https://github.com/NixOS/hydra/blob/master/src/lib/Hydra/Controller/API.pm A large part of it is about monitoring the status of Hydra: queued builds, recently-completed builds, jobs, etc. That is the most pressing need, I think. > The principal objective of this part is to provide a more complete API > which allows users to query the build results and to add their > specifications to the current set. The design of this API will apply > the principles of the Representational State Transfert (REST) > architectural pattern[2]. Sensitive requests should be done with an > authentification mechanism which is not determined yet. I currently > have no experience with any and lack the knowledge to properly choose > one. pelzflorian already gave good ideas, so that=E2=80=99s good. :-) > This roadmap lacks details about the concrete tasks and its associated > timeline. However I prefer having some feedback about this proposal > first, before providing a complete roadmap. > > =E2=80=A2 From May 30 to June 26, I will work on the testing infrastruc= ture,. > =E2=80=A2 From June 27 to August 29, I will work on the HTTP API. You may find that you=E2=80=99ll want to interleave tasks. I don=E2=80=99t= know about you, but sometimes I get bored if I keep working on the same task or domain for too long, so switching to something else for a while can be a relief. :-) There are a couple of isolated tasks that come to mind, which do not really fit in the proposal but are worth looking into as time permits: =E2=80=A2 improved error reporting upon evaluation failures and so on; =E2=80=A2 improved error recovery; =E2=80=A2 use of Guile-Git instead of Git. > 4 Feedback > =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90=E2=95=90 > > Some people have expressed their interest in having a Web /frontend/ > for Cuirass. To avoid possible confusions I have no plan in working > on it this summer. While this would be a very valuable effort, I > don't feel up to the task. Understood. That said, once the JSON API is in place, Web-savvy people could certainly come with a JSish frontend. It might be that would work. Thanks for posting this proposal! Ludo=E2=80=99.