From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Cuirass news Date: Fri, 26 Jan 2018 12:12:15 +0100 Message-ID: <87wp043mcw.fsf@gnu.org> References: <877es6x5xj.fsf@gnu.org> <87efmdy7fb.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:51754) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ef4EA-0008QB-Dc for guix-devel@gnu.org; Fri, 26 Jan 2018 08:39:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ef4E7-0001u0-88 for guix-devel@gnu.org; Fri, 26 Jan 2018 08:39:34 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:50258) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ef4E7-0001tS-1Z for guix-devel@gnu.org; Fri, 26 Jan 2018 08:39:31 -0500 In-Reply-To: <87efmdy7fb.fsf@elephly.net> (Ricardo Wurmus's message of "Thu, 25 Jan 2018 22:06:48 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel Hello! Ricardo Wurmus skribis: >> =E2=80=A2 Logging is improved: useful events are logged, including bui= ld >> started/succeeded/failed (using a variant of what I proposed in the >> Guix =E2=80=98wip-ui=E2=80=99 branch). This makes it much easier to= understand >> what=E2=80=99s going on! > > Finally! Better logging alone would be a reason to celebrate :) > > IIRC the wip-ui branch parsed the =E2=80=9C@=E2=80=9D-prefixed messages o= f the daemon. > I didn=E2=80=99t find this in your commits to Cuirass, though. This is done in 1f701262e1a4a706a341b820796ba31954e1be11. It sometimes misparses these lines or misses some of them because of interleaving, but I think that=E2=80=99s fixable. >> =E2=80=A2 Restarting unfinished builds: it=E2=80=99s common, especiall= y when testing, >> to interrupt Cuirass, leaving a number of builds unfinished or not >> even started. Now Cuirass restarts those upon startup. > > Also very useful. Does this mean Cuirass resumes work more quickly now > whereas previously it would have to compute the full evaluation after a > restart? Yes, it restarts builds upfront without having to perform an evaluation. > I wonder about commit 49a341866afabe64c8ac3b8d93c64d2b6b20895d: you=E2=80= =99re > chunking the number of derivations because guix-daemon doesn=E2=80=99t pe= rform > well when it is asked to build lots of derivations at once. Is it > possible to parse, lock, and run individual derivations in the daemon > when presented with lots of them, or is there a good reason why each of > these phases is executed for all derivations? The daemon-side implementation of =E2=80=98build-derivations=E2=80=99 alway= s does that parse/lock/run/monitor sequence, not just when it is passed many derivations. It=E2=80=99s OK when =E2=80=98build-derivations=E2=80=99 is passed a reason= able number of derivations, because the parse/lock/run part will take a few seconds at most, so the monitor part (reading the stdout/stderr of the build processes started in the =E2=80=98run=E2=80=99 part) happens soon enough. The problem is when there are too many of them: first the parse/lock part takes a lot of time, and then the real problem is that the daemon spawns lots of build processes and starts reading their stdout/stderr long after. A proper fix would be for guix-daemon to perform these activities concurrently=E2=80=A6 but there=E2=80=99s no Fibers for C++. :-) >> And! This brings a whole set of new bugs that I=E2=80=99m hunting notab= ly on >> berlin (which may thus lag behind=E2=80=A6). > > I see that there are a bunch of spawn-fiber invocations with > =E2=80=9Cwith-database=E2=80=9D bodies. Maybe I remember this wrong, but= I thought > sqlite doesn=E2=80=99t support concurrent database access. I thought that too, but it seems to work, so I thought that maybe it=E2=80= =99s OK if those accesses come from the same process. We should definitely check, though! Ludo=E2=80=99.