Re: Cuirass news

[ludo@gnu.org (Ludovic Courtès)]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

>>   • Logging is improved: useful events are logged, including build
>>     started/succeeded/failed (using a variant of what I proposed in the
>>     Guix ‘wip-ui’ branch).  This makes it much easier to understand
>>     what’s going on!
>
> Finally!  Better logging alone would be a reason to celebrate :)
>
> IIRC the wip-ui branch parsed the “@”-prefixed messages of the daemon.
> I didn’t find this in your commits to Cuirass, though.

This is done in 1f701262e1a4a706a341b820796ba31954e1be11.

It sometimes misparses these lines or misses some of them because of
interleaving, but I think that’s fixable.

>>   • Restarting unfinished builds: it’s common, especially when testing,
>>     to interrupt Cuirass, leaving a number of builds unfinished or not
>>     even started.  Now Cuirass restarts those upon startup.
>
> Also very useful.  Does this mean Cuirass resumes work more quickly now
> whereas previously it would have to compute the full evaluation after a
> restart?

Yes, it restarts builds upfront without having to perform an evaluation.

> I wonder about commit 49a341866afabe64c8ac3b8d93c64d2b6b20895d: you’re
> chunking the number of derivations because guix-daemon doesn’t perform
> well when it is asked to build lots of derivations at once.  Is it
> possible to parse, lock, and run individual derivations in the daemon
> when presented with lots of them, or is there a good reason why each of
> these phases is executed for all derivations?

The daemon-side implementation of ‘build-derivations’ always does that
parse/lock/run/monitor sequence, not just when it is passed many
derivations.

It’s OK when ‘build-derivations’ is passed a reasonable number of
derivations, because the parse/lock/run part will take a few seconds at
most, so the monitor part (reading the stdout/stderr of the build
processes started in the ‘run’ part) happens soon enough.

The problem is when there are too many of them: first the parse/lock
part takes a lot of time, and then the real problem is that the daemon
spawns lots of build processes and starts reading their stdout/stderr
long after.

A proper fix would be for guix-daemon to perform these activities
concurrently… but there’s no Fibers for C++.  :-)

>> And!  This brings a whole set of new bugs that I’m hunting notably on
>> berlin (which may thus lag behind…).
>
> I see that there are a bunch of spawn-fiber invocations with
> “with-database” bodies.  Maybe I remember this wrong, but I thought
> sqlite doesn’t support concurrent database access.

I thought that too, but it seems to work, so I thought that maybe it’s
OK if those accesses come from the same process.  We should definitely
check, though!

Ludo’.