bug#30618: Cuirass dies with locked database

[ludo@gnu.org (Ludovic Courtès)]

Hello,

Andreas Enge <andreas@enge.fr> skribis:

> one of my latest cuirass runs ended in this:
>
> ...
> In ice-9/boot-9.scm:
>     705:2 12 (call-with-prompt _ _ #<procedure 191f060 at ice-9/eval…>)
>     705:2 11 (call-with-prompt _ _ #<procedure 19233e0 at ice-9/eval…>)
> In ice-9/eval.scm:
>     619:8 10 (_ #(#(#<directory (guile-user) 131d140> #<variable…> …)))
> In srfi/srfi-1.scm:
>     640:9  9 (for-each #<procedure process (spec)> (((#:name . #) …)))
> In cuirass/base.scm:
>    576:15  8 (process ((#:name . "guix") (#:url . "https://git.s…") …))
> In cuirass/database.scm:
>     68:18  7 (%sqlite-exec _ _ . _)
> In ice-9/eval.scm:
>    293:34  6 (_ #(#(#<directory (sqlite3) 1358820> #<<sqlite-stm…> …)))
>     619:8  5 (_ #(#(#(#(#<directory (sqlite3) 1358820> #<…>) #) 5) 5))
> In ice-9/boot-9.scm:
>    751:25  4 (dispatch-exception 0 sqlite-error (#f 5 "database is…"))
> In cuirass/utils.scm:
>     115:8  3 (_ _ #f 5 "database is locked")
> In ice-9/boot-9.scm:
>     829:9  2 (catch #t #<procedure 91c0f00 at cuirass/utils.scm:115…> …)
> In cuirass/utils.scm:
>    116:22  1 (_)
> In unknown file:
>            0 (make-stack #t)
> ERROR: In procedure make-stack:
> Throw to key `sqlite-error' with args `(#f 5 "database is locked")'.

I’ve noticed this occasionally on berlin.  I believe this has to do with
concurrent accesses to the database, which can happen if, for instance,
the evaluation fiber adds lots of builds to the database while at the
same time there are lots of builds completed, and thus
‘db-update-build-status!’ calls.

I’m not sure what’s the right way to handle this.  Should we just catch
the exception and retry?  Is there something smarter to do?  WDYT Danny?

Thanks,
Ludo’.