bug#30618: Cuirass dies with locked database

bug#30618: Cuirass dies with locked database

[Andreas Enge]

Hello,

one of my latest cuirass runs ended in this:

...
In ice-9/boot-9.scm:
    705:2 12 (call-with-prompt _ _ #<procedure 191f060 at ice-9/eval…>)
    705:2 11 (call-with-prompt _ _ #<procedure 19233e0 at ice-9/eval…>)
In ice-9/eval.scm:
    619:8 10 (_ #(#(#<directory (guile-user) 131d140> #<variable…> …)))
In srfi/srfi-1.scm:
    640:9  9 (for-each #<procedure process (spec)> (((#:name . #) …)))
In cuirass/base.scm:
   576:15  8 (process ((#:name . "guix") (#:url . "https://git.s…") …))
In cuirass/database.scm:
    68:18  7 (%sqlite-exec _ _ . _)
In ice-9/eval.scm:
   293:34  6 (_ #(#(#<directory (sqlite3) 1358820> #<<sqlite-stm…> …)))
    619:8  5 (_ #(#(#(#(#<directory (sqlite3) 1358820> #<…>) #) 5) 5))
In ice-9/boot-9.scm:
   751:25  4 (dispatch-exception 0 sqlite-error (#f 5 "database is…"))
In cuirass/utils.scm:
    115:8  3 (_ _ #f 5 "database is locked")
In ice-9/boot-9.scm:
    829:9  2 (catch #t #<procedure 91c0f00 at cuirass/utils.scm:115…> …)
In cuirass/utils.scm:
   116:22  1 (_)
In unknown file:
           0 (make-stack #t)
ERROR: In procedure make-stack:
Throw to key `sqlite-error' with args `(#f 5 "database is locked")'.

This is probably not dramatic when running it as a service, since then
it will be respawned some time later, but it should be caught in the code.

Andreas

bug#30618: Cuirass dies with locked database

[Ludovic Courtès]

Hello,

Andreas Enge <andreas@enge.fr> skribis:

> one of my latest cuirass runs ended in this:
>
> ...
> In ice-9/boot-9.scm:
>     705:2 12 (call-with-prompt _ _ #<procedure 191f060 at ice-9/eval…>)
>     705:2 11 (call-with-prompt _ _ #<procedure 19233e0 at ice-9/eval…>)
> In ice-9/eval.scm:
>     619:8 10 (_ #(#(#<directory (guile-user) 131d140> #<variable…> …)))
> In srfi/srfi-1.scm:
>     640:9  9 (for-each #<procedure process (spec)> (((#:name . #) …)))
> In cuirass/base.scm:
>    576:15  8 (process ((#:name . "guix") (#:url . "https://git.s…") …))
> In cuirass/database.scm:
>     68:18  7 (%sqlite-exec _ _ . _)
> In ice-9/eval.scm:
>    293:34  6 (_ #(#(#<directory (sqlite3) 1358820> #<<sqlite-stm…> …)))
>     619:8  5 (_ #(#(#(#(#<directory (sqlite3) 1358820> #<…>) #) 5) 5))
> In ice-9/boot-9.scm:
>    751:25  4 (dispatch-exception 0 sqlite-error (#f 5 "database is…"))
> In cuirass/utils.scm:
>     115:8  3 (_ _ #f 5 "database is locked")
> In ice-9/boot-9.scm:
>     829:9  2 (catch #t #<procedure 91c0f00 at cuirass/utils.scm:115…> …)
> In cuirass/utils.scm:
>    116:22  1 (_)
> In unknown file:
>            0 (make-stack #t)
> ERROR: In procedure make-stack:
> Throw to key `sqlite-error' with args `(#f 5 "database is locked")'.

I’ve noticed this occasionally on berlin.  I believe this has to do with
concurrent accesses to the database, which can happen if, for instance,
the evaluation fiber adds lots of builds to the database while at the
same time there are lots of builds completed, and thus
‘db-update-build-status!’ calls.

I’m not sure what’s the right way to handle this.  Should we just catch
the exception and retry?  Is there something smarter to do?  WDYT Danny?

Thanks,
Ludo’.

bug#30618: Cuirass dies with locked database

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> one of my latest cuirass runs ended in this:
>
> ...
> In ice-9/boot-9.scm:
>     705:2 12 (call-with-prompt _ _ #<procedure 191f060 at ice-9/eval…>)
>     705:2 11 (call-with-prompt _ _ #<procedure 19233e0 at ice-9/eval…>)
> In ice-9/eval.scm:
>     619:8 10 (_ #(#(#<directory (guile-user) 131d140> #<variable…> …)))
> In srfi/srfi-1.scm:
>     640:9  9 (for-each #<procedure process (spec)> (((#:name . #) …)))
> In cuirass/base.scm:
>    576:15  8 (process ((#:name . "guix") (#:url . "https://git.s…") …))
> In cuirass/database.scm:
>     68:18  7 (%sqlite-exec _ _ . _)
> In ice-9/eval.scm:
>    293:34  6 (_ #(#(#<directory (sqlite3) 1358820> #<<sqlite-stm…> …)))
>     619:8  5 (_ #(#(#(#(#<directory (sqlite3) 1358820> #<…>) #) 5) 5))
> In ice-9/boot-9.scm:
>    751:25  4 (dispatch-exception 0 sqlite-error (#f 5 "database is…"))
> In cuirass/utils.scm:
>     115:8  3 (_ _ #f 5 "database is locked")
> In ice-9/boot-9.scm:
>     829:9  2 (catch #t #<procedure 91c0f00 at cuirass/utils.scm:115…> …)
> In cuirass/utils.scm:
>    116:22  1 (_)
> In unknown file:
>            0 (make-stack #t)
> ERROR: In procedure make-stack:
> Throw to key `sqlite-error' with args `(#f 5 "database is locked")'.

I believe this is fixed by Cuirass commit
aa4c7784940581b5e271b9c7c4ac80b6ee1ee309, available in
cuirass@0.0.1-13.aa4c778 in Guix.

Ludo’.