From: ludo@gnu.org (Ludovic Courtès)
To: "Clément Lassieur" <clement@lassieur.org>
Cc: 32234@debbugs.gnu.org
Subject: bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler
Date: Mon, 23 Jul 2018 11:18:43 +0200 [thread overview]
Message-ID: <87bmaytimk.fsf@gnu.org> (raw)
In-Reply-To: <87k1ponc62.fsf@lassieur.org> ("Clément Lassieur"'s message of "Sat, 21 Jul 2018 11:57:57 +0200")
Hi,
Clément Lassieur <clement@lassieur.org> skribis:
> I'm trying to understand why Berlin web API times out[1].
Just to be clear, it does block on things involving non-trivial database
queries:
--8<---------------cut here---------------start------------->8---
ludo@berlin ~$ time wget -q -O - http://localhost:8081/specifications
[{"name" : "core-updates-core-updates","load-path-inputs" : ["core-updates"],"package-path-inputs" : [],"proc-input" : "core-updates","proc-file" : "build-aux/cuirass/gnu-system.scm","proc" : "cuirass-jobs","proc-args" : {"subset" : "core","systems" : ["x86_64-linux", "i686-linux"]},"inputs" : [{"name" : "core-updates","url" : "https://git.savannah.gnu.org/git/guix.git","load-path" : ".","branch" : "core-updates","tag" : false,"commit" : false,"no-compile?" : true}]}, {"name" : "staging-staging","load-path-inputs" : ["staging"],"package-path-inputs" : [],"proc-input" : "staging","proc-file" : "build-aux/cuirass/gnu-system.scm","proc" : "cuirass-jobs","proc-args" : {"systems" : ["x86_64-linux", "i686-linux"]},"inputs" : [{"name" : "staging","url" : "https://git.savannah.gnu.org/git/guix.git","load-path" : ".","branch" : "staging","tag" : false,"commit" : false,"no-compile?" : true}]}, {"name" : "guix-modular-master","load-path-inputs" : ["guix-modular"],"package-path-inputs" : [],"proc-input" : "guix-modular","proc-file" : "build-aux/cuirass/guix-modular.scm","proc" : "cuirass-jobs","proc-args" : {"systems" : ["x86_64-linux", "i686-linux"]},"inputs" : [{"name" : "guix-modular","url" : "https://git.savannah.gnu.org/git/guix.git","load-path" : ".","branch" : "master","tag" : false,"commit" : false,"no-compile?" : true}]}, {"name" : "guix-master","load-path-inputs" : ["guix"],"package-path-inputs" : [],"proc-input" : "guix","proc-file" : "build-aux/cuirass/gnu-system.scm","proc" : "cuirass-jobs","proc-args" : {"subset" : "all","systems" : ["x86_64-linux", "i686-linux"]},"inputs" : [{"name" : "guix","url" : "https://git.savannah.gnu.org/git/guix.git","load-path" : ".","branch" : "master","tag" : false,"commit" : false,"no-compile?" : true}]}]
real 0m0.007s
user 0m0.004s
sys 0m0.000s
ludo@berlin ~$ time wget -q -O - http://localhost:8081/api/latestbuilds?nr=2
[{"id" : 4484197,"jobset" : "core-updates-core-updates","job" : "ghc-alex-3.2.3.i686-linux","timestamp" : 1532336715,"starttime" : 1532336542,"stoptime" : 1532336715,"derivation" : "/gnu/store/b072sfmw44ww9dsm1ahpqafjk05zsl04-ghc-alex-3.2.3.drv","buildoutputs" : {"out" : {"path" : "/gnu/store/qk79zl1gcg6lh4k2nrp72x8iywy317d2-ghc-alex-3.2.3"}},"system" : "i686-linux","nixname" : "ghc-alex-3.2.3","buildstatus" : 0,"busy" : 0,"priority" : 0,"finished" : 1,"buildproducts" : null,"releasename" : null,"buildinputs_builds" : null}, {"id" : 4799156,"jobset" : "core-updates-core-updates","job" : "ghc-alex-3.2.3.i686-linux","timestamp" : 1532336715,"starttime" : 1532336542,"stoptime" : 1532336715,"derivation" : "/gnu/store/b072sfmw44ww9dsm1ahpqafjk05zsl04-ghc-alex-3.2.3.drv","buildoutputs" : {"out" : {"path" : "/gnu/store/qk79zl1gcg6lh4k2nrp72x8iywy317d2-ghc-alex-3.2.3"}},"system" : "i686-linux","nixname" : "ghc-alex-3.2.3","buildstatus" : 0,"busy" : 0,"priority" : 0,"finished" : 1,"buildproducts" : null,"releasename" : null,"buildinputs_builds" : null}]
real 0m28.033s
user 0m0.004s
sys 0m0.000s
--8<---------------cut here---------------end--------------->8---
I’m not sure it’s due to locking though; we may lack an index or two,
“explain query plan” should help.
> I think the SQlite built in busy handler may block the Fibers scheduler.
> We use "PRAGMA busy_timeout = 30000;", which is an alternative to
> calling sqlite3_busy_timeout(), whose description[2] is:
>
> This routine sets a busy handler that sleeps for a specified amount
> of time when a table is locked. The handler will sleep multiple
> times until at least "ms" milliseconds of sleeping have
> accumulated. After at least "ms" milliseconds of sleeping, the
> handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY.
>
> To me this sounds like non-cooperative and non-resumable code.
Indeed!
> A solution would be to set a custom handler through
> sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would
> let the scheduler schedule other fibers instead of just sleeping, using
> Fibers 'sleep' procedure[4].
AIUI the handler is called from C, and thus it’s a non-resumable
continuation, so this wouldn’t work.
Perhaps instead we need to set the timeout to a small value and handle
SQLITE_BUSY at the call site in our code. We could define a macro that
automatically retries upon SQLITE_BUSY.
Thoughts?
Ludo’.
next prev parent reply other threads:[~2018-07-23 9:19 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-21 9:57 bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler Clément Lassieur
2018-07-23 9:18 ` Ludovic Courtès [this message]
2018-07-23 13:42 ` Clément Lassieur
2018-07-23 14:57 ` Ludovic Courtès
2018-07-23 16:18 ` Clément Lassieur
2018-07-23 20:11 ` Ludovic Courtès
2018-08-06 19:27 ` bug#32234: [PATCH 1/2] utils: Avoid deadlock when WITH-CRITICAL-SECTION calls are nested Clément Lassieur
2018-08-06 19:27 ` bug#32234: [PATCH 2/2] database: Serialize all database accesses in a thread Clément Lassieur
2018-08-06 19:35 ` Clément Lassieur
2018-08-19 14:06 ` Ludovic Courtès
2018-08-26 14:07 ` Clément Lassieur
2018-08-26 21:16 ` Ludovic Courtès
2018-08-27 13:47 ` Clément Lassieur
2018-08-27 12:41 ` Danny Milosavljevic
2018-08-27 13:18 ` Clément Lassieur
2018-08-27 14:23 ` Danny Milosavljevic
2018-08-27 15:05 ` Clément Lassieur
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87bmaytimk.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=32234@debbugs.gnu.org \
--cc=clement@lassieur.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).