From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Cl=C3=A9ment?= Lassieur Subject: bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler Date: Mon, 23 Jul 2018 18:18:14 +0200 Message-ID: <87r2jusz7d.fsf@lassieur.org> References: <87k1ponc62.fsf@lassieur.org> <87bmaytimk.fsf@gnu.org> <87tvoqt6f3.fsf@lassieur.org> <87y3e2ngnm.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:44411) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fhdY9-0000Er-Kp for bug-guix@gnu.org; Mon, 23 Jul 2018 12:19:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fhdY6-0001y6-GY for bug-guix@gnu.org; Mon, 23 Jul 2018 12:19:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:49252) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fhdY6-0001y2-Cb for bug-guix@gnu.org; Mon, 23 Jul 2018 12:19:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fhdY6-0007xM-62 for bug-guix@gnu.org; Mon, 23 Jul 2018 12:19:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-reply-to: <87y3e2ngnm.fsf@gnu.org> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 32234@debbugs.gnu.org Ludovic Court=C3=A8s writes: > Cl=C3=A9ment Lassieur skribis: > >> Ludovic Court=C3=A8s writes: > > [...] > >>> Perhaps instead we need to set the timeout to a small value and handle >>> SQLITE_BUSY at the call site in our code. We could define a macro that >>> automatically retries upon SQLITE_BUSY. >> >> That would limit the issue to the first timeout span: for that short >> time the scheduler would be blocked. I think a timeout of 0 would be >> better. > > Yes, 0 is an acceptable =E2=80=9Csmall value.=E2=80=9D ;-) Perhaps 100m= s would be > acceptable if the situation is rare enough, dunno. > >> Another solution would be to serialize all the database accesses as we >> do already with the url handler, and stop using the SQLITE >> multithreading features. It would probably make the code simpler >> because we would use the same paradigm everywhere, and we would avoid >> looping until SQLITE isn't busy at each request. > > In essence we=E2=80=99d introduce a =E2=80=9Cdatabase server=E2=80=9D run= ning as a fiber, and > everyone would talk to that server. Why not as a thread? There would be only one thread dedicated to SQLITE (thus adding a constant overhead), and that would prevent the Scheduler from being bloqued by long SQLite queries. > I considered doing that before but then though sqlite would probably be > able to do better than this, but I don=E2=80=99t know. And it's hard to test... > What=E2=80=99s a bit annoying with switching to a database server model i= s that > we=E2=80=99d need to adapt every call site. > > Thoughts? I'm a tiny bit in favor of switching to a database server model because it's more consistent overall (we already use it for the url handler) and the scheduler wouldn't be bloqued by long SQLite queries. It's annoying to adapt every call site, but it would be done once and for all. :-) So... as you wish! Cl=C3=A9ment