From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Cl=C3=A9ment?= Lassieur Subject: bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler Date: Mon, 23 Jul 2018 15:42:24 +0200 Message-ID: <87tvoqt6f3.fsf@lassieur.org> References: <87k1ponc62.fsf@lassieur.org> <87bmaytimk.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:56362) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fhb7B-0005G1-9y for bug-guix@gnu.org; Mon, 23 Jul 2018 09:43:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fhb78-0003QE-3K for bug-guix@gnu.org; Mon, 23 Jul 2018 09:43:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:48209) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fhb77-0003Q2-UN for bug-guix@gnu.org; Mon, 23 Jul 2018 09:43:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fhb77-00043N-NV for bug-guix@gnu.org; Mon, 23 Jul 2018 09:43:01 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-reply-to: <87bmaytimk.fsf@gnu.org> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 32234@debbugs.gnu.org Ludovic Court=C3=A8s writes: >> I think the SQlite built in busy handler may block the Fibers scheduler. >> We use "PRAGMA busy_timeout =3D 30000;", which is an alternative to >> calling sqlite3_busy_timeout(), whose description[2] is: >> >> This routine sets a busy handler that sleeps for a specified amount >> of time when a table is locked. The handler will sleep multiple >> times until at least "ms" milliseconds of sleeping have >> accumulated. After at least "ms" milliseconds of sleeping, the >> handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY. >> >> To me this sounds like non-cooperative and non-resumable code. > > Indeed! > >> A solution would be to set a custom handler through >> sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would >> let the scheduler schedule other fibers instead of just sleeping, using >> Fibers 'sleep' procedure[4]. > > AIUI the handler is called from C, and thus it=E2=80=99s a non-resumable > continuation, so this wouldn=E2=80=99t work. Oh, I see. > Perhaps instead we need to set the timeout to a small value and handle > SQLITE_BUSY at the call site in our code. We could define a macro that > automatically retries upon SQLITE_BUSY. That would limit the issue to the first timeout span: for that short time the scheduler would be blocked. I think a timeout of 0 would be better. Another solution would be to serialize all the database accesses as we do already with the url handler, and stop using the SQLITE multithreading features. It would probably make the code simpler because we would use the same paradigm everywhere, and we would avoid looping until SQLITE isn't busy at each request. WDYT? Cl=C3=A9ment