From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:37444) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwwrP-0001fw-E4 for guix-patches@gnu.org; Wed, 29 Jan 2020 18:35:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwwrO-0004yN-77 for guix-patches@gnu.org; Wed, 29 Jan 2020 18:35:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:56969) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iwwrO-0004xz-41 for guix-patches@gnu.org; Wed, 29 Jan 2020 18:35:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1iwwrN-0003Zt-Vo for guix-patches@gnu.org; Wed, 29 Jan 2020 18:35:01 -0500 Subject: [bug#39258] Faster guix search using an sqlite cache Resent-Message-ID: MIME-Version: 1.0 References: In-Reply-To: From: zimoun Date: Thu, 30 Jan 2020 00:33:56 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Arun Isaac Cc: 39258@debbugs.gnu.org Hi Arun, Thank you for the patch! Cool! :-) I have not tested it yet. Sorry. On Thu, 23 Jan 2020 at 20:53, Arun Isaac wrote: > At the moment, I am having some difficulty populating the sqlite > database. generate-package-cache populates the database correctly when > invoked from a normal guile REPL using geiser, but fails to do so when > run by the guix daemon during guix pull. [...] > On examining package-cache.sqlite, I find that no records have been > written. And, there is a lingering journal file that shouldn't be > there. For some reason, populating the sqlite database does not work > with guix pull. sqlite probably crashes and leaves the journal file. Hum? weird... Is it possible that a module is loaded when Guile repl is used and not with Guix pull? What about "guix repl"? > If I try to populate the database with each package record being > inserted in its own transaction, at least some of the insertions You mean 'commit' the database after each insertion, right? > work. But the journal file still lingers. My unverified guess is that > everything except the last transaction was successful. And this does not happen with the repl, right? > Any ideas what's going on? I have no idea. Weird. What about adding 'last-insert-row-id' from 'guix/store/database.scm'? I mean without really understanding and just grepping 'sqlite-finalize' spots that 'last-insert-row-id' seems often around. :-) > Also, inserting each package in its own transaction is ridiculously slow > and so that is out of the question. See https://www.sqlite.org/faq.html#q19 Agree that it is not an option. :-) Otherwise, 'list->string' is defined twice. And the first one is not necessary, I guess. The docstring of 'cache-lookup' is not coherent anymore. :-) Cheers, simon