From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:39468) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1aS3-0001KA-80 for guix-patches@gnu.org; Tue, 11 Feb 2020 13:40:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1aS2-0000lc-6I for guix-patches@gnu.org; Tue, 11 Feb 2020 13:40:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:51571) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j1aS2-0000lE-2g for guix-patches@gnu.org; Tue, 11 Feb 2020 13:40:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j1aS1-0001Ca-V9 for guix-patches@gnu.org; Tue, 11 Feb 2020 13:40:01 -0500 Subject: [bug#39258] Faster guix search using an sqlite cache Resent-Message-ID: From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <8736bhytn9.fsf@gnu.org> Date: Tue, 11 Feb 2020 19:39:19 +0100 In-Reply-To: (zimoun's message of "Tue, 11 Feb 2020 19:21:55 +0100") Message-ID: <87sgjhx92g.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: zimoun Cc: Arun Isaac , 39258@debbugs.gnu.org zimoun skribis: > On Tue, 11 Feb 2020 at 17:29, Ludovic Court=C3=A8s wrote: > >> I would rather keep the current package cache as-is instead of inserting >> sqlite in here. I don=E2=80=99t expect it to bring much compared >> performance-wise to the current simple cache (especially if we look at >> load time), and it does increase complexity quite a bit. > > Complexity is about taste. ;-) It=E2=80=99s measurable to some extent (lines of code, cyclomatic complexit= y, etc.) > About performance, the idea was to first implement something with > sqlite and then see if it makes the difference. I mean I have > understood that. Yes. But keep in mind that this package cache is used exclusively for package lookups by name. Namely, the goal is to speed package lookup in operations like =E2=80=9Cguix install foo=E2=80=9D (mapping =E2=80=9Cfoo=E2= =80=9D to the right in the right module without walking through all the modules) and =E2=80=9Cg= uix package -A=E2=80=9D (which is what the shell completion hooks use). Currently =E2=80=9Cguix package -A=E2=80=9D runs in .5s on my laptop, and I= suspect it=E2=80=99s going to be hard to do better just by touching the cache. >> However, using sqlite for keyword search as you initially proposed on >> guix-devel does sound like a great idea to me. > > If I understand correctly, you are proposing 2 caches, right? > Or are you proposing an inverted index (VHash/VList table) based on > trigrams to speed up the lookup? Arun started the discussion on guix-devel with the idea of an inverted index, and I thought this would become a second index (possibly implemented using SQLite). Perhaps I misunderstood the discussion all along though, let me know! :-) Thanks, Ludo=E2=80=99.