From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:33687) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j9teP-0004mn-9M for guix-patches@gnu.org; Thu, 05 Mar 2020 11:47:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j9teL-00038q-NP for guix-patches@gnu.org; Thu, 05 Mar 2020 11:47:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:37667) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j9teI-00030H-5s for guix-patches@gnu.org; Thu, 05 Mar 2020 11:47:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j9teI-00007f-3x for guix-patches@gnu.org; Thu, 05 Mar 2020 11:47:02 -0500 Subject: [bug#39258] [PATCH 0/4] Xapian for Guix package search Resent-Message-ID: From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20200227204150.30985-1-arunisaac@systemreboot.net> Date: Thu, 05 Mar 2020 17:46:01 +0100 In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> (Arun Isaac's message of "Fri, 28 Feb 2020 02:11:46 +0530") Message-ID: <87h7z292d2.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Arun Isaac Cc: mail@ambrevar.xyz, 39258@debbugs.gnu.org, zimon.toutoune@gmail.com Hello Arun, Arun Isaac skribis: > * Speed improvement > > Despite search-package-index in gnu/packages.scm taking only around 1.5ms= , I > see an overall speedup in `guix search` of only a factor of 2 -- from aro= und > 2s to around 1s. I wonder what else in `guix search` is taking up so much > time. Note that =E2=80=98guix search=E2=80=99 time is largely dominated by I/O. = On my laptop, I get (first measurement is cold cache, second one is warm cache): --8<---------------cut here---------------start------------->8--- $ sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' $ time guix search foo >/dev/null real 0m2.631s user 0m1.134s sys 0m0.124s $ time guix search foo >/dev/null real 0m0.836s user 0m1.027s sys 0m0.053s --8<---------------cut here---------------end--------------->8--- It=E2=80=99s hard to do better on the warm cache case because at this level, there may be other things to optimize having little to do with searching itself. Note that this is on an SSD; the cold-cache case must be worse on NFS or on a spinning disk, and there we could gain a lot. I think we should weigh the pros and cons on all these aspects: speed, complexity and maintenance cost, search result quality, search features, etc. Thanks, Ludo=E2=80=99. PS: I have not yet looked at the whole series as I=E2=80=99m just coming ba= ck to the keyboard. :-)