From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:55459) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1YQF-0002mM-Fo for guix-patches@gnu.org; Tue, 11 Feb 2020 11:30:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1YQE-0002iO-Ci for guix-patches@gnu.org; Tue, 11 Feb 2020 11:30:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:51439) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j1YQE-0002hs-8l for guix-patches@gnu.org; Tue, 11 Feb 2020 11:30:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j1YQE-0004SK-6I for guix-patches@gnu.org; Tue, 11 Feb 2020 11:30:02 -0500 Subject: [bug#39258] Faster guix search using an sqlite cache Resent-Message-ID: From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: Date: Tue, 11 Feb 2020 17:29:30 +0100 In-Reply-To: (Arun Isaac's message of "Thu, 06 Feb 2020 07:28:22 +0530") Message-ID: <8736bhytn9.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Arun Isaac Cc: 39258@debbugs.gnu.org, zimoun Hello Arun! Arun Isaac skribis: > From 4c883fcff1f44339b28df6ccdb2b10c906439e3d Mon Sep 17 00:00:00 2001 > From: Arun Isaac > Date: Tue, 21 Jan 2020 20:45:43 +0530 > Subject: [PATCH] fast search [...] > --- a/gnu/packages.scm > +++ b/gnu/packages.scm > @@ -43,6 +43,7 @@ > #:use-module (srfi srfi-34) > #:use-module (srfi srfi-35) > #:use-module (srfi srfi-39) > + #:use-module (sqlite3) > #:export (search-patch > search-patches > search-auxiliary-file > @@ -204,10 +205,8 @@ PROC is called along these lines: > PROC can use #:allow-other-keys to ignore the bits it's not interested i= n. > When a package cache is available, this procedure does not actually load= any > package module." > - (define cache > - (load-package-cache (current-profile))) > - > - (if (and cache (cache-is-authoritative?)) > + (if (and (cache-is-authoritative?) > + (current-profile)) > (vhash-fold (lambda (name vector result) > (match vector > (#(name version module symbol outputs > @@ -220,7 +219,7 @@ package module." > #:supported? supported? > #:deprecated? deprecated?)))) > init > - cache) > + (cache-lookup (current-profile))) > (fold-packages (lambda (package result) > (proc (package-name package) > (package-version package) > @@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twi= ce." >=20=20 > (define %package-cache-file > ;; Location of the package cache. > - "/lib/guix/package.cache") > - > -(define load-package-cache [...] > +(define* (cache-lookup profile #:optional name) > "Lookup package NAME in CACHE. Return a list sorted in increasing ver= sion > order." > (define (package-version (version>? (vector-ref v2 1) (vector-ref v1 1))) >=20=20 > - (sort (vhash-fold* cons '() name cache) > - package-version + (define (int->boolean n) > + (case n > + ((0) #f) > + ((1) #t))) > + > + (define (string->list str) > + (call-with-input-string str read)) > + > + (define select-statement > + (string-append > + "SELECT name, version, module, symbol, outputs, supported, supersed= ed, locationFile, locationLine, locationColumn from packages" > + (if name " WHERE name =3D :name" ""))) I would rather keep the current package cache as-is instead of inserting sqlite in here. I don=E2=80=99t expect it to bring much compared performance-wise to the current simple cache (especially if we look at load time), and it does increase complexity quite a bit. However, using sqlite for keyword search as you initially proposed on guix-devel does sound like a great idea to me. WDYT? Thanks, Ludo=E2=80=99.