From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GLVfLUncMl/DWAAA0tVLHw (envelope-from ) for ; Tue, 11 Aug 2020 17:58:33 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 6IS2J0ncMl/mEgAA1q6Kng (envelope-from ) for ; Tue, 11 Aug 2020 17:58:33 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2F5549403EC for ; Tue, 11 Aug 2020 17:58:33 +0000 (UTC) Received: from localhost ([::1]:59630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k5YXg-0000FD-6B for larch@yhetil.org; Tue, 11 Aug 2020 13:58:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k5YXR-0000F0-F5 for guix-devel@gnu.org; Tue, 11 Aug 2020 13:58:17 -0400 Received: from relay11.mail.gandi.net ([217.70.178.231]:54973) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k5YXP-0002rl-Fv for guix-devel@gnu.org; Tue, 11 Aug 2020 13:58:17 -0400 Received: from bababa (lfbn-idf2-1-572-13.w86-246.abo.wanadoo.fr [86.246.37.13]) (Authenticated sender: mail@ambrevar.xyz) by relay11.mail.gandi.net (Postfix) with ESMTPSA id 1233C100004; Tue, 11 Aug 2020 17:58:11 +0000 (UTC) From: Pierre Neidhardt To: Ricardo Wurmus Subject: Re: File search progress: database review and question on triggers In-Reply-To: <87364tgja3.fsf@ambrevar.xyz> References: <87sgcuh8rb.fsf@ambrevar.xyz> <87y2ml429i.fsf@elephly.net> <87364tgja3.fsf@ambrevar.xyz> Date: Tue, 11 Aug 2020 19:58:11 +0200 Message-ID: <87y2mlf4jw.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=217.70.178.231; envelope-from=mail@ambrevar.xyz; helo=relay11.mail.gandi.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/11 13:58:12 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, FROM_SUSPICIOUS_NTLD=1, PDS_OTHER_BAD_TLD=1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -3.11 X-TUID: OU0fh9MZDBf/ --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Pierre Neidhardt writes: > Ricardo Wurmus writes: > >> I=E2=80=99m not suggesting to use updatedb, but I think it can be instru= ctive to >> look at how the file database is implemented there. We don=E2=80=99t ha= ve to >> use SQlite if it is much slower and heavier than a custom inverted >> index. > > Good call, I'll benchmark against an inverted index. > > Some cost may also be induced by the Guix store queries, not sure if we > can optimize these. With an s-exp based file, or a trivial text-based format, the downside is that it needs a bit of extra work to only load select entries, e.g. just the entries matching a specific Guix version. Would you happen to know a serialization library that allows for loading only a select portion of a file? Otherwise a trivial workaround would be to persist one index file per Guix generation. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl8y3DMACgkQm9z0l6S7 zH+UKwgAl2aIOE2qV0bopP7U6JEYCC4oMOThnpEGfnFhNOAWvqi/Q8W/sAAHMCyN PbZdAxW0xTHUpRTevhqxVhY0+1wFf5mUOKknw2GUhdxK2ZCQBvruE04lo6Y1DHiX oZqjPbWVbH9Zs2pSCvCc1kismFJUavjy7hmo+sXEY7kfqf+aVlw9yP6xmyxYXKzr fE0WJJp7gdGd6z2qMM/6XvK7tmm4V2gUm5Ob/sNmQdVGz/EVJ37eKSnFUxYfsbEm IUQQiXxfnc2p0eZ9aT0P1zp8XN7S3TxOG8jvF3utOkU93m5sGQOq7gJV+X/YROHd 3On8KLuh48jE3350kxaLWLhIEW/Ruw== =gjvY -----END PGP SIGNATURE----- --=-=-=--