From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id 4Cf2LmYTj2OCXwEAbAwnHQ (envelope-from ) for ; Tue, 06 Dec 2022 11:03:18 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id GKcILmYTj2MCewEAG6o9tA (envelope-from ) for ; Tue, 06 Dec 2022 11:03:18 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4E439B7FE for ; Tue, 6 Dec 2022 11:03:18 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2UmU-00032f-QO; Tue, 06 Dec 2022 05:02:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2UmE-0002wy-M8 for guix-devel@gnu.org; Tue, 06 Dec 2022 05:02:19 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Um8-0006Wk-D7; Tue, 06 Dec 2022 05:02:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=DMzhnfsj2HCoLzPnxZi8SMJ/n0sKcxe4QGCD57MUC2A=; b=TiC2svWVPuQQqfww/Aqo ilCFemiF4xRGFlFjOl18HdDHyWmgReANIhSGQ1xbqhGboH8Lp/BI2hf5b8A7Ax7eSspP2XjehMhX8 Uq+BhokAwIt/5i1Tx47FfvEaup5lR6ro8w+8yQY8pL0SJRVFCUub5Qukm0cKdnRQljtu73a1QwZ8C AB9M+xdTFi3itNxemykCfVu9qN6+VtZsNAmzraP0IAffSo80iqIYRcsw+B9rnC+AB93pPp3DM5FyS 3pm2+ekVT7/lgQalyiyLOiNyLfSaJxyg1KkjoAsuPAIiuGvhW+kBRAiFEb00zESmSYiil7rq7tIjc whZ2pOiOIpVnKg==; Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Um1-0004NL-Fk; Tue, 06 Dec 2022 05:02:03 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: "Antoine R. Dumont (@ardumont)" Cc: guix-devel@gnu.org Subject: Re: File search References: <87pmd1r8kt.fsf@gmail.com> <87mt85r7h8.fsf@gmail.com> <87lenonydz.fsf@gnu.org> <877cz7qg80.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 16 Frimaire an 231 de la =?utf-8?Q?R=C3=A9vo?= =?utf-8?Q?lution=2C?= jour de l'Ajonc X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 06 Dec 2022 11:01:58 +0100 In-Reply-To: <877cz7qg80.fsf@gmail.com> (Antoine R. Dumont's message of "Sun, 04 Dec 2022 17:35:43 +0100") Message-ID: <878rjkn949.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1670320998; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=DMzhnfsj2HCoLzPnxZi8SMJ/n0sKcxe4QGCD57MUC2A=; b=t0uTC9PI6beDtEj3D6ydlQkgO9b4ZJe4uusHzZVGcnl9rsYH41907dsjcYEnxxrvfhiDka n+2oO2XdWB7pFLtpRf988xlNcZfoKxxc1jAmZmT+UGqVxvWO3AMO4dkxzLxtz/hVOo9cvC 0wP8wf+3nZ7tx2tuzpiGoOUMIMgDR9rKnXJyv4L84LWkXgi6bZamVKw5AyY7wwpGY0ZWYC qOBZQKUPCHjAs/9tDkjHvLudy8s/6e1xoK315LRRCMZhQi6b8DdpoIgbwuALJdSvG/nwFi C8TpQpVW1BrwKb76wRDulO7fwSrtHY0KPF1rITm5W88VNQGCSQbd510idELZjw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1670320998; a=rsa-sha256; cv=none; b=jn+HJcztJix16UHWmSQE9ZjXdp98yfZMDFDegRnfD8uaAo4NJzOsmMcFEm0aGDjVko8Bxn 0LmUfVE1UQjmepP++FiALtaMMzNkILfNnk5YSq46uTFuiOaHwAbLqLHUNgt1FNnFtkhwjb EP5UlJYvFgkyATld1/tW4veWnTL0eFzwrAPcK/ZYrxKCjeneoyo7MHdZ45bLj0ofl+lklG cD90IOvxvYK68d1axbrztbD89g2q7v2kpBpAXs5inSJA+otcJQKvbCFY2wzkYTK+98u7oh O2jj2YIItsh/uEdOsGr8KrILvfmRKhjs1/X8mQPNOGulOuF2tgHc/3nzvD92MA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gnu.org header.s=fencepost-gnu-org header.b=TiC2svWV; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -8.96 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gnu.org header.s=fencepost-gnu-org header.b=TiC2svWV; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 4E439B7FE X-Spam-Score: -8.96 X-Migadu-Scanner: scn0.migadu.com X-TUID: 9v0AGO06lwON Howdy! "Antoine R. Dumont (@ardumont)" skribis: > Please, find enclosed the latest implementation as a patch (somewhat vcs > code ;). I've edited commits to mark Ludo as author with his > started/amended implementations first [0] (that should be in the patch). Nice! > For information, I extracted some number from runs to compare our > iterations (see the org-file attachment). The first iteration being > "extracts packages from the store" and the second one "extracts packages > from the system manifest". Those runs happened both on a guixified > debian host and a raw guix host (more packages). So we went from 413s to 11s (on the Guix System node) for only 6% fewer files in the latter case? Do I get that right? That=E2=80=99s pretty cool. The implementation based on manifests can of course miss packages, so it=E2=80=99s a tradeoff. Purely local indexing will only find packages you already have anyway, so eventually we=E2=80=99ll need a second mode that wo= uld download a database. BTW, I noticed outputs are not properly handled so far, as in this example: --8<---------------cut here---------------start------------->8--- $ GUIX_EXTENSIONS_PATH=3D$HOME/tmp/guix-index guix index search git-send-em= ail git@2.38.1 /gnu/store/g3lgyzr749l76qma7srycclgsm0f78iq-git-2.38.1= -send-email/libexec/git-core/git-send-email git@2.37.1 /gnu/store/n3hkzz5ydm0qm1c2jja2pwy2v19mq1k0-git-2.37.1= -send-email/libexec/git-core/git-send-email --8<---------------cut here---------------end--------------->8--- It should instead show =E2=80=9Cgit@2.38.1:send-email=E2=80=9D. We probabl= y need an =E2=80=98output=E2=80=99 field in the =E2=80=98Packages=E2=80=99 table. Also going forward we=E2=80=99ll need a schema version, as in: --8<---------------cut here---------------start------------->8--- create table SchemaVersion ( version integer not null; ); --8<---------------cut here---------------end--------------->8--- so that the tool can upgrade or discard databases that have the wrong version. Oh, and progress bars too. And a pony. :-) Thanks for your work! Ludo=E2=80=99.