From mboxrd@z Thu Jan 1 00:00:00 1970 From: zimoun Subject: Re: Inverted index to accelerate guix package search Date: Wed, 15 Jan 2020 15:49:48 +0100 Message-ID: References: <87a76r68u6.fsf@ambrevar.xyz> <87muaqnmod.fsf@ambrevar.xyz> <87h80y12k3.fsf@roquette.mug.biscuolo.net> <87wo9tylzw.fsf@roquette.mug.biscuolo.net> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:45764) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1irjze-0005RI-CV for guix-devel@gnu.org; Wed, 15 Jan 2020 09:50:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1irjzc-0004yY-Gi for guix-devel@gnu.org; Wed, 15 Jan 2020 09:50:02 -0500 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]:42631) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1irjzc-0004xy-95 for guix-devel@gnu.org; Wed, 15 Jan 2020 09:50:00 -0500 Received: by mail-qt1-x831.google.com with SMTP id j5so15940188qtq.9 for ; Wed, 15 Jan 2020 06:50:00 -0800 (PST) In-Reply-To: <87wo9tylzw.fsf@roquette.mug.biscuolo.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: Giovanni Biscuolo Cc: Guix Devel Hi Giovanni, On Wed, 15 Jan 2020 at 12:53, Giovanni Biscuolo wrote: > zimoun writes: > > If Guix goes to Xapian, all the Git history of all the packages should > > be indexed. Today, it is really hard to find the right commit > > corresponding to the right version -- the only solution I have is "git > > log"+grep; not really friendly. > > indexing the all the history could be very interesting, but it's enough > interesting to me having a system to query (and tag if needed) Guix info > from a point in time, i.e. index creation Currently, "guix search" already does a good job (modulo the scoring already discussed in length :-)). If you read the doc about Guile regexp and recutils then you can query (almost) all you want; or explain what you are not able to query. :-) However, the current implementation cannot scale for all the packages; other said find removed packages, or old version. And it is very frustrating, especially when you use "guix time-machine". A concrete example is given here [1] and "git log --grep" done in [2] is not convenient at all. Another concrete example, see point 1. in [3]. [1] https://lists.gnu.org/archive/html/help-guix/2019-06/msg00094.html [2] https://lists.gnu.org/archive/html/help-guix/2019-06/msg00098.html [3] https://lists.gnu.org/archive/html/help-guix/2020-01/msg00087.html > thinking about implementation, IMHO indexing the output of "guix search" > is doable but indexing the commit logs of git is complex, probably too > much complex I do not want to index the commit log message. But today, AFAIK, use these commit logs is the only way to find the commit providing the version an user want. And it is not cool, IMHO. The whishlist is: be able to search through all the packages you can manipulate with Guix. Other said index all the packages after the big overhaul (inferior). (Note that we are talking about packages but it is the same thing for services.) > me too, as I said I'd like something like notmuch - guixmuch :-) - that > indexes my mailbox; in Guix terms I see "my mailbox" [1] as > packages/services/machines and all other useful metadata I have in all > my channels/profiles What do you mean by "other useful metadata"? > ...then I have muchsync to sync the notmuch database across different > machines, and a similar feature whould be nice for guixmuch :-O It is already possible using manifests. > [1] where IMAP is replaced by git (in various repos, for > packages/config/channels) and offlineimap is replaced by guix pull What you want already exist: manifest. :-) And "guix publish" to expose your own substitutes if the package is not build in the Guix farm and you have already built it. > > However, note that index all the packages of all the Guix history > > using guile-git+fold-packages is not straightforward: resource > > consuming and piece of well-thought data structures. > > > > Well, now "guix time-machine" is here, Guix is lacking tools to browse > > the history of all the packages. > > we are asking too much to Guix, but it is an interesting feature I do not think so it is "too much". :-) All the best, simon