From: zimoun <zimon.toutoune@gmail.com>
To: Arun Isaac <arunisaac@systemreboot.net>
Cc: Guix Devel <guix-devel@gnu.org>
Subject: Re: Inverted index to accelerate guix package search
Date: Wed, 15 Jan 2020 12:54:52 +0100 [thread overview]
Message-ID: <CAJ3okZ2by92MCETq6ZnK29EcYdVsZGhOS87JV1Nn3H94+8huAg@mail.gmail.com> (raw)
In-Reply-To: <cu74kwxe0jv.fsf@systemreboot.net>
Hi,
On Wed, 15 Jan 2020 at 06:44, Arun Isaac <arunisaac@systemreboot.net> wrote:
> > Well, the issue is 'scoring' a query.
>
> I think the issue of whether to use an inverted index is orthogonal to
> the quest to improve the relevance of the search results. Implementing
> tf-idf like you suggested could greatly benefit from having fast
> searches.
I think it is not so orthogonal. In general, a fast and good system is
a combination of relevant scoring adapted to the good data structure,
IMHO.
However, I agree that adding an inverted index will improve the
current situation of "guix search" -- keeping the current scoring
function -- and ease the end-user experience.
> Pierre Neidhardt <mail@ambrevar.xyz> writes:
>
> > By the way, what about using Xapian in Guix?
> >
> > https://en.wikipedia.org/wiki/Xapian
> >
> > If it's relevant, maybe we can follow up with a discussion in a new
> > thread.
>
> I feel xapian is too much work (considering that we don't yet have guile
> bindings) compared to our own simple implementation of an inverted
> index. But, of course, I am biased since I wrote the inverted index
> code! :-)
It depends on how long run we are talking. :-)
Xapian avoids to reinvent the wheel. ;-)
> But, on a more serious note, if we move to xapian, we will not be able
> to support regular expression based search queries that we support
> today.
I am not convinced...
> On the question of whether xapian is too heavy, I think we should make
> it an optional dependency of Guix so that it remains possible to build
> and use a more minimalistic Guix for those interested in such things.
Guix comes with SQLite and it is ok.
The question is: how Xapian is minimalist. :-)
(need some investigation)
All the best,
simon
next prev parent reply other threads:[~2020-01-15 11:55 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-12 15:03 Inverted index to accelerate guix package search Arun Isaac
2020-01-13 8:48 ` Pierre Neidhardt
2020-01-13 15:08 ` Arun Isaac
2020-01-13 17:54 ` zimoun
2020-01-14 3:53 ` Bengt Richter
2020-01-14 14:21 ` Pierre Neidhardt
2020-01-14 15:27 ` Giovanni Biscuolo
2020-01-14 20:55 ` zimoun
2020-01-14 21:41 ` Pierre Neidhardt
2020-01-14 21:59 ` zimoun
2020-01-15 9:12 ` Pierre Neidhardt
2020-01-15 14:25 ` zimoun
2020-01-15 11:53 ` Giovanni Biscuolo
2020-01-15 14:49 ` zimoun
2020-01-16 19:06 ` Arun Isaac
2020-01-16 20:08 ` zimoun
2020-01-17 17:13 ` Pierre Neidhardt
2020-01-17 19:11 ` Pierre Neidhardt
2020-01-18 20:31 ` Arun Isaac
2020-01-15 5:44 ` Arun Isaac
2020-01-15 9:06 ` Pierre Neidhardt
2020-01-15 12:00 ` zimoun
2020-01-15 13:41 ` Pierre Neidhardt
2020-01-15 11:54 ` zimoun [this message]
2020-01-15 21:03 ` Ricardo Wurmus
2020-01-15 21:19 ` zimoun
2020-01-16 14:44 ` Ludovic Courtès
2020-01-16 15:04 ` zimoun
2020-01-16 19:11 ` Arun Isaac
2020-01-16 19:53 ` zimoun
2020-01-17 19:29 ` Arun Isaac
2020-01-20 19:14 ` zimoun
2020-01-20 20:42 ` Arun Isaac
2020-01-21 10:46 ` Ludovic Courtès
2020-01-23 20:49 ` Arun Isaac
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJ3okZ2by92MCETq6ZnK29EcYdVsZGhOS87JV1Nn3H94+8huAg@mail.gmail.com \
--to=zimon.toutoune@gmail.com \
--cc=arunisaac@systemreboot.net \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.