From: "Ludovic Courtès" <ludo@gnu.org>
To: Ryan Prior <rprior@protonmail.com>
Cc: Guix Devel <guix-devel@gnu.org>
Subject: Re: File search
Date: Sat, 05 Feb 2022 12:18:44 +0100 [thread overview]
Message-ID: <87sfsxk1d7.fsf@gnu.org> (raw)
In-Reply-To: <zl3uRxrzl-BRZEiMNycrLJNvKk2GkJRepPJrUjlRXWOltQYv7r9JPDxjLx8yKC3kKh7Pw2Tj6PY_Ze9pH6ct3aER-qS56b4tb-p_cObRWMo=@protonmail.com> (Ryan Prior's message of "Tue, 25 Jan 2022 23:45:35 +0000")
Hi,
Ryan Prior <rprior@protonmail.com> skribis:
> On Friday, January 21st, 2022 at 9:03 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> The database for 18K packages is quite big:
>>
>> --8<---------------cut here---------------start------------->8---
>>
>> $ du -h /tmp/db*
>>
>> 389M /tmp/db
>>
>> 82M /tmp/db.gz
>>
>> 61M /tmp/db.zst
>>
>> --8<---------------cut here---------------end--------------->8---
>> [snip]
>> In terms of privacy, I think it’s better if we can avoid making
>> one request per file searched for. Off-line operation would be
>> sweet, and it comes with responsiveness; fast off-line search is
>> necessary for things like ‘command-not-found’ (where the shell
>> tells you what package to install when a command is not found).
>
> Offline operation is crucial, and I don't think it's desirable to download tens or hundreds of megabytes. What about creating & distributing a bloom filter per package, with members being file names? This would allow us to dramatically reduce the size of data we distribute, at the cost of not giving 100% reliable answers. We've established, though, that some information is better than none, and the uncertainty can be resolved by querying a web service or building the package locally and searching its directory.
My understanding is that Bloom filters are sets essentially, but here we
need more than that: we need to map files to package names.
Or am I misunderstanding what you have in mind?
Thanks,
Ludo’.
next prev parent reply other threads:[~2022-02-05 11:19 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-21 9:03 File search Ludovic Courtès
2022-01-21 10:35 ` Mathieu Othacehe
2022-01-22 0:35 ` Ludovic Courtès
2022-01-21 19:00 ` Vagrant Cascadian
2022-01-22 0:37 ` Ludovic Courtès
2022-01-22 2:53 ` Maxim Cournoyer
2022-01-25 11:15 ` Ludovic Courtès
2022-01-25 11:20 ` Oliver Propst
2022-01-25 11:22 ` Oliver Propst
2022-01-22 4:46 ` raingloom
2022-01-22 7:55 ` Ricardo Wurmus
2022-01-24 15:48 ` Ludovic Courtès
2022-01-24 17:03 ` Ricardo Wurmus
2022-02-02 16:14 ` Maxim Cournoyer
2022-02-05 11:15 ` Ludovic Courtès
2022-01-25 23:45 ` Ryan Prior
2022-02-05 11:18 ` Ludovic Courtès [this message]
2022-02-06 13:27 ` André A. Gomes
-- strict thread matches above, loose matches on Subject: below --
2022-12-02 17:58 antoine.romain.dumont
2022-12-02 18:22 ` Antoine R. Dumont (@ardumont)
2022-12-03 18:19 ` Ludovic Courtès
2022-12-04 16:35 ` Antoine R. Dumont (@ardumont)
2022-12-06 10:01 ` Ludovic Courtès
2022-12-06 12:59 ` zimoun
2022-12-06 18:27 ` (
2022-12-08 15:41 ` Ludovic Courtès
2022-12-09 10:05 ` Antoine R. Dumont (@ardumont)
2022-12-09 18:05 ` zimoun
2022-12-11 10:22 ` Ludovic Courtès
2022-12-15 17:03 ` Antoine R. Dumont (@ardumont)
2022-12-19 21:25 ` Ludovic Courtès
2022-12-19 22:44 ` zimoun
2022-12-20 11:13 ` Antoine R. Dumont (@ardumont)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sfsxk1d7.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
--cc=rprior@protonmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.