Re: Package file indexing

[Pierre Neidhardt <mail@ambrevar.xyz>]

[-- Attachment #1: Type: text/plain, Size: 3729 bytes --]

Hello again!

I'm resurrecting this since I've just started working on this as part of
the NGI application! :)

>>> Internally it’d call ‘guix substitute’ to fetch the file index from
>>> the substitute server, check its signature, cache it locally, and then
>>> look up the file.

What about storing the file listing in the narinfo instead?
Is this doable?  If so, then it should be quite simple to implement, it
would basically mimic "guix size."

>>> You should look at how NixOS does it for its ‘command-not-found’ support
>>> (I think it’s part of NixOS, not Nix).  IIRC they distribute an SQLite
>>> database, but it’s a pretty ad-hoc mechanism without authentication.

I'll see if I can find the code for this.

If we embed the file listing in the narinfo as I suggested above, then
there we be no point in maintaining a separate database.

>> I could work on this, but that seems like a lot of work, especially for
>> me who knows nothing about the daemon (but hey, it's a great opportunity
>> to learn!).
>
> Note that the daemon would act as an intermediary, but in practice the
> functionality would be very much peripheral to the daemon.  IOW, you
> don’t need to know about the daemon internals.

Any files you recommend looking at to get started?
I suppose that "guix/scripts/size.scm" is a good start.



Last but not least: previously we suggested adding a subcommand like
"guix which" or "guix filesearch".  In another thread, Simon suggested
that this would be a bad idea and factoring the file search into "guix
search" is probably better.  For instance, we could do

  guix search bin/foo

and it would report the packages containing the "bin/foo" path.  This
could mean that we need to adapt the output to display the file listing
as well.  If listing all files would be too verbose, we can list only
the matching files:

--8<---------------cut here---------------start------------->8---
name: jami
version: 20191101.3.67671e7
outputs: out
systems: x86_64-linux i686-linux
dependencies: adwaita-icon-theme@3.32.0 clutter-gtk@1.8.4 clutter@1.26.2 doxygen@1.8.15
+ evolution-data-server@3.32.4 gettext@0.20.1 glib@2.60.6 gtk+@3.24.12 libcanberra@0.30
+ libnotify@0.7.7 libring@20191101.3.67671e7 libringclient@20191101.3.67671e7
+ pkg-config@0.29.2 qrencode@4.0.2 sqlite-with-column-metadata@3.28.0 webkitgtk@2.26.2
location: gnu/packages/telephony.scm:890:2
homepage: https://jami.net
license: GPL 3+
synopsis: Distributed, privacy-respecting communication program  
description: Jami (formerly GNU Ring) is a secure and distributed voice, video and chat
+ communication platform that requires no centralized server and leaves the power of privacy
+ in the hands of the user.  It supports the SIP and IAX protocols, as well as decentralized
+ calling using P2P-DHT.
+ 
+ This package provides the Jami client for the GNOME desktop.
filepaths:
+ bin/foo
+ share/bar/bin/foo-blah
relevance: 24
--8<---------------cut here---------------end--------------->8---

That said, some terms may match too frequently.  For instance, "guix
search lib" would match almost all packages that have libraries and
result in a huge, useless output.

I suggest the following:

- Add a "--search-file-paths=[auto|on|off]" option.
- When --search-file-paths is "auto", file paths are automatically
  searched for against terms that contain a slash.  E.g. "lib" won't
  return file paths but "lib/" will.

Another feature that could be nice: list the file paths for the given
packages.
I think we need a separate subcommand for this, e.g. "guix list-files".

Thoughts?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]