From: Kyle Meyer <kyle@kyleam.com>
To: Eric Abrahamsen <eric@ericabrahamsen.net>
Cc: emacs-devel@gnu.org
Subject: Re: How to search on emacs-devel?
Date: Thu, 10 Mar 2022 00:49:31 -0500 [thread overview]
Message-ID: <87wnh22w7o.fsf@kyleam.com> (raw)
In-Reply-To: <87wnh3lzo1.fsf@ericabrahamsen.net>
Eric Abrahamsen writes:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Andreas Schwab <schwab@linux-m68k.org> writes:
>>
>>> public-inbox is git under the hood (see
>>> https://yhetil.org/emacs-devel/_/text/mirror/).
>>
>> Maybe local indexing with notmuch or what have you is still the easiest
>> solution, to.
For those that want to go this route, the main thing is getting the
Git-stored messages [1] into a format that the indexer understands.
There are some existing tools (not part of public-inbox) that can
convert a public-inbox archive into a Maildir. l2md [2] and impibe [3]
are the ones listed at
<https://public-inbox.org/public-inbox.git/plain/Documentation/clients.txt>,
though I haven't used either of them myself. (There's also
public-inbox's lei, which I'll get to below.)
In the case of Notmuch, the idea of making the Maildir-based Notmuch
understand public-inbox archives was mentioned briefly on the Notmuch
mailing list, but that hasn't gone anywhere yet:
https://yhetil.org/notmuch/20210501051612.GB6409@dcvr/
[1] Documentation on public-inbox's archive format:
https://public-inbox.org/public-inbox.git/plain/Documentation/public-inbox-v2-format.pod
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git
[3] https://leahneukirchen.org/dotfiles/bin/impibe
> Oh I see, if you have the public-inbox package installed locally, then
> it can be used to index the git repo and provide a search interface.
> Installing public-inbox on Arch has a dependency on approximately 9,000
> perl packages, so I won't be doing that, but it looks like that would be
> the easiest solution.
I suspect you're referring to public-inbox's lei (local email interface)
that was introduced in the most recent release (1.7.0). One way to use
it is to wire up some number of archives as "externals". These can be
remote, local, or some combination.
For example, below adds emacs-devel as a local external. This clones
the corresponding Git repo and indexes it with public-inbox-index. It
unsurprisingly takes some time (maybe around 20 minutes on my machine).
$ mkdir -p inboxes
$ lei add-external --mirror https://yhetil.org/emacs-devel inboxes/emacs-devel
Later, you can update the Git repo and index new messages with
$ make -C inboxes/emacs-devel update
And here's an example of adding a remote external:
$ lei add-external https://yhetil.org/emacs-bugs
The lei-q command is the main interface for searching:
$ lei q f:eric d:2.days.ago.. | wc -l
8
$ lei q f:eric d:2.days.ago.. | head -1
[{"blob":"91ec6216f8daa3b731c832f5c5de722e891b49b7","dt":"2022-03-09T22:23:54Z","f":[["Eric Abrahamsen" ...
If there are any local externals configured, lei-q won't reach out to
the remote externals unless explicitly requested with --remote:
$ lei q --remote f:eric d:2.days.ago.. | wc -l
9
You can also limit a search to only a set of externals:
$ lei q --only emacs-bugs f:eric d:2.days.ago.. | jq '.[0].s'
"bug#54324: 29.0.50; Don't error during debbugs-gnu-rescan"
Or specify a one-off remote URL that wasn't added via add-external:
$ lei q --only https://lore.kernel.org/git userdiff NEAR emacs
All of the above commands show JSON output, but there are several output
formats available, including various mbox flavors and Maildir.
Anyway, I've already written too much, but, for those interested,
https://people.kernel.org/monsieuricon/lore-lei-part-1-getting-started
along with lei-overview(7) and the other lei- manpages may be worth
checking out.
next prev parent reply other threads:[~2022-03-10 5:49 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-09 18:51 How to search on emacs-devel? Eric Abrahamsen
2022-03-09 18:59 ` Eric Abrahamsen
2022-03-09 19:24 ` Stefan Monnier
2022-03-09 19:49 ` Eric Abrahamsen
2022-03-09 21:46 ` Stefan Monnier
2022-03-09 22:01 ` Eric Abrahamsen
2022-03-09 22:12 ` Stefan Monnier
2022-03-09 22:23 ` Eric Abrahamsen
2022-03-10 5:49 ` Kyle Meyer [this message]
2022-03-10 6:51 ` Eli Zaretskii
2022-03-10 17:43 ` Eric Abrahamsen
2022-03-10 17:56 ` Eli Zaretskii
2022-03-10 18:12 ` Eric Abrahamsen
2022-03-10 17:54 ` Eric Abrahamsen
-- strict thread matches above, loose matches on Subject: below --
2022-03-08 15:09 dalanicolai
2022-03-08 15:17 ` dalanicolai
2022-03-08 15:57 ` andrés ramírez
2022-03-08 16:06 ` dalanicolai
2022-03-08 17:49 ` Philip Kaludercic
2022-03-08 18:13 ` Eric Abrahamsen
2022-03-09 13:34 ` Robert Pluim
2022-03-09 15:44 ` Eric Abrahamsen
2022-03-09 17:36 ` Stefan Monnier
2022-03-09 17:43 ` Andreas Schwab
2022-03-10 15:50 ` Felix Dietrich
2022-03-10 16:46 ` Felix Dietrich
2022-03-09 11:41 ` dalanicolai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wnh22w7o.fsf@kyleam.com \
--to=kyle@kyleam.com \
--cc=emacs-devel@gnu.org \
--cc=eric@ericabrahamsen.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).