From: Eric Abrahamsen <eric@ericabrahamsen.net>
To: Kyle Meyer <kyle@kyleam.com>
Cc: emacs-devel@gnu.org
Subject: Re: How to search on emacs-devel?
Date: Thu, 10 Mar 2022 09:54:06 -0800 [thread overview]
Message-ID: <878rthithd.fsf@ericabrahamsen.net> (raw)
In-Reply-To: <87wnh22w7o.fsf@kyleam.com> (Kyle Meyer's message of "Thu, 10 Mar 2022 00:49:31 -0500")
On 03/10/22 00:49 AM, Kyle Meyer wrote:
> Eric Abrahamsen writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> Andreas Schwab <schwab@linux-m68k.org> writes:
>>>
>>>> public-inbox is git under the hood (see
>>>> https://yhetil.org/emacs-devel/_/text/mirror/).
>>>
>>> Maybe local indexing with notmuch or what have you is still the easiest
>>> solution, to.
>
> For those that want to go this route, the main thing is getting the
> Git-stored messages [1] into a format that the indexer understands.
> There are some existing tools (not part of public-inbox) that can
> convert a public-inbox archive into a Maildir. l2md [2] and impibe [3]
> are the ones listed at
> <https://public-inbox.org/public-inbox.git/plain/Documentation/clients.txt>,
> though I haven't used either of them myself. (There's also
> public-inbox's lei, which I'll get to below.)
>
> In the case of Notmuch, the idea of making the Maildir-based Notmuch
> understand public-inbox archives was mentioned briefly on the Notmuch
> mailing list, but that hasn't gone anywhere yet:
>
> https://yhetil.org/notmuch/20210501051612.GB6409@dcvr/
Thanks for all this background! I guess I don't think there's much value
in converting public-inbox formats to something that notmuch can read,
if public-inbox comes with its own search facilities.
>
> [1] Documentation on public-inbox's archive format:
> https://public-inbox.org/public-inbox.git/plain/Documentation/public-inbox-v2-format.pod
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git
> [3] https://leahneukirchen.org/dotfiles/bin/impibe
>
>> Oh I see, if you have the public-inbox package installed locally, then
>> it can be used to index the git repo and provide a search interface.
>> Installing public-inbox on Arch has a dependency on approximately 9,000
>> perl packages, so I won't be doing that, but it looks like that would be
>> the easiest solution.
>
> I suspect you're referring to public-inbox's lei (local email interface)
> that was introduced in the most recent release (1.7.0). One way to use
> it is to wire up some number of archives as "externals". These can be
> remote, local, or some combination.
>
> For example, below adds emacs-devel as a local external. This clones
> the corresponding Git repo and indexes it with public-inbox-index. It
> unsurprisingly takes some time (maybe around 20 minutes on my machine).
>
> $ mkdir -p inboxes
> $ lei add-external --mirror https://yhetil.org/emacs-devel inboxes/emacs-devel
>
> Later, you can update the Git repo and index new messages with
>
> $ make -C inboxes/emacs-devel update
>
> And here's an example of adding a remote external:
>
> $ lei add-external https://yhetil.org/emacs-bugs
>
> The lei-q command is the main interface for searching:
>
> $ lei q f:eric d:2.days.ago.. | wc -l
> 8
> $ lei q f:eric d:2.days.ago.. | head -1
> [{"blob":"91ec6216f8daa3b731c832f5c5de722e891b49b7","dt":"2022-03-09T22:23:54Z","f":[["Eric Abrahamsen" ...
>
> If there are any local externals configured, lei-q won't reach out to
> the remote externals unless explicitly requested with --remote:
>
> $ lei q --remote f:eric d:2.days.ago.. | wc -l
> 9
>
> You can also limit a search to only a set of externals:
>
> $ lei q --only emacs-bugs f:eric d:2.days.ago.. | jq '.[0].s'
> "bug#54324: 29.0.50; Don't error during debbugs-gnu-rescan"
>
> Or specify a one-off remote URL that wasn't added via add-external:
>
> $ lei q --only https://lore.kernel.org/git userdiff NEAR emacs
>
> All of the above commands show JSON output, but there are several output
> formats available, including various mbox flavors and Maildir.
>
> Anyway, I've already written too much, but, for those interested,
> https://people.kernel.org/monsieuricon/lore-lei-part-1-getting-started
> along with lei-overview(7) and the other lei- manpages may be worth
> checking out.
All this is really interesting, thank you. It looks like one tool (or
tool suite) will handle mail transport, storage, and search -- that's
really nice. I'll read the overview, hold my nose and install all the
dependencies, and play around with it.
Thanks again,
Eric
next prev parent reply other threads:[~2022-03-10 17:54 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-09 18:51 How to search on emacs-devel? Eric Abrahamsen
2022-03-09 18:59 ` Eric Abrahamsen
2022-03-09 19:24 ` Stefan Monnier
2022-03-09 19:49 ` Eric Abrahamsen
2022-03-09 21:46 ` Stefan Monnier
2022-03-09 22:01 ` Eric Abrahamsen
2022-03-09 22:12 ` Stefan Monnier
2022-03-09 22:23 ` Eric Abrahamsen
2022-03-10 5:49 ` Kyle Meyer
2022-03-10 6:51 ` Eli Zaretskii
2022-03-10 17:43 ` Eric Abrahamsen
2022-03-10 17:56 ` Eli Zaretskii
2022-03-10 18:12 ` Eric Abrahamsen
2022-03-10 17:54 ` Eric Abrahamsen [this message]
-- strict thread matches above, loose matches on Subject: below --
2022-03-08 15:09 dalanicolai
2022-03-08 15:17 ` dalanicolai
2022-03-08 15:57 ` andrés ramírez
2022-03-08 16:06 ` dalanicolai
2022-03-08 17:49 ` Philip Kaludercic
2022-03-08 18:13 ` Eric Abrahamsen
2022-03-09 13:34 ` Robert Pluim
2022-03-09 15:44 ` Eric Abrahamsen
2022-03-09 17:36 ` Stefan Monnier
2022-03-09 17:43 ` Andreas Schwab
2022-03-10 15:50 ` Felix Dietrich
2022-03-10 16:46 ` Felix Dietrich
2022-03-09 11:41 ` dalanicolai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=878rthithd.fsf@ericabrahamsen.net \
--to=eric@ericabrahamsen.net \
--cc=emacs-devel@gnu.org \
--cc=kyle@kyleam.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).