From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Kyle Meyer Newsgroups: gmane.emacs.devel Subject: Re: How to search on emacs-devel? Date: Thu, 10 Mar 2022 00:49:31 -0500 Message-ID: <87wnh22w7o.fsf@kyleam.com> References: <874k47nemv.fsf@ericabrahamsen.net> <87wnh3lzo1.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5465"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Eric Abrahamsen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Mar 10 06:50:43 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nSBhC-0001DM-JI for ged-emacs-devel@m.gmane-mx.org; Thu, 10 Mar 2022 06:50:43 +0100 Original-Received: from localhost ([::1]:53328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nSBhB-0000vT-6z for ged-emacs-devel@m.gmane-mx.org; Thu, 10 Mar 2022 00:50:41 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:60896) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nSBgE-0000Cu-CU for emacs-devel@gnu.org; Thu, 10 Mar 2022 00:49:43 -0500 Original-Received: from out0.migadu.com ([94.23.1.103]:47903) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nSBgB-0004Li-9m for emacs-devel@gnu.org; Thu, 10 Mar 2022 00:49:42 -0500 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kyleam.com; s=key1; t=1646891373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yS+mTWX9uyV40N8PnJZMYKRg4+H9KVDGpAmTtnqCOhs=; b=u+HBLUrX8677doh0SIBydoUIAFJ/s3LjdiNYOM8AFGyr4s3/2ME/rYsCruIrDtfGEaEgVw 0/1ith3hzr+reuqSYCQXirj3N96ZCA29w6VsjT2SA4+wTmblLmwej6X8M6cdMAfbLXem8q SvPOTWGvxglP3Xx/YF0DSFtf3QfMcFAfELe16Bhj6ZWLzAlJatjOilwPw/wmxtlXm9/0RB KqVS0HxFg+BuymEB3FZ1i1v84Q7BfWRgduemjogk+s0bNvka2MFbLZq/aZXkpqEBW7wt5j o4anTYiFG+1UB0VpQKldgzOFgBxGNXFKu4oj0HXWpeYwG2SONjDZYjLuNa1K4g== In-Reply-To: <87wnh3lzo1.fsf@ericabrahamsen.net> X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: kyleam.com Received-SPF: pass client-ip=94.23.1.103; envelope-from=kyle@kyleam.com; helo=out0.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:286982 Archived-At: Eric Abrahamsen writes: > Eric Abrahamsen writes: > >> Andreas Schwab writes: >> >>> public-inbox is git under the hood (see >>> https://yhetil.org/emacs-devel/_/text/mirror/). >> >> Maybe local indexing with notmuch or what have you is still the easiest >> solution, to. For those that want to go this route, the main thing is getting the Git-stored messages [1] into a format that the indexer understands. There are some existing tools (not part of public-inbox) that can convert a public-inbox archive into a Maildir. l2md [2] and impibe [3] are the ones listed at , though I haven't used either of them myself. (There's also public-inbox's lei, which I'll get to below.) In the case of Notmuch, the idea of making the Maildir-based Notmuch understand public-inbox archives was mentioned briefly on the Notmuch mailing list, but that hasn't gone anywhere yet: https://yhetil.org/notmuch/20210501051612.GB6409@dcvr/ [1] Documentation on public-inbox's archive format: https://public-inbox.org/public-inbox.git/plain/Documentation/public-inbox-v2-format.pod [2] https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git [3] https://leahneukirchen.org/dotfiles/bin/impibe > Oh I see, if you have the public-inbox package installed locally, then > it can be used to index the git repo and provide a search interface. > Installing public-inbox on Arch has a dependency on approximately 9,000 > perl packages, so I won't be doing that, but it looks like that would be > the easiest solution. I suspect you're referring to public-inbox's lei (local email interface) that was introduced in the most recent release (1.7.0). One way to use it is to wire up some number of archives as "externals". These can be remote, local, or some combination. For example, below adds emacs-devel as a local external. This clones the corresponding Git repo and indexes it with public-inbox-index. It unsurprisingly takes some time (maybe around 20 minutes on my machine). $ mkdir -p inboxes $ lei add-external --mirror https://yhetil.org/emacs-devel inboxes/emacs-devel Later, you can update the Git repo and index new messages with $ make -C inboxes/emacs-devel update And here's an example of adding a remote external: $ lei add-external https://yhetil.org/emacs-bugs The lei-q command is the main interface for searching: $ lei q f:eric d:2.days.ago.. | wc -l 8 $ lei q f:eric d:2.days.ago.. | head -1 [{"blob":"91ec6216f8daa3b731c832f5c5de722e891b49b7","dt":"2022-03-09T22:23:54Z","f":[["Eric Abrahamsen" ... If there are any local externals configured, lei-q won't reach out to the remote externals unless explicitly requested with --remote: $ lei q --remote f:eric d:2.days.ago.. | wc -l 9 You can also limit a search to only a set of externals: $ lei q --only emacs-bugs f:eric d:2.days.ago.. | jq '.[0].s' "bug#54324: 29.0.50; Don't error during debbugs-gnu-rescan" Or specify a one-off remote URL that wasn't added via add-external: $ lei q --only https://lore.kernel.org/git userdiff NEAR emacs All of the above commands show JSON output, but there are several output formats available, including various mbox flavors and Maildir. Anyway, I've already written too much, but, for those interested, https://people.kernel.org/monsieuricon/lore-lei-part-1-getting-started along with lei-overview(7) and the other lei- manpages may be worth checking out.