From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4EfhACXkjGD+xAAAgWs5BA (envelope-from ) for ; Sat, 01 May 2021 07:16:21 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id MOwoOCTkjGDcKwAAB5/wlQ (envelope-from ) for ; Sat, 01 May 2021 05:16:20 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 430A115993 for ; Sat, 1 May 2021 07:16:20 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id 620DE1FBD2; Sat, 1 May 2021 01:16:15 -0400 (EDT) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by mail.notmuchmail.org (Postfix) with ESMTPS id 4AC151FAC2 for ; Sat, 1 May 2021 01:16:13 -0400 (EDT) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 7FA9A1F5AE; Sat, 1 May 2021 05:16:12 +0000 (UTC) Date: Sat, 1 May 2021 05:16:12 +0000 From: Eric Wong To: Felipe Contreras Subject: Re: notmuch and public-inbox Message-ID: <20210501051612.GB6409@dcvr> References: <608c9185115fd_251d208e0@natae.notmuch> <20210501000500.GA20219@dcvr> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Message-ID-Hash: VWDZOZ4UKAYLSVSOLFMGEPKTQXKOQSLE X-Message-ID-Hash: VWDZOZ4UKAYLSVSOLFMGEPKTQXKOQSLE X-MailFrom: e@80x24.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0 CC: notmuch@notmuchmail.org, "W. Trevor King" X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619846180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=qsWKPz1s1ozz2T5fvocz/RgOkyPcZukMNYxa/KLJe80=; b=k/YeIXMcShnvjrREFVSNucN547+GDIy4WE+Zk6S4OUuJ20/0GLHVtvM5RL3IH69S2XKLFx jTVIJn3S5rQ+qmf0UOC0SjNA6uM5e7HlsNRBCnwcRMK9RLF6onyxcAuMocyHOtbgRETfUr hqHMl7hrAakMHQ6g6HTQo9e3ng5D4yB/HrA/YFf3oD0I7P8N8Wht1QSLbEQIJzur8Pg/2b o7MRo/R1mpcLJWXZRe3NKdsv26G85R96uPlIeXerHvm2W5y3fEqA6+pEgT5wK/t2rTUF3a 59hnx9h9+uuoSgHABPsi3OD9OKSSxoWEd590GGQ2EJCdmbpAuWlXI9GSukYPaA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619846180; a=rsa-sha256; cv=none; b=XMY8uROrM8ldh/mjNpYq85V83pVthFnLDGAahekzqlfWSnOTf2XPx8xprnAsSL3WZW4fb0 ESwT560NCfDacVgJzcSLISj5oRGsTNbRuenuqPpmrZaAPa2mJ/7cuiCTEOY78QGGqksUIX ri+ZeyWfBZo9rMHQwqa+GXIlSLepJfy36Br0tTRH3jAmXL02zm8PXiA86bvobvwFcbj8II MYFGRppEZJZ5rhhPqj1fVfPJ82i02QOMiFgEb5JXYy/Rhq4bqtb4yGiYJH44iAMfezWVXj WDPYrUeCp5PvMXwdC1z5VPqULo64nYbEUj/1jsBJS1hUVHI+E5lBh/FFj35WUw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -1.55 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 430A115993 X-Spam-Score: -1.55 X-Migadu-Scanner: scn0.migadu.com X-TUID: Eo1DMfPTeip3 Felipe Contreras wrote: > On Fri, Apr 30, 2021 at 7:05 PM Eric Wong wrote: > > > > Felipe Contreras wrote: > > > My workflow with notmuch is near to perfect, however, the only pain > > > point I have is fetching all the mail of a particular mailing list. > > > > > > To do this efficiently public-inbox seems ideal, however, when > > > searching information to link notmuch to public-inbox I don't find > > > anything of value. In fact, I can't find an URL of a public-inbox > > > repository of the notmuch mailing list. > > > > Kyle maintains an unofficial mirror at https://yhetil.org/notmuch > > Nice. Who is Kyle? A notmuch user and public-inbox user/contributor; beyond that I don't know. public-inbox is all designed so anybody can make mirrors of any mail they have. (as I've mirrored a bunch of lists myself without ever asking permission) > > There's no real relationship between them aside from they both > > use Xapian (and I learned Xapian from reading the notmuch source). > > I don't mean sharing the Xapian database (although that could be > interesting for the future). I'm talking about as a client of > public-inbox, not as a server. > > I mean doing a git clone for a public-inbox repository and notmuch > indexing that repository. Ah, the git repository formats are documented at: https://public-inbox.org/public-inbox-v2-format.html https://public-inbox.org/public-inbox-v1-format.html > > > Am I missing something or has nobody really worked on linking these two > > > tools? Seems like an obvious area of opportunity. > > > > I think W. Trevor King (Cc-ed) also started looking something > > many years ago, but I'm not sure if anything became of it. > > > > I never had the interest in using notmuch since Maildirs are a > > non-starter with millions of messages with current FSes/OSes. > > mairix + gzipped mboxes mostly works for me, (though mairix > > indexing is silly expensive[1]) > > If notmuch was patched to support the public-inbox format--as an > alternative to Maildir--then users of public-inbox could clone a > repository, and use notmuch to index that. > > I don't see how that could be difficult. But then again, I haven't > looked at the Maildir code. That would be cool; always room for more tools to interoperate with each other. (I'm quite busy with public-inbox and trying to avoid AOT languages as much as possible). Keep in mind some users are already happy with l2md and impibe for writing Maildir, so there's already (space inefficient ways) to make notmuch index data from public-inboxes: * l2md - Maildir and procmail importer using C + libgit2 https://git.kernel.org/pub/scm/linux/kernel/git/dborkman/l2md.git * impibe - Perl script to import v1 or v2 to Maildir https://leahneukirchen.org/dotfiles/bin/impibe discussion: https://public-inbox.org/meta/87v9m0l8t1.fsf@vuxu.org/ (maybe more will appear at )