From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id C6A1A1F66E for ; Sat, 22 Aug 2020 06:06:27 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/5] "mairix -t" workalike for mbox.gz downloads Date: Sat, 22 Aug 2020 06:06:22 +0000 Message-Id: <20200822060627.15595-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Actually, the Xapian aspect of it turned out to be easy once I learned ->set_collapse_key. Getting the tests and compatibility with existing (pre-upgrade) inboxes was more work. It requires "public-inbox-index --reindex" to activate; but PATCH 5/5 makes it safe to upgrade WWW either before or after --reindex. That means BOFHs can upgrade without regard to ordering. Tested with w3m, links, and lynx (I actually split out my lynx fix separately): https://public-inbox.org/meta/20200822004125.9458-1-e@80x24.org/ TODO: CLI tool support, HTML interface, JMAP, etc... Eric Wong (5): searchidxshard: clear $msgref buffer properly searchidx: put all shard-related stuff in SearchIdxShard.pm searchidx: index THREADID in Xapian search: support downloading mboxes results with full thread mbox: disable "&t" on existing Xapian until full reindex Documentation/standards.perl | 4 +++ lib/PublicInbox/Mbox.pm | 54 +++++++++++++++++++++++++------ lib/PublicInbox/Over.pm | 31 +++++++++++++++++- lib/PublicInbox/OverIdx.pm | 18 +++++------ lib/PublicInbox/Search.pm | 16 +++++++-- lib/PublicInbox/SearchIdx.pm | 51 +++++++++-------------------- lib/PublicInbox/SearchIdxShard.pm | 48 ++++++++++++++++++++++----- lib/PublicInbox/SearchQuery.pm | 8 +++-- lib/PublicInbox/SearchView.pm | 30 +++++++++++------ lib/PublicInbox/Smsg.pm | 3 +- lib/PublicInbox/V2Writable.pm | 19 ++++++++--- t/init.t | 1 + t/over.t | 13 ++++---- t/psgi_search.t | 39 ++++++++++++++++++++-- 14 files changed, 244 insertions(+), 91 deletions(-)