From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id DF6901F406 for ; Wed, 13 Dec 2023 10:54:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1702464853; bh=wVVm+elOywvLMs2eILOrYFJ6v4z82v+Qd7qTuk6Emqs=; h=From:To:Subject:Date:From; b=ZoUZNs6kIvaEUTLget33i2p9Dx6WWP3a6eywQ9IZ08XU1+u0QhpPDVc6yShiXnGmW QtinAv3udVmozfe6uUyaqcB9O+2EgPQXyshOMFlxLAZM/Cs/9Kufv7kEC59qIpWTE8 qvN6JL5gBnxdud2/6R1vzvYaZdvSbvlj58z9divI= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH] doc: RelNotes: more 2.0 updates Date: Wed, 13 Dec 2023 10:51:07 +0000 Message-ID: <20231213105107.60704-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: But new ideas keep popping into muh brain :x --- Documentation/RelNotes/v2.0.0.wip | 50 ++++++++++++++++++++++----- Documentation/public-inbox-tuning.pod | 23 ++++++++++++ Documentation/txt2pre | 1 + 3 files changed, 66 insertions(+), 8 deletions(-) diff --git a/Documentation/RelNotes/v2.0.0.wip b/Documentation/RelNotes/v2.0.0.wip index d4037800..c473e8d7 100644 --- a/Documentation/RelNotes/v2.0.0.wip +++ b/Documentation/RelNotes/v2.0.0.wip @@ -4,11 +4,24 @@ MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline -This release includes several new features and fixes. +This release includes several new features and fixes; mostly +around improved integration between inboxes and coderepos for +solver. Portability and reliability is also improved. -There is also ongoing work to integrate coderepo handling into -the codebase and the idea of `lei p2q' is integrated into the -WWW UI. +public-inbox-cindex is a new command to index coderepos for +WWW search and perform automatic associations between +coderepos and inboxes. This makes solver vastly more useful +for the WWW UI as admins will no longer have to manually +associate coderepos with inboxes. + +public-inbox-clone gains the ability to mirror entire (or partial) +grokmirror-compatible manifests. + +Internal process and object management data structures are vastly +simplified throughout and error handling made more robust. + +git SHA-256 support remains a work-in-progress for inboxes and +extindex due to the need to interoperate with SHA-1 epochs. Upgrading: @@ -32,7 +45,7 @@ treewide * standardize on `#' prefix for stderr diagnostics (previously `I:') - * avoid O(n) behavior for reaping dead processes in daemons + * SHA-256 coderepos are fully-supported (inboxes not yet) PublicInbox::WWW @@ -54,6 +67,10 @@ PublicInbox::WWW * add /$INBOX/$MSGID/d/ endpoint to show diffs in reused Message-IDs (`lei mail-diff' for the web) + * support POST /$INBOX/$MSGID/?x=m&q= to limit mbox results to a thread + + * add topics_(new|active).(html|atom) endpoints + lei * use http.proxy / http..proxy from system-wide git-config if @@ -71,6 +88,12 @@ lei * `lei import' now sets labels+keywords consistently on all already imported messages + * fix `lei up' on saved local queries which previously used -t/--threads + + * `lei convert' output to v2 public-inboxes is now idempotent + + * improved bash completion for labels (see contrib/completion) + solver (used by lei (rediff|blob), and PublicInbox::WWW) * handle copies in patches properly @@ -86,6 +109,14 @@ portability * fix excessive pipelining to `git cat-file' on systems with small getdelim(3) buffers (mainly affects musl) + * support Alpine Linux, Dragonfly, NetBSD and OpenBSD. This resulted + not only in bugfixes to our code, but also to Dragonfly and OpenBSD. + +public-inbox-pop3d + + * support `limit=NUM' and `initial_limit=NUM' query parameters + in mailbox names to limit results + public-inbox-nntpd * fix LISTGROUP with range (affects neomutt) @@ -96,11 +127,13 @@ public-inbox-clone / public-inbox-fetch / `lei add-external --mirror' public-inbox-clone: - * parallel mirroring of multiple inboxes/coderepos via manifest + * parallel mirroring of multiple inboxes/coderepos via manifest, + public-inbox-fetch is not used in this mode * new flags to support manifest mirroring include: - --dry-run, --inbox-config=, --project-list=, --prune, - --keep-going, --jobs, --include=, --exclude=, --objstore=, ... + --dry-run, --inbox-config=, --project-list=, --prune, --purge, + --keep-going, --jobs, --include=, --exclude=, --objstore=, + --manifest=, --remote-manifest= See public-inbox-clone(1) man page for more details. PublicInbox::SaPlugin::ListMirror @@ -111,6 +144,7 @@ Search improvements (lei and PublicInbox::WWW) * quoted text inside base-85 binary patches is no longer indexed + Thanks to all the bug reporters and users who made this release possible. diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 58a4d9bc..38810ce6 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -42,6 +42,14 @@ Other OS tuning knobs Scalability to many inboxes +=item 9 + +public-inbox-cindex --join performance + +=item 10 + +public-inbox-clone with shared object stores + =back =head2 New inboxes: public-inbox-init -V2 @@ -175,6 +183,21 @@ for thousands of inboxes with L. Frequent packing (via L) both improves performance and reduces the need to increase C. +=head2 public-inbox-cindex --join performance + +A C++ compiler and the Xapian development files makes C<--join> or +C<--join=aggressive> orders of magnitude faster in L. +On Debian-based systems this is C. RPM-based distros have +these in C or C. *BSDs typically +package development files together with runtime libraries, so the C +or C package will already have the development files. + +=head2 public-inbox-clone with shared object stores + +When mirroring manifests with many forks using the same objstore, +git 2.41+ is highly recommended for performance as we automatically +use the C feature to speed up negotiation. + =head1 CONTACT Feedback encouraged via plain-text mail to L diff --git a/Documentation/txt2pre b/Documentation/txt2pre index c258a90e..89a77199 100755 --- a/Documentation/txt2pre +++ b/Documentation/txt2pre @@ -50,6 +50,7 @@ for (qw[lei(1) lei-tag(1) lei-up(1) public-inbox.cgi(1) + public-inbox-cindex(1) public-inbox-clone(1) public-inbox-config(5) public-inbox-config(5)