[ANNOUNCE] public-inbox 1.7.0

[ANNOUNCE] public-inbox 1.7.0

[Eric Wong]

Another big release focused on multi-inbox search and scalability.

Special thanks to Konstantin Ryabitsev and Kyle Meyer for
numerous bug reports and documentation help.

* general changes

  - config file parsing is 2x faster with 50K inboxes

  - deduplication ignores whitespace differences within address fields

  - "PRAGMA optimize" is now issued on commits for SQLite 3.18+

* public-inbox-extindex

  A new Xapian + SQLite index able to search across several inboxes.
  This may be configured to replace per-inbox Xapian DBs,
  (but not per-inbox SQLite indices) and speed up manifest.js.gz
  generation.

  See public-inbox-extindex-format(5) and
  public-inbox-extindex(1) manpages for more details.

  Using it with "--all" speeds up various multi-inbox operations in
  PublicInbox::WWW, public-inbox-nntpd, and public-inbox-imapd.

* read-only public-inbox-daemon (-httpd, -nntpd, -imapd):

  libgit2 may be used via Inline::C to avoid hitting system pipe
  and process limits.  See public-inbox-tuning(7) manpage
  for more details.

* various memory usage reductions and workarounds for leaks in
  Encode <3.15, these mainly affect PublicInbox::WWW

* public-inbox-nntpd

  - startup is 6x faster with 50K inboxes if using -extindex

* PublicInbox::WWW

  - mboxrd search results are returned in reverse Xapian docid order,
    so more recent results are more likely to show up first

  - d: and dt: search prefixes allow "approxidate" formats supported
    by "git log --since="

  - manifest.js.gz generation is ~25x faster with -extindex

  - minor navigation improvements in search results HTML page

* lei - local email interface

  An experimental, subject-to-change, likely-to-eat-your-mail tool for
  personal mail as well as interacting with public-inboxes on the local
  filesystem or over HTTP(S).  See lei(1), lei-overview(7), and other
  lei-* manpages for details.  This is only ready-to-use w.r.t. external
  public-inbox instances, but mail synchronization for personal mail
  remains clunky.

* public-inbox-index

  - non-strict (Subject-based) threading supports non-ASCII characters,
    reindexing is necessary for old messages with non-ASCII subjects.

  - --batch-size is now 8M on 64-bit systems for throughput improvements,
    higher values are still advised for more powerful hardware.

* public-inbox-watch

  - IMAP and NNTP code shared with lei, fixing an off-by-one error
    in IMAP synchronization for single-message IMAP folders.

  - \Deleted and \Draft messages ignored for IMAP, as they are for
    Maildir.

  - IMAP and NNTP connection establishment (including git-credential
    prompts) ordering is now tied to config file order.

Compatibility:

* Rollbacks all the way to public-inbox 1.2.0 remain supported

Internal changes

* public-inbox-index switched to new internal IPC code shared
  with lei

Please report bugs via plain-text mail to: meta@public-inbox.org

See archives at https://public-inbox.org/meta/ for all history.
See https://public-inbox.org/TODO for what the future holds.

Re: [ANNOUNCE] public-inbox 1.7.0

[Konstantin Ryabitsev]

On Thu, Nov 04, 2021 at 07:52:00AM +0000, Eric Wong wrote:
> Another big release focused on multi-inbox search and scalability.

Congratulations on the release, Eric! Happy to be one of the frontline users,
and I'm sure a lot more kernel devs will jump on it now that lei is becoming
more available to them.

-K