unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@yhbt.net>
To: meta@public-inbox.org
Subject: [PATCH] doc: update TODO and WIP 1.6.0 release notes
Date: Wed, 10 Jun 2020 18:39:09 +0000	[thread overview]
Message-ID: <20200610183909.8402-1-e@yhbt.net> (raw)

Lots of big changes coming   Thanks to The Linux Foundation for
sponsoring me to hack on this in 2020 :)
---
 Documentation/RelNotes/v1.6.0.eml | 45 +++++++++++++++++++++++++++++++
 TODO                              | 33 ++++++++++++++++++++---
 2 files changed, 74 insertions(+), 4 deletions(-)
 create mode 100644 Documentation/RelNotes/v1.6.0.eml

diff --git a/Documentation/RelNotes/v1.6.0.eml b/Documentation/RelNotes/v1.6.0.eml
new file mode 100644
index 00000000000..283f42c898f
--- /dev/null
+++ b/Documentation/RelNotes/v1.6.0.eml
@@ -0,0 +1,45 @@
+From: Eric Wong <e@yhbt.net>
+To: meta@public-inbox.org
+Subject: [WIP] public-inbox 1.6.0
+MIME-Version: 1.0
+Content-Type: text/plain; charset=utf-8
+Content-Disposition: inline
+
+* General changes:
+
+  - ~/.cache/public-inbox/inline-c is automatically used for Inline::C
+    if it exists.  PERL_INLINE_DIRECTORY in env remains supported
+    and prioritized to support `nobody'-type users without HOME.
+
+  - msgmap.sqlite3 uses journal_mode=TRUNCATE, matching over.sqlite3
+    behavior for a minor reduction in VFS traffic
+
+  - message/{rfc822,news,global} attachments are decoded recursively
+    and indexed for search.  Use `public-inbox-index --reindex' to
+    ensure these attachments are indexed in old messages.
+
+* public-inbox-index
+
+  - --batch-size=BYTES or publicinbox.indexBatchSize parameter
+
+  - parallelize updates by default, "-j0" is (once again) allowed
+    parallelization
+
+* public-inbox-learn
+
+  - `rm' supports `--all' to remove from all configured inboxes
+
+* PublicInbox::WWW
+
+  - use consistent blank line around attachment links
+
+  - Attachments in message/{rfc822,news,global} messages can be
+    individually downloaded.  Downloading the entire message/rfc822
+    file in full remains supported
+
+  - $INBOX_DIR/description is treated as UTF-8
+
+Please report bugs via plain-text mail to: meta@public-inbox.org
+
+See archives at https://public-inbox.org/meta/ for all history.
+See https://public-inbox.org/TODO for what the future holds.
diff --git a/TODO b/TODO
index 16de36bf200..9396f661137 100644
--- a/TODO
+++ b/TODO
@@ -19,7 +19,7 @@ all need to be considered for everything we introduce)
   Meaning users can run this without needing a full copy of the
   archives in git repositories.
 
-* HTTP and NNTP proxy support.  Allow us to be a frontend for
+* HTTP, IMAP and NNTP proxy support.  Allow us to be a frontend for
   firewalled off (or Tor-exclusive) instances.  The use case is
   for offering a publicly accessible IP with a cheap VPS,
   yet storing large amounts of data on computers without a
@@ -32,7 +32,7 @@ all need to be considered for everything we introduce)
   archive locations to avoid SPOF.
 
 * optional Cache::FastMmap support so production deployments won't
-  need Varnish (Varnish doesn't protect NNTP, either)
+  need Varnish (Varnish doesn't protect NNTP or IMAP, either)
 
 * dogfood and take advantage of new kernel APIs (while maintaining
   portability to older Linux, free BSDs and maybe Hurd).
@@ -44,7 +44,8 @@ all need to be considered for everything we introduce)
 * Support more of RFC 3977 (NNTP)
   Is there anything left for read-only support?
 
-* Combined "super server" for NNTP/HTTP/POP3 to reduce memory overhead
+* Combined "super server" for NNTP/HTTP/POP3/IMAP to reduce memory,
+  process, and FD overhead
 
 * Configurable linkification for per-inbox shorthands:
   "$gmane/123456" could be configured to expand to the
@@ -111,8 +112,31 @@ all need to be considered for everything we introduce)
 * imperfect scraper importers for obfuscated list archives
   (e.g. obfuscated Mailman stuff, Google Groups, etc...)
 
+* extend public-inbox-watch to support IMAP, NNTP
+
 * improve performance and avoid head-of-line blocking on slow storage
 
+* HTTP(S) search API (likely JMAP, but GraphQL could be an option)
+  It should support git-specific prefixes (dfpre:, dfpost:, dfn:, etc)
+  as extensions.  If JMAP, it should have HTTP(S) analogues to
+  various IMAP extensions.
+
+* search across multiple inboxes, or admin-definable groups of inboxes
+
+* scalability to tens/hundreds of thousands of inboxes
+
+  - pagination for WwwListing
+
+  - inotify-based manifest.js.gz updates
+
+  - process/FD reduction (needs to be slow-storage friendly)
+
+  ...
+
+* command-line tool (similar to mairix/notmuch, but solver+git-aware)
+
+* consider removing doc_data from Xapian, redundant with over.sqlite3
+
 * share "git cat-file --batch" processes across inboxes to avoid
   bumping into /proc/sys/fs/pipe-user-pages-* limits
 
@@ -125,7 +149,8 @@ all need to be considered for everything we introduce)
 * linter to check validity of config file
 
 * linter option and WWW endpoint to graph relationships and flows
-  between inboxes, addresses maildirs, coderepos, etc...
+  between inboxes, addresses, Maildirs, coderepos, newsgroups,
+  IMAP mailboxes, etc...
 
 * pygments support - via Python script similar to `git cat-file --batch'
   to avoid startup penalty.  pygments.rb (Ruby) can be inspiration, too.

                 reply	other threads:[~2020-06-10 18:39 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200610183909.8402-1-e@yhbt.net \
    --to=e@yhbt.net \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).