* [PATCH 1/2] doc: add lei-mail-formats(5) manpage
2021-03-31 0:41 [PATCH 0/2] lei doc: mail formats Eric Wong
@ 2021-03-31 0:41 ` Eric Wong
2021-03-31 3:15 ` Kyle Meyer
2021-03-31 0:41 ` [PATCH 2/2] doc: lei-overview: favor Maildir for mutt examples Eric Wong
1 sibling, 1 reply; 4+ messages in thread
From: Eric Wong @ 2021-03-31 0:41 UTC (permalink / raw)
To: meta
While plenty of online documentation exists, it's good to have
a locally-available summary for users to look at offline.
Fix a URL in Watch.pm while we're at it, too.
---
Documentation/lei-mail-formats.pod | 101 +++++++++++++++++++++++++++++
MANIFEST | 1 +
Makefile.PL | 4 +-
lib/PublicInbox/Watch.pm | 2 +-
4 files changed, 106 insertions(+), 2 deletions(-)
create mode 100644 Documentation/lei-mail-formats.pod
diff --git a/Documentation/lei-mail-formats.pod b/Documentation/lei-mail-formats.pod
new file mode 100644
index 00000000..8accedb4
--- /dev/null
+++ b/Documentation/lei-mail-formats.pod
@@ -0,0 +1,101 @@
+=head1 NAME
+
+lei-mail-formats - description of mail formats supported by lei
+
+=head1 DESCRIPTION
+
+L<lei-q(1)> supports writing to several existing mail formats
+for interoperability with existing mail user agents (MUA);
+below is an overview of them to help users choose.
+
+=head1 Maildir
+
+The default output format when given a filesystem path, it supports
+parallel read-write access. Performance is acceptable for smaller
+directories, but degrades as mailboxes get larger. Speed and
+scalability are limited by kernel and filesystem performance
+due to the use of small files and large number of syscalls.
+
+See also: L<https://cr.yp.to/proto/maildir.html> and
+L<https://wiki2.dovecot.org/MailboxFormat/Maildir>
+
+=head1 Mbox family
+
+The mbox family consists of several incompatible formats.
+Locking for parallel access is supported, but may not be
+compatible across tools. With compression (e.g. L<gzip(1)>),
+they require the least amount of space while offering good
+read-only performance.
+
+Keyword updates (C<Status:> and/or C<X-Status:> headers)
+generally require rewriting the entire mbox.
+
+See also:
+L<https://www.loc.gov/preservation/digital/formats/fdd/fdd000383.shtml>,
+L<mbox(5)>
+
+=head2 mboxo
+
+The traditional BSD format. It quotes C<From > to C<E<gt>From >,
+but lines already beginning with C<E<gt>From > do not get quoted,
+thus automatic reversibility is not guaranteed. MUAs which favor
+L</mboxcl> or L</mboxcl2> may convert these automatically to their
+preferred format.
+
+Truncation is undetectable unless compressed with gzip or similar.
+
+=head2 mboxrd
+
+An evolution of L</mboxo>, but quotes C<From > lines prefixed
+with any number of C<E<gt>> characters and is thus fully
+reversible.
+
+This format is emitted by L<PublicInbox::WWW(3pm)> with gzip.
+It is supported by L<git-am(1)> since git 2.10.
+
+As with uncompressed L</mboxo>, uncompressed mboxrd are vulnerable
+to undetectable truncation.
+
+It gracefully degrades to being treated as L</mboxo> by MUAs
+unaware of the format as excessive C<E<gt>From > quoting is
+recognizable to humans.
+
+=head2 mboxcl
+
+L</mboxo> with a C<Content-Length:> header, C<From > lines
+remain quoted to retain readability with L</mboxo> and L</mboxrd> MUAs.
+However, it is easy to corrupt these files when using tools
+which are not aware of C<Content-Length:> and write out updates
+as L</mboxo>.
+
+L<mutt(1)> will convert L</mboxo> and L</mboxrd> to mboxcl upon opening.
+
+See also: L<https://www.jwz.org/doc/content-length.html>
+
+=head2 mboxcl2
+
+Like L</mboxcl>, but without C<From > any quoting. It is wholly
+incompatible with MUAs which only handle L</mboxo> and/or L</mboxrd>.
+This is format is generated by L<mutt(1)> when writing to a new
+mbox.
+
+=head1 MH
+
+Not yet supported, locking semantics (or lack thereof) appear to
+make it unsuitable for parallel access.
+
+=head1 IMAP
+
+Depending on the IMAP server software and configuration, IMAP
+servers may use any (or combination) of the aforementioned
+formats or a non-standard database backend.
+
+=head1 COPYRIGHT
+
+Copyright 2021 all contributors L<mailto:meta@public-inbox.org>
+
+License: AGPL-3.0+ L<http://www.gnu.org/licenses/agpl-3.0.txt>
+
+=head1 SEE ALSO
+
+L<lei(1)>, L<lei-q(1)>, L<lei-convert(1)>, L<lei-overview(7)>
diff --git a/MANIFEST b/MANIFEST
index f3cb0147..49d273fc 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -31,6 +31,7 @@ Documentation/lei-import.pod
Documentation/lei-init.pod
Documentation/lei-ls-external.pod
Documentation/lei-ls-label.pod
+Documentation/lei-mail-formats.pod
Documentation/lei-overview.pod
Documentation/lei-p2q.pod
Documentation/lei-q.pod
diff --git a/Makefile.PL b/Makefile.PL
index 27b49c53..feb89ec1 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -48,7 +48,9 @@ $v->{-m1} = [ map {
lei-forget-external lei-import lei-init lei-ls-external lei-ls-label
lei-tag lei-p2q lei-q)];
$v->{-m5} = [ qw(public-inbox-config public-inbox-v1-format
- public-inbox-v2-format public-inbox-extindex-format) ];
+ public-inbox-v2-format public-inbox-extindex-format
+ lei-mail-formats
+ ) ];
$v->{-m7} = [ qw(lei-overview public-inbox-overview public-inbox-tuning
public-inbox-glossary) ];
$v->{-m8} = [ qw(public-inbox-daemon) ];
diff --git a/lib/PublicInbox/Watch.pm b/lib/PublicInbox/Watch.pm
index 4fbc9640..05956cbb 100644
--- a/lib/PublicInbox/Watch.pm
+++ b/lib/PublicInbox/Watch.pm
@@ -2,7 +2,7 @@
# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
#
# ref: https://cr.yp.to/proto/maildir.html
-# httsp://wiki2.dovecot.org/MailboxFormat/Maildir
+# https://wiki2.dovecot.org/MailboxFormat/Maildir
package PublicInbox::Watch;
use strict;
use v5.10.1;
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH 2/2] doc: lei-overview: favor Maildir for mutt examples
2021-03-31 0:41 [PATCH 0/2] lei doc: mail formats Eric Wong
2021-03-31 0:41 ` [PATCH 1/2] doc: add lei-mail-formats(5) manpage Eric Wong
@ 2021-03-31 0:41 ` Eric Wong
1 sibling, 0 replies; 4+ messages in thread
From: Eric Wong @ 2021-03-31 0:41 UTC (permalink / raw)
To: meta
mboxes are generally horrible for interactive read-write use due
to locking. Describe our parallel behavior with mutt, since
writing mail can take a long while and being able to read
results as they're written is nice.
We'll also use a gzipped mboxrd for the import example, since
we can decompress gzipped mboxrds automatically, now.
---
Documentation/lei-overview.pod | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/Documentation/lei-overview.pod b/Documentation/lei-overview.pod
index c1f952c9..70dbf2b5 100644
--- a/Documentation/lei-overview.pod
+++ b/Documentation/lei-overview.pod
@@ -23,9 +23,9 @@ to memoize messages from remotes.
=over
-=item $ lei import mboxrd:t.mbox
+=item $ lei import mboxrd:t.mbox.gz
-Import the messages from an mbox into the local storage.
+Import the messages from a gzipped mboxrd into the local storage.
=item $ lei blob 59ec517f9
@@ -82,10 +82,11 @@ Search for messages whose subject includes "lei" and "skeleton".
Do the same, but also report unmatched messages that are in the same
thread as a matched message.
-=item $ lei q -t -o mboxcl2:t.mbox --mua=mutt s:lei s:skeleton
+=item $ lei q -t -o mdir --mua=mutt s:lei s:skeleton
-Write mboxcl2-formatted results to t.mbox and enter mutt to view the
-file by invoking C<mutt -f %f>.
+Write results to a Maildir at "mdir". Mutt will be invoked
+to open mfolder (C<mutt -f %f>) while results are being fetched
+and written.
=item $ lei q kw:flagged L:next
^ permalink raw reply related [flat|nested] 4+ messages in thread