Re: [Patch v2 16/17] notmuch-{dump, restore}.1: document new format options

[Jani Nikula <jani@nikula.org>]

On Sat, 24 Nov 2012, david@tethera.net wrote:
> From: David Bremner <bremner@debian.org>
>
> More or less arbitrarily, notmuch-dump.1 gets the more detailed
> description of the format.
> ---
>  man/man1/notmuch-dump.1    |   58 +++++++++++++++++++++++++++++++++++++++++++
>  man/man1/notmuch-restore.1 |   59 +++++++++++++++++++++++++++++++++++++++-----
>  2 files changed, 111 insertions(+), 6 deletions(-)
>
> diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
> index 230deec..9f59905 100644
> --- a/man/man1/notmuch-dump.1
> +++ b/man/man1/notmuch-dump.1
> @@ -5,6 +5,7 @@ notmuch-dump \- creates a plain-text dump of the tags of each message
>  .SH SYNOPSIS
>  
>  .B "notmuch dump"
> +.RB  [ "\-\-format=(sup|batch-tag)"  "] [--]"
>  .RI "[ --output=<" filename "> ] [--]"
>  .RI "[ <" search-term ">...]"
>  
> @@ -19,6 +20,63 @@ recreated from the messages themselves.  The output of notmuch dump is
>  therefore the only critical thing to backup (and much more friendly to
>  incremental backup than the native database files.)
>  
> +.TP 4
> +.B \-\-format=(sup|batch-tag)
> +
> +Notmuch restore supports two plain text dump formats, both with one message-id
> +per line, followed by a list of tags.

"followed by tags" is not entirely accurate for batch-tag.

> +
> +.RS 4
> +.TP 4
> +.B sup
> +
> +The
> +.B sup
> +dump file format is specifically chosen to be
> +compatible with the format of files produced by sup-dump.
> +So if you've previously been using sup for mail, then the
> +.B "notmuch restore"
> +command provides you a way to import all of your tags (or labels as
> +sup calls them).
> +Each line has the following form

Should we deprecate the sup format for new dumps at the same time? Issue
a warning message on dumping too (unless --format is explicitly
specified), telling about the new batch-tag format. I think we should
eventually make batch-tag the default.

> +
> +.RS 4
> +.RI < message-id >
> +.B (
> +.RI < tag "> ..."
> +.B )
> +
> +with zero or more tags are separated by spaces. Note that (malformed)
> +message-ids may contain arbitrary non-null characters. Note also
> +that tags with spaces will not be correctly restored with this format.
> +
> +.RE
> +
> +.RE
> +.RS 4
> +.TP 4
> +.B batch-tag
> +
> +The
> +.B batch-tag
> +dump format is intended to more robust against malformed message-ids
> +and tags containing whitespace or non-\fBascii\fR(7) characters.
> +Each line has the form
> +
> +.RS 4
> +.RI "+<" "encoded-tag" "> " "" "+<" "encoded-tag" "> ... -- " "" " <" encoded-message-id >

Mention that the id: prefix is present in dump and required in restore.

BR,
Jani.

> +
> +where encoded means that every byte not matching the regex
> +.B [A-Za-z0-9+-_@=.:,]
> +is replace by
> +.B %nn
> +where nn is the two digit hex encoding.
> +The astute reader will notice this is a special case of the batch input
> +format for \fBnotmuch-tag\fR(1).
> +
> +.RE
> +
> +
>  With no search terms, a dump of all messages in the database will be
>  generated.  A "--" argument instructs notmuch that the
>  remaining arguments are search terms.
> diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
> index 2fa8733..3860829 100644
> --- a/man/man1/notmuch-restore.1
> +++ b/man/man1/notmuch-restore.1
> @@ -6,6 +6,7 @@ notmuch-restore \- restores the tags from the given file (see notmuch dump)
>  
>  .B "notmuch restore"
>  .RB [ "--accumulate" ]
> +.RB [ "--format=(auto|batch-tag|sup)" ]
>  .RI "[ --input=<" filename "> ]"
>  
>  .SH DESCRIPTION
> @@ -15,19 +16,51 @@ Restores the tags from the given file (see
>  
>  The input is read from the given filename, if any, or from stdin.
>  
> -Note: The dump file format is specifically chosen to be
> +
> +Supported options for
> +.B restore
> +include
> +.RS 4
> +.TP 4
> +.B \-\-accumulate
> +
> +The union of the existing and new tags is applied, instead of
> +replacing each message's tags as they are read in from the dump file.
> +
> +.RE
> +.RS 4
> +.TP 4
> +.B \-\-format=(sup|batch-tag|auto)
> +
> +Notmuch restore supports two plain text dump formats, with one message-id
> +per line, and a list of tags.
> +For details of the actual formats, see \fBnotmuch-dump\fR(1).
> +
> +.RS 4
> +.TP 4
> +.B sup
> +
> +The
> +.B sup
> +dump file format is specifically chosen to be
>  compatible with the format of files produced by sup-dump.
>  So if you've previously been using sup for mail, then the
>  .B "notmuch restore"
>  command provides you a way to import all of your tags (or labels as
>  sup calls them).
>  
> -The --accumulate switch causes the union of the existing and new tags to be
> -applied, instead of replacing each message's tags as they are read in from the
> -dump file.
> +.RE
> +.RS 4
> +.TP 4
> +.B batch-tag
>  
> -See \fBnotmuch-search-terms\fR(7)
> -for details of the supported syntax for <search-terms>.
> +The
> +.B batch-tag
> +dump format is intended to more robust against malformed message-ids
> +and tags containing whitespace or non-\fBascii\fR(7) characters.  This
> +format hex-escapes all characters those outside of a small character
> +set, intended to be suitable for e.g. pathnames in most UNIX-like
> +systems.
>  
>  .B "notmuch restore"
>  updates the maildir flags according to tag changes if the
> @@ -36,6 +69,20 @@ configuration option is enabled. See \fBnotmuch-config\fR(1) for
>  details.
>  
>  .RE
> +
> +.RS 4
> +.TP 4
> +.B auto
> +
> +This option (the default) tries to guess the format from the
> +input. For correctly formed input in either supported format, this
> +heuristic, based the fact that batch-tag format contains no parentheses,
> +should be accurate.
> +
> +.RE
> +
> +.RE
> +
>  .SH SEE ALSO
>  
>  \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch