[PATCH] cli/show: add --format=pretty

[PATCH] cli/show: add --format=pretty

[Hannu Hartikainen]

The rationale for this feature is twofold:

1. It is useful to be able to view messages in as human-friendly format
   as possible.
2. The same format should still be machine-readable, too.

The email format is mostly human-readable as is. The things difficult
for a human eye are the huge amount of headers that are common these
days and telling different messages apart when there are many.

This commit adds the display format `pretty`. It recognizes if the
output is a TTY and if so, applies coloring to headers. This turns out
to be a good visual message separator. Additionally it only shows the
most important headers, similarly to `--format=text`, and only displays
plaintext parts (ie. text/* but not text/html).

While human readability is the main goal, another design goal was that
piping the output to `git am` works, at least for individual messages
sent with `git send-email`.
---

I'm a new Notmuch user and have been trying out different MUAs. And
reading email directly with Notmuch feels easier to me than adding to my
cognitive load with something like Mutt. At least when I want to read
something specific that Notmuch queries are well suited for.
            
This is my first (but possibly not last) patch aimed at enhancing the
usability of the Notmuch CLI. Any and all feedback is welcome!

 NEWS                               |  7 ++++
 completion/notmuch-completion.bash |  2 +-
 completion/zsh/_notmuch            |  2 +-
 doc/man1/notmuch-show.rst          |  8 +++-
 notmuch-show.c                     | 65 ++++++++++++++++++++++++++++++
 5 files changed, 81 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 538ec168..d099bd69 100644
--- a/NEWS
+++ b/NEWS
@@ -1,6 +1,13 @@
 Notmuch 0.33 (UNRELEASED)
 =========================
 
+CLI
+---
+
+`notmuch show` now has `--format=pretty`, optimized for reading plain
+text emails on the command line. It only shows the most important
+headers and plain text parts and uses colors for headers.
+
 Emacs
 -----
 
diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash
index 15425697..86cbbcdc 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -514,7 +514,7 @@ _notmuch_show()
 	    return
 	    ;;
 	--format)
-	    COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
+	    COMPREPLY=( $( compgen -W "text pretty json sexp mbox raw" -- "${cur}" ) )
 	    return
 	    ;;
 	--exclude|--body)
diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
index e920f10b..5cc386e2 100644
--- a/completion/zsh/_notmuch
+++ b/completion/zsh/_notmuch
@@ -237,7 +237,7 @@ _notmuch_search() {
 _notmuch_show() {
   _arguments -S \
     '--entire-thread=[output entire threads]:show thread:(true false)' \
-    '--format=[set output format]:output format:(text json sexp mbox raw)' \
+    '--format=[set output format]:output format:(text pretty json sexp mbox raw)' \
     '--format-version=[set output format version]:format version: ' \
     '--part=[output a single decoded mime part]:part number: ' \
     '--verify[verify signed MIME parts]' \
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index fc6bec62..1fe4dcc7 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -34,7 +34,7 @@ Supported options for **show** include
    the matching messages. For ``--format=json`` and ``--format=sexp``
    this defaults to true. For other formats, this defaults to false.
 
-.. option:: --format=(text|json|sexp|mbox|raw)
+.. option:: --format=(text|pretty|json|sexp|mbox|raw)
 
    **text** (default for messages)
      The default plain-text format has all text-content MIME parts
@@ -46,6 +46,12 @@ Supported options for **show** include
      '}'), to either open or close the component. For a multipart
      MIME message, these parts will be nested.
 
+   **pretty**
+     The plain-text parts of all matching messages are printed in a
+     format optimized for readability. Only the most important
+     headers are displayed. If the output is to a TTY, the headers
+     are colored.
+
    **json**
      The output is formatted with Javascript Object Notation
      (JSON). This format is more robust than the text format for
diff --git a/notmuch-show.c b/notmuch-show.c
index bdb87321..fe3b753e 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -601,6 +601,63 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
     return NOTMUCH_STATUS_SUCCESS;
 }
 
+static notmuch_status_t
+format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
+		    int indent, const notmuch_show_params_t *params)
+{
+    /* The disposition and content-type metadata are associated with
+     * the envelope for message parts */
+    GMimeObject *meta = node->envelope_part ? (
+	GMIME_OBJECT (node->envelope_part) ) : node->part;
+    GMimeContentType *content_type = g_mime_object_get_content_type (meta);
+    GMimeStream *stream = params->out_stream;
+    int i;
+    bool color = isatty (fileno (stdout));
+
+    if (GMIME_IS_MESSAGE (node->part)) {
+	GMimeMessage *message = GMIME_MESSAGE (node->part);
+	char *recipients_string;
+	char *date_string;
+
+	if (color)
+	    g_mime_stream_printf (stream, "\e[36m");
+	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[33m");
+	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
+	if (recipients_string) {
+	    if (color)
+		g_mime_stream_printf (stream, "\e[31m");
+	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
+	}
+	g_free (recipients_string);
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_CC);
+	if (recipients_string) {
+	    if (color)
+		g_mime_stream_printf (stream, "\e[31m");
+	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
+	}
+	g_free (recipients_string);
+	date_string = g_mime_message_get_date_string (node, message);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[35m");
+	g_mime_stream_printf (stream, "Date: %s\n\n", date_string);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[0m");
+    }
+
+    if (GMIME_IS_PART (node->part) &&
+	g_mime_content_type_is_type (content_type, "text", "*") &&
+	! g_mime_content_type_is_type (content_type, "text", "html"))
+	show_text_part_content (node->part, stream, 0);
+
+    for (i = 0; i < node->nchildren; i++)
+	format_part_pretty (ctx, sp, mime_node_child (node, i), indent, params);
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static void
 format_omitted_part_meta_sprinter (sprinter_t *sp, GMimeObject *meta, GMimePart *part)
 {
@@ -1187,6 +1244,7 @@ enum {
     NOTMUCH_FORMAT_JSON,
     NOTMUCH_FORMAT_SEXP,
     NOTMUCH_FORMAT_TEXT,
+    NOTMUCH_FORMAT_PRETTY,
     NOTMUCH_FORMAT_MBOX,
     NOTMUCH_FORMAT_RAW
 };
@@ -1206,6 +1264,11 @@ static const notmuch_show_format_t format_text = {
     .part = format_part_text,
 };
 
+static const notmuch_show_format_t format_pretty = {
+    .new_sprinter = sprinter_text_create,
+    .part = format_part_pretty,
+};
+
 static const notmuch_show_format_t format_mbox = {
     .new_sprinter = sprinter_text_create,
     .part = format_part_mbox,
@@ -1220,6 +1283,7 @@ static const notmuch_show_format_t *formatters[] = {
     [NOTMUCH_FORMAT_JSON] = &format_json,
     [NOTMUCH_FORMAT_SEXP] = &format_sexp,
     [NOTMUCH_FORMAT_TEXT] = &format_text,
+    [NOTMUCH_FORMAT_PRETTY] = &format_pretty,
     [NOTMUCH_FORMAT_MBOX] = &format_mbox,
     [NOTMUCH_FORMAT_RAW] = &format_raw,
 };
@@ -1249,6 +1313,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	{ .opt_keyword = &format, .name = "format", .keywords =
 	      (notmuch_keyword_t []){ { "json", NOTMUCH_FORMAT_JSON },
 				      { "text", NOTMUCH_FORMAT_TEXT },
+				      { "pretty", NOTMUCH_FORMAT_PRETTY },
 				      { "sexp", NOTMUCH_FORMAT_SEXP },
 				      { "mbox", NOTMUCH_FORMAT_MBOX },
 				      { "raw", NOTMUCH_FORMAT_RAW },
-- 
2.32.0

[PATCH] cli/show: add --color/--no-color

[Hannu Hartikainen]

These arguments allow setting color explicitly on or off when using
`--format=pretty`. The default is still to use color iff stdout is a
TTY.
---

I got a reply off-list to my previous patch (cli/show: add
--format=pretty) about explicitly enabling color (for use with $PAGER).
It's quite a minor addition so I just implemented it. IMHO this warrants
its own commit; it's a separate feature from --format=pretty. But I'm ok
with squashing these together if that's considered better.

Of course, with an explicit --color command-line argument in one command
for one purpose, one could reasonably ask if other commands should
support colored output as well. What do you all think?

 doc/man1/notmuch-show.rst | 6 ++++++
 notmuch-client.h          | 1 +
 notmuch-show.c            | 4 +++-
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 1fe4dcc7..e61000a6 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -209,6 +209,12 @@ Supported options for **show** include
    "text/html" parts, no part with content type "text/html" is included
    in the output.
 
+.. option:: --color
+
+   Use colored output for formats that support that (currently only
+   ``--format=pretty``). By default, color is used if the output
+   goes to a TTY. To disable color, use ``--no-color``.
+
 A common use of **notmuch show** is to display a single thread of
 email messages. For this, use a search term of "thread:<thread-id>" as
 can be seen in the first column of output from the
diff --git a/notmuch-client.h b/notmuch-client.h
index 8227fea4..c4f7cfd6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -78,6 +78,7 @@ typedef struct notmuch_show_params {
     int part;
     _notmuch_crypto_t crypto;
     bool include_html;
+    bool color;
     GMimeStream *out_stream;
 } notmuch_show_params_t;
 
diff --git a/notmuch-show.c b/notmuch-show.c
index fe3b753e..6274a99a 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -612,7 +612,7 @@ format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
     GMimeContentType *content_type = g_mime_object_get_content_type (meta);
     GMimeStream *stream = params->out_stream;
     int i;
-    bool color = isatty (fileno (stdout));
+    bool color = params->color;
 
     if (GMIME_IS_MESSAGE (node->part)) {
 	GMimeMessage *message = GMIME_MESSAGE (node->part);
@@ -1300,6 +1300,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	.part = -1,
 	.omit_excluded = true,
 	.output_body = true,
+	.color = isatty (fileno (stdout)),
 	.crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
     };
     int format = NOTMUCH_FORMAT_NOT_SPECIFIED;
@@ -1334,6 +1335,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	{ .opt_bool = &params.crypto.verify, .name = "verify" },
 	{ .opt_bool = &params.output_body, .name = "body" },
 	{ .opt_bool = &params.include_html, .name = "include-html" },
+	{ .opt_bool = &params.color, .name = "color" },
 	{ .opt_inherit = notmuch_shared_options },
 	{ }
     };
-- 
2.32.0

[PATCH v2] cli/show: add --format=pretty

[Hannu Hartikainen]

The rationale for this feature is twofold:

1. It is useful to be able to view messages in as human-friendly format
   as possible.
2. The same format should still be machine-readable, too.

The email format is mostly human-readable as is. The things difficult
for a human eye are the huge amount of headers that are common these
days and telling different messages apart when there are many.

This commit adds the display format `pretty`. It recognizes if the
output is a TTY and if so, applies coloring to headers. This turns out
to be a good visual message separator. Additionally it only shows the
most important headers, similarly to `--format=text`, and only displays
plaintext parts (ie. text/* but not text/html).

While human readability is the main goal, another design goal was that
piping the output to `git am` works, at least for individual messages
sent with `git send-email`.
---

I wrote a v2 of this patch. I've been dogfooding for a while now and
wanted a couple of enhancements, and also had learned about the notmuch
test harness. The differences to the first version are:

- add a unit test
- show Message-ID, making replying etc. much easier
- print a newline after each part, which helps a lot with messages that
  do not end in a newline

I'm using this as a daily driver and am happy with it.

 NEWS                               |  7 ++++
 completion/notmuch-completion.bash |  2 +-
 completion/zsh/_notmuch            |  2 +-
 doc/man1/notmuch-show.rst          |  8 +++-
 notmuch-show.c                     | 67 ++++++++++++++++++++++++++++++
 test/T520-show.sh                  | 32 ++++++++++++++
 6 files changed, 115 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 3e776009..f5142ff1 100644
--- a/NEWS
+++ b/NEWS
@@ -1,6 +1,13 @@
 Notmuch 0.33 (UNRELEASED)
 =========================
 
+CLI
+---
+
+`notmuch show` now has `--format=pretty`, optimized for reading plain
+text emails on the command line. It only shows the most important
+headers and plain text parts and uses colors for headers.
+
 Emacs
 -----
 
diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash
index 15425697..86cbbcdc 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -514,7 +514,7 @@ _notmuch_show()
 	    return
 	    ;;
 	--format)
-	    COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
+	    COMPREPLY=( $( compgen -W "text pretty json sexp mbox raw" -- "${cur}" ) )
 	    return
 	    ;;
 	--exclude|--body)
diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
index e920f10b..5cc386e2 100644
--- a/completion/zsh/_notmuch
+++ b/completion/zsh/_notmuch
@@ -237,7 +237,7 @@ _notmuch_search() {
 _notmuch_show() {
   _arguments -S \
     '--entire-thread=[output entire threads]:show thread:(true false)' \
-    '--format=[set output format]:output format:(text json sexp mbox raw)' \
+    '--format=[set output format]:output format:(text pretty json sexp mbox raw)' \
     '--format-version=[set output format version]:format version: ' \
     '--part=[output a single decoded mime part]:part number: ' \
     '--verify[verify signed MIME parts]' \
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index fc6bec62..1fe4dcc7 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -34,7 +34,7 @@ Supported options for **show** include
    the matching messages. For ``--format=json`` and ``--format=sexp``
    this defaults to true. For other formats, this defaults to false.
 
-.. option:: --format=(text|json|sexp|mbox|raw)
+.. option:: --format=(text|pretty|json|sexp|mbox|raw)
 
    **text** (default for messages)
      The default plain-text format has all text-content MIME parts
@@ -46,6 +46,12 @@ Supported options for **show** include
      '}'), to either open or close the component. For a multipart
      MIME message, these parts will be nested.
 
+   **pretty**
+     The plain-text parts of all matching messages are printed in a
+     format optimized for readability. Only the most important
+     headers are displayed. If the output is to a TTY, the headers
+     are colored.
+
    **json**
      The output is formatted with Javascript Object Notation
      (JSON). This format is more robust than the text format for
diff --git a/notmuch-show.c b/notmuch-show.c
index 232557d5..c417ec00 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -606,6 +606,65 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
     return NOTMUCH_STATUS_SUCCESS;
 }
 
+static notmuch_status_t
+format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
+		    int indent, const notmuch_show_params_t *params)
+{
+    /* The disposition and content-type metadata are associated with
+     * the envelope for message parts */
+    GMimeObject *meta = node->envelope_part ? (
+	GMIME_OBJECT (node->envelope_part) ) : node->part;
+    GMimeContentType *content_type = g_mime_object_get_content_type (meta);
+    GMimeStream *stream = params->out_stream;
+    int i;
+    bool color = isatty (fileno (stdout));
+
+    if (GMIME_IS_MESSAGE (node->part)) {
+	GMimeMessage *message = GMIME_MESSAGE (node->part);
+	char *recipients_string;
+	char *date_string;
+
+	if (color)
+	    g_mime_stream_printf (stream, "\e[36m");
+	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[33m");
+	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[31m");
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
+	if (recipients_string)
+	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
+	g_free (recipients_string);
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_CC);
+	if (recipients_string)
+	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
+	g_free (recipients_string);
+	date_string = g_mime_message_get_date_string (node, message);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[35m");
+	g_mime_stream_printf (stream, "Date: %s\n", date_string);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[32m");
+	g_mime_stream_printf (stream, "Message-ID: <%s>\n\n", g_mime_message_get_message_id (
+				  message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[0m");
+    }
+
+    if (GMIME_IS_PART (node->part) &&
+	g_mime_content_type_is_type (content_type, "text", "*") &&
+	! g_mime_content_type_is_type (content_type, "text", "html")) {
+	show_text_part_content (node->part, stream, 0);
+	g_mime_stream_printf (stream, "\n");
+    }
+
+    for (i = 0; i < node->nchildren; i++)
+	format_part_pretty (ctx, sp, mime_node_child (node, i), indent, params);
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static void
 format_omitted_part_meta_sprinter (sprinter_t *sp, GMimeObject *meta, GMimePart *part)
 {
@@ -1192,6 +1251,7 @@ enum {
     NOTMUCH_FORMAT_JSON,
     NOTMUCH_FORMAT_SEXP,
     NOTMUCH_FORMAT_TEXT,
+    NOTMUCH_FORMAT_PRETTY,
     NOTMUCH_FORMAT_MBOX,
     NOTMUCH_FORMAT_RAW
 };
@@ -1211,6 +1271,11 @@ static const notmuch_show_format_t format_text = {
     .part = format_part_text,
 };
 
+static const notmuch_show_format_t format_pretty = {
+    .new_sprinter = sprinter_text_create,
+    .part = format_part_pretty,
+};
+
 static const notmuch_show_format_t format_mbox = {
     .new_sprinter = sprinter_text_create,
     .part = format_part_mbox,
@@ -1225,6 +1290,7 @@ static const notmuch_show_format_t *formatters[] = {
     [NOTMUCH_FORMAT_JSON] = &format_json,
     [NOTMUCH_FORMAT_SEXP] = &format_sexp,
     [NOTMUCH_FORMAT_TEXT] = &format_text,
+    [NOTMUCH_FORMAT_PRETTY] = &format_pretty,
     [NOTMUCH_FORMAT_MBOX] = &format_mbox,
     [NOTMUCH_FORMAT_RAW] = &format_raw,
 };
@@ -1254,6 +1320,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	{ .opt_keyword = &format, .name = "format", .keywords =
 	      (notmuch_keyword_t []){ { "json", NOTMUCH_FORMAT_JSON },
 				      { "text", NOTMUCH_FORMAT_TEXT },
+				      { "pretty", NOTMUCH_FORMAT_PRETTY },
 				      { "sexp", NOTMUCH_FORMAT_SEXP },
 				      { "mbox", NOTMUCH_FORMAT_MBOX },
 				      { "raw", NOTMUCH_FORMAT_RAW },
diff --git a/test/T520-show.sh b/test/T520-show.sh
index 16222650..e555b284 100755
--- a/test/T520-show.sh
+++ b/test/T520-show.sh
@@ -10,4 +10,36 @@ notmuch show foo..
 exit_code=$?
 test_expect_equal 1 $exit_code
 
+test_begin_subtest "--format=pretty"
+output=$(notmuch show --format=pretty id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com 2>&1 && echo OK)
+test_expect_equal "$output" "Subject: [notmuch] preliminary FreeBSD support
+From: Alex Botero-Lowry <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 11:36:14 -0800
+Message-ID: <cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com>
+
+I saw the announcement this morning, and was very excited, as I had been
+hoping sup would be turned into a library,
+since I like the concept more than the UI (I'd rather an emacs interface).
+
+I did a preliminary compile which worked out fine, but
+sysconf(_SC_SC_GETPW_R_SIZE_MAX) returns -1 on
+FreeBSD, so notmuch_config_open segfaulted.
+
+Attached is a patch that supplies a default buffer size of 64 in cases where
+-1 is returned.
+
+http://www.opengroup.org/austin/docs/austin_328.txt - seems to indicate this
+is acceptable behavior,
+and http://mail-index.netbsd.org/pkgsrc-bugs/2006/06/07/msg016808.htmlspecifically
+uses 64 as the
+buffer size.
+
+_______________________________________________
+notmuch mailing list
+notmuch@notmuchmail.org
+http://notmuchmail.org/mailman/listinfo/notmuch
+
+OK"
+
 test_done
-- 
2.32.0

Re: [PATCH v2] cli/show: add --format=pretty

[Tomi Ollila]

On Fri, Jul 02 2021, Hannu Hartikainen wrote:

> The rationale for this feature is twofold:
>
> 1. It is useful to be able to view messages in as human-friendly format
>    as possible.
> 2. The same format should still be machine-readable, too.
>
> The email format is mostly human-readable as is. The things difficult
> for a human eye are the huge amount of headers that are common these
> days and telling different messages apart when there are many.
>
> This commit adds the display format `pretty`. It recognizes if the
> output is a TTY and if so, applies coloring to headers. This turns out
> to be a good visual message separator. Additionally it only shows the
> most important headers, similarly to `--format=text`, and only displays
> plaintext parts (ie. text/* but not text/html).
>
> While human readability is the main goal, another design goal was that
> piping the output to `git am` works, at least for individual messages
> sent with `git send-email`.
> ---
>
> I wrote a v2 of this patch. I've been dogfooding for a while now and
> wanted a couple of enhancements, and also had learned about the notmuch
> test harness. The differences to the first version are:
>
> - add a unit test
> - show Message-ID, making replying etc. much easier
> - print a newline after each part, which helps a lot with messages that
>   do not end in a newline
>
> I'm using this as a daily driver and am happy with it.
>
>  NEWS                               |  7 ++++
>  completion/notmuch-completion.bash |  2 +-
>  completion/zsh/_notmuch            |  2 +-
>  doc/man1/notmuch-show.rst          |  8 +++-
>  notmuch-show.c                     | 67 ++++++++++++++++++++++++++++++
>  test/T520-show.sh                  | 32 ++++++++++++++
>  6 files changed, 115 insertions(+), 3 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 3e776009..f5142ff1 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,6 +1,13 @@
>  Notmuch 0.33 (UNRELEASED)
>  =========================
>  
> +CLI
> +---
> +
> +`notmuch show` now has `--format=pretty`, optimized for reading plain
> +text emails on the command line. It only shows the most important
> +headers and plain text parts and uses colors for headers.
> +
>  Emacs
>  -----
>  
> diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash
> index 15425697..86cbbcdc 100644
> --- a/completion/notmuch-completion.bash
> +++ b/completion/notmuch-completion.bash
> @@ -514,7 +514,7 @@ _notmuch_show()
>  	    return
>  	    ;;
>  	--format)
> -	    COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
> +	    COMPREPLY=( $( compgen -W "text pretty json sexp mbox raw" -- "${cur}" ) )
>  	    return
>  	    ;;
>  	--exclude|--body)
> diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
> index e920f10b..5cc386e2 100644
> --- a/completion/zsh/_notmuch
> +++ b/completion/zsh/_notmuch
> @@ -237,7 +237,7 @@ _notmuch_search() {
>  _notmuch_show() {
>    _arguments -S \
>      '--entire-thread=[output entire threads]:show thread:(true false)' \
> -    '--format=[set output format]:output format:(text json sexp mbox raw)' \
> +    '--format=[set output format]:output format:(text pretty json sexp mbox raw)' \
>      '--format-version=[set output format version]:format version: ' \
>      '--part=[output a single decoded mime part]:part number: ' \
>      '--verify[verify signed MIME parts]' \
> diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
> index fc6bec62..1fe4dcc7 100644
> --- a/doc/man1/notmuch-show.rst
> +++ b/doc/man1/notmuch-show.rst
> @@ -34,7 +34,7 @@ Supported options for **show** include
>     the matching messages. For ``--format=json`` and ``--format=sexp``
>     this defaults to true. For other formats, this defaults to false.
>  
> -.. option:: --format=(text|json|sexp|mbox|raw)
> +.. option:: --format=(text|pretty|json|sexp|mbox|raw)
>  
>     **text** (default for messages)
>       The default plain-text format has all text-content MIME parts
> @@ -46,6 +46,12 @@ Supported options for **show** include
>       '}'), to either open or close the component. For a multipart
>       MIME message, these parts will be nested.
>  
> +   **pretty**
> +     The plain-text parts of all matching messages are printed in a
> +     format optimized for readability. Only the most important
> +     headers are displayed. If the output is to a TTY, the headers
> +     are colored.
> +
>     **json**
>       The output is formatted with Javascript Object Notation
>       (JSON). This format is more robust than the text format for
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 232557d5..c417ec00 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -606,6 +606,65 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
>      return NOTMUCH_STATUS_SUCCESS;
>  }
>  
> +static notmuch_status_t
> +format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
> +		    int indent, const notmuch_show_params_t *params)
> +{
> +    /* The disposition and content-type metadata are associated with
> +     * the envelope for message parts */
> +    GMimeObject *meta = node->envelope_part ? (
> +	GMIME_OBJECT (node->envelope_part) ) : node->part;
> +    GMimeContentType *content_type = g_mime_object_get_content_type (meta);
> +    GMimeStream *stream = params->out_stream;
> +    int i;
> +    bool color = isatty (fileno (stdout));
> +
> +    if (GMIME_IS_MESSAGE (node->part)) {
> +	GMimeMessage *message = GMIME_MESSAGE (node->part);
> +	char *recipients_string;
> +	char *date_string;
> +
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[36m");

The code looks good to me, just that these "hardcoded" color values gives
me a bit of suspicion...

When running 

$ bash -c 'printf "\e[31m 31 \e[32m 32 \e[33m 33 \e[35m 35 \e[36m 36\n"'

(use \033 instead of \e with dash(1))

the colors look OK when background is dark (black). on light background
(white) color 33 (yellow), color 36 (cyan) and color 32 (green) are
somewhat hard to read (in decreasing hardness)...

I did not try to apply this and see how those looks like when displaying
real emails (no setup on this machine for now...)

Another thing is terminal support -- what if terminal did not have support
for these escape sequences. Surprisingly that color testing worked OK
in emacs shell (which claimed $TERM as 'dumb')

if the option were '--ugly' -- then I would not have had any comments >;)

One option would be to first send this without color support and then
add *configurable* color support -- I don't know which way is better
but as a rewiever that would be easier to accept...

Tomi

> +	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[33m");
> +	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[31m");
> +	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
> +	if (recipients_string)
> +	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
> +	g_free (recipients_string);
> +	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_CC);
> +	if (recipients_string)
> +	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
> +	g_free (recipients_string);
> +	date_string = g_mime_message_get_date_string (node, message);
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[35m");
> +	g_mime_stream_printf (stream, "Date: %s\n", date_string);
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[32m");
> +	g_mime_stream_printf (stream, "Message-ID: <%s>\n\n", g_mime_message_get_message_id (
> +				  message));
> +	if (color)
> +	    g_mime_stream_printf (stream, "\e[0m");
> +    }
> +
> +    if (GMIME_IS_PART (node->part) &&
> +	g_mime_content_type_is_type (content_type, "text", "*") &&
> +	! g_mime_content_type_is_type (content_type, "text", "html")) {
> +	show_text_part_content (node->part, stream, 0);
> +	g_mime_stream_printf (stream, "\n");
> +    }
> +
> +    for (i = 0; i < node->nchildren; i++)
> +	format_part_pretty (ctx, sp, mime_node_child (node, i), indent, params);
> +
> +    return NOTMUCH_STATUS_SUCCESS;
> +}
> +
>  static void
>  format_omitted_part_meta_sprinter (sprinter_t *sp, GMimeObject *meta, GMimePart *part)
>  {
> @@ -1192,6 +1251,7 @@ enum {
>      NOTMUCH_FORMAT_JSON,
>      NOTMUCH_FORMAT_SEXP,
>      NOTMUCH_FORMAT_TEXT,
> +    NOTMUCH_FORMAT_PRETTY,
>      NOTMUCH_FORMAT_MBOX,
>      NOTMUCH_FORMAT_RAW
>  };
> @@ -1211,6 +1271,11 @@ static const notmuch_show_format_t format_text = {
>      .part = format_part_text,
>  };
>  
> +static const notmuch_show_format_t format_pretty = {
> +    .new_sprinter = sprinter_text_create,
> +    .part = format_part_pretty,
> +};
> +
>  static const notmuch_show_format_t format_mbox = {
>      .new_sprinter = sprinter_text_create,
>      .part = format_part_mbox,
> @@ -1225,6 +1290,7 @@ static const notmuch_show_format_t *formatters[] = {
>      [NOTMUCH_FORMAT_JSON] = &format_json,
>      [NOTMUCH_FORMAT_SEXP] = &format_sexp,
>      [NOTMUCH_FORMAT_TEXT] = &format_text,
> +    [NOTMUCH_FORMAT_PRETTY] = &format_pretty,
>      [NOTMUCH_FORMAT_MBOX] = &format_mbox,
>      [NOTMUCH_FORMAT_RAW] = &format_raw,
>  };
> @@ -1254,6 +1320,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
>  	{ .opt_keyword = &format, .name = "format", .keywords =
>  	      (notmuch_keyword_t []){ { "json", NOTMUCH_FORMAT_JSON },
>  				      { "text", NOTMUCH_FORMAT_TEXT },
> +				      { "pretty", NOTMUCH_FORMAT_PRETTY },
>  				      { "sexp", NOTMUCH_FORMAT_SEXP },
>  				      { "mbox", NOTMUCH_FORMAT_MBOX },
>  				      { "raw", NOTMUCH_FORMAT_RAW },
> diff --git a/test/T520-show.sh b/test/T520-show.sh
> index 16222650..e555b284 100755
> --- a/test/T520-show.sh
> +++ b/test/T520-show.sh
> @@ -10,4 +10,36 @@ notmuch show foo..
>  exit_code=$?
>  test_expect_equal 1 $exit_code
>  
> +test_begin_subtest "--format=pretty"
> +output=$(notmuch show --format=pretty id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com 2>&1 && echo OK)
> +test_expect_equal "$output" "Subject: [notmuch] preliminary FreeBSD support
> +From: Alex Botero-Lowry <alex.boterolowry@gmail.com>
> +To: notmuch@notmuchmail.org
> +Date: Tue, 17 Nov 2009 11:36:14 -0800
> +Message-ID: <cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com>
> +
> +I saw the announcement this morning, and was very excited, as I had been
> +hoping sup would be turned into a library,
> +since I like the concept more than the UI (I'd rather an emacs interface).
> +
> +I did a preliminary compile which worked out fine, but
> +sysconf(_SC_SC_GETPW_R_SIZE_MAX) returns -1 on
> +FreeBSD, so notmuch_config_open segfaulted.
> +
> +Attached is a patch that supplies a default buffer size of 64 in cases where
> +-1 is returned.
> +
> +http://www.opengroup.org/austin/docs/austin_328.txt - seems to indicate this
> +is acceptable behavior,
> +and http://mail-index.netbsd.org/pkgsrc-bugs/2006/06/07/msg016808.htmlspecifically
> +uses 64 as the
> +buffer size.
> +
> +_______________________________________________
> +notmuch mailing list
> +notmuch@notmuchmail.org
> +http://notmuchmail.org/mailman/listinfo/notmuch
> +
> +OK"
> +
>  test_done
> -- 
> 2.32.0
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

Re: [PATCH v2] cli/show: add --format=pretty

[Hannu Hartikainen]

Thanks for the review!

On Sun, 04 Jul 2021 12:40:59 +0300, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> The code looks good to me, just that these "hardcoded" color values gives
> me a bit of suspicion...

That's true and I do think the colors could eventually be configurable,
but right now I don't know how the configuration should work. Should it
be a palette, should each header have a configurable color (and should
the displayed headers be configurable too) etc. It's easy to make this
hugely bloated so I'll avoid going there until I have a clear idea of a
good, minimal approach.

> the colors look OK when background is dark (black). on light background
> (white) color 33 (yellow), color 36 (cyan) and color 32 (green) are
> somewhat hard to read (in decreasing hardness)...

Most terminals have user-configurable colors so I don't think this is a
showstopper. But as I said, yes, it would be best to make the colors
configurable.

> I did not try to apply this and see how those looks like when displaying
> real emails (no setup on this machine for now...)

Here's a screenshot: https://i.imgur.com/7dlP7lt.png

I'm happy to receive any suggestions but I think it's best to stay with
the original 3-bit spec (SGR codes 30-37, or in practice 31-36: red,
green, yellow, blue, magenta, cyan) if the values are hardcoded. Those
are most widely supported.

> One option would be to first send this without color support and then
> add *configurable* color support -- I don't know which way is better
> but as a rewiever that would be easier to accept...

That's an excellent idea! I'll post PATCH v3 later. I don't really want
to try real feature detection because some terminals have weird corner
cases with their ANSI sequence support. `isatty` is what other CLI tools
tend to use as default AFAIK.

Hannu

Re: [PATCH] cli/show: add --format=pretty

[David Bremner]

Hannu Hartikainen <hannu@hrtk.in> writes:

> The rationale for this feature is twofold:
>
> 1. It is useful to be able to view messages in as human-friendly format
>    as possible.
> 2. The same format should still be machine-readable, too.

This is not really directed at Hannu, but at the notmuch community. As
you can imagine I'm not super enthusiastic an every growing number of
output formats to maintain.

The usual argument for keeping --format=text is that it useful for
scripting. There is also the vim UI, but afaict that is no longer using
--format=text.  It would be nice if a new text like format could
(eventually) replace the old one. So what would the new format need to
do so that we could at least deprecate the old one?

One thing the old format did not do, but a generically useful on the
command-line format probably should is deal with signature verification
and decryption. There is obviously potential for visual spoofing, but
maybe color can help.

>
> While human readability is the main goal, another design goal was that
> piping the output to `git am` works, at least for individual messages
> sent with `git send-email`.
> ---

In my experience, notmuch show --format=raw works pretty well
for this. There was an issue with encoded line endings but that is fixed
in git 2.32. What advantage does this new format bring for patches?

d

Re: [PATCH] cli/show: add --format=pretty

[Hannu Hartikainen]

Hi!

Thanks for starting discussion on the matter. Do feel free to tell me if
I have the completely wrong idea about project goals.

On Sun, 04 Jul 2021 11:38:45 -0300, David Bremner <david@tethera.net> wrote:
> This is not really directed at Hannu, but at the notmuch community. As
> you can imagine I'm not super enthusiastic an every growing number of
> output formats to maintain.

I can appreciate that. I've maintained open source software before and I
know people come up with the weirdest feature requests that simply don't
fit the scenario I'm building the software for. If this text format that
I personally like to use isn't good for the project, it definitely
should not be merged.

What do you see as the mission statement for notmuch-cli? I'd like to
make it ergonomic enough to be usable without a MUA, and it's really
close already. But if notmuch-cli is meant to be something completely
different I might just have my own set of patches or consider starting
my own project.

> One thing the old format did not do, but a generically useful on the
> command-line format probably should is deal with signature verification
> and decryption. There is obviously potential for visual spoofing, but
> maybe color can help.

I'm pretty sure you can embed ^[[31mANSI escapes^[[0m in email and
they'll be displayed by `notmuch show` as color in a typical terminal.
Not sure if anyone should be worried about attacks specifically against
notmuch users, though.

> In my experience, notmuch show --format=raw works pretty well
> for this. There was an issue with encoded line endings but that is fixed
> in git 2.32. What advantage does this new format bring for patches?

--format=pretty is not any better than --format=raw for use with git-am
but the point is that it's as good. I have the shell alias ns="notmuch
show --format=pretty" and I can use something like `ns tag:unread` for
reading and `ns id:some-id | git am` for applying patches. I personally
really, really like simple things that work for multiple purposes.

For human consumption the pretty format is nicer than the raw format,
and it also supports showing multiple messages.

Hannu

Yet another revision of --format=pretty

[Hannu Hartikainen]

I was hoping for more discussion about the supported formats. Alas,
there has been none. I'm still posting another revision addressing the
review comment that the color implementation should be in its own
commit.

In my personal opinion this output format is better than `--format=text`
both for humans and machines. I might consider deprecating `text` now,
making `pretty` the default later, and then much later removing `text`
altogether. But I don't know the users and if there would be politics
involved in such a change.

As I said before, if this is a non-goal for notmuch, I can understand
this change being unwanted. Still, I'd like some discussion on the
matter. But maybe most readers of this list don't care either way?

Hannu

PS. I was smarter this time and didn't make this patch a reply to the
earlier ones. Hopefully I got the References: header right.

[PATCH v3 1/2] cli/show: add --format=pretty

[Hannu Hartikainen]

This commit adds the display format `pretty`. It only shows the most
important headers, similarly to `--format=text`, and only displays
plaintext parts (ie. text/* but not text/html). However, compared to
`--format=text` the output is kept as close to raw as possible.

The rationale for this feature is twofold:

1. It is useful to be able to view messages in as human-friendly format
   as possible.
2. The same format should still be machine-readable, too.

The email format is mostly human-readable as is. The things difficult
for a human eye are the huge amount of headers that are common these
days and telling different messages apart when there are many.

While human readability is the main goal, another design goal was that
piping the output to `git am` works, at least for individual messages
sent with `git send-email`. This way the format is suitable to be used
as the default, both for reading and piping output to other commands.
---
 NEWS                               |  7 ++++
 completion/notmuch-completion.bash |  2 +-
 completion/zsh/_notmuch            |  2 +-
 doc/man1/notmuch-show.rst          |  7 +++-
 notmuch-show.c                     | 54 ++++++++++++++++++++++++++++++
 test/T520-show.sh                  | 32 ++++++++++++++++++
 6 files changed, 101 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 3e776009..57826734 100644
--- a/NEWS
+++ b/NEWS
@@ -1,6 +1,13 @@
 Notmuch 0.33 (UNRELEASED)
 =========================
 
+CLI
+---
+
+`notmuch show` now has `--format=pretty`, optimized for reading plain
+text emails on the command line. It only shows the most important
+headers and plain text parts.
+
 Emacs
 -----
 
diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash
index 15425697..86cbbcdc 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -514,7 +514,7 @@ _notmuch_show()
 	    return
 	    ;;
 	--format)
-	    COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
+	    COMPREPLY=( $( compgen -W "text pretty json sexp mbox raw" -- "${cur}" ) )
 	    return
 	    ;;
 	--exclude|--body)
diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
index e920f10b..5cc386e2 100644
--- a/completion/zsh/_notmuch
+++ b/completion/zsh/_notmuch
@@ -237,7 +237,7 @@ _notmuch_search() {
 _notmuch_show() {
   _arguments -S \
     '--entire-thread=[output entire threads]:show thread:(true false)' \
-    '--format=[set output format]:output format:(text json sexp mbox raw)' \
+    '--format=[set output format]:output format:(text pretty json sexp mbox raw)' \
     '--format-version=[set output format version]:format version: ' \
     '--part=[output a single decoded mime part]:part number: ' \
     '--verify[verify signed MIME parts]' \
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 64639174..c68f0b2b 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -34,7 +34,7 @@ Supported options for **show** include
    the matching messages. For ``--format=json`` and ``--format=sexp``
    this defaults to true. For other formats, this defaults to false.
 
-.. option:: --format=(text|json|sexp|mbox|raw)
+.. option:: --format=(text|pretty|json|sexp|mbox|raw)
 
    **text** (default for messages)
      The default plain-text format has all text-content MIME parts
@@ -46,6 +46,11 @@ Supported options for **show** include
      '}'), to either open or close the component. For a multipart
      MIME message, these parts will be nested.
 
+   **pretty**
+     The plain-text parts of all matching messages are printed in a
+     format optimized for readability. Only the most important
+     headers are displayed.
+
    **json**
      The output is formatted with Javascript Object Notation
      (JSON). This format is more robust than the text format for
diff --git a/notmuch-show.c b/notmuch-show.c
index c8f1a40f..69b1e697 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -606,6 +606,52 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
     return NOTMUCH_STATUS_SUCCESS;
 }
 
+static notmuch_status_t
+format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
+		    int indent, const notmuch_show_params_t *params)
+{
+    /* The disposition and content-type metadata are associated with
+     * the envelope for message parts */
+    GMimeObject *meta = node->envelope_part ? (
+	GMIME_OBJECT (node->envelope_part) ) : node->part;
+    GMimeContentType *content_type = g_mime_object_get_content_type (meta);
+    GMimeStream *stream = params->out_stream;
+    int i;
+
+    if (GMIME_IS_MESSAGE (node->part)) {
+	GMimeMessage *message = GMIME_MESSAGE (node->part);
+	char *recipients_string;
+	char *date_string;
+
+	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
+	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
+	if (recipients_string)
+	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
+	g_free (recipients_string);
+	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_CC);
+	if (recipients_string)
+	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
+	g_free (recipients_string);
+	date_string = g_mime_message_get_date_string (node, message);
+	g_mime_stream_printf (stream, "Date: %s\n", date_string);
+	g_mime_stream_printf (stream, "Message-ID: <%s>\n\n", g_mime_message_get_message_id (
+				  message));
+    }
+
+    if (GMIME_IS_PART (node->part) &&
+	g_mime_content_type_is_type (content_type, "text", "*") &&
+	! g_mime_content_type_is_type (content_type, "text", "html")) {
+	show_text_part_content (node->part, stream, 0);
+	g_mime_stream_printf (stream, "\n");
+    }
+
+    for (i = 0; i < node->nchildren; i++)
+	format_part_pretty (ctx, sp, mime_node_child (node, i), indent, params);
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static void
 format_omitted_part_meta_sprinter (sprinter_t *sp, GMimeObject *meta, GMimePart *part)
 {
@@ -1192,6 +1238,7 @@ enum {
     NOTMUCH_FORMAT_JSON,
     NOTMUCH_FORMAT_SEXP,
     NOTMUCH_FORMAT_TEXT,
+    NOTMUCH_FORMAT_PRETTY,
     NOTMUCH_FORMAT_MBOX,
     NOTMUCH_FORMAT_RAW
 };
@@ -1211,6 +1258,11 @@ static const notmuch_show_format_t format_text = {
     .part = format_part_text,
 };
 
+static const notmuch_show_format_t format_pretty = {
+    .new_sprinter = sprinter_text_create,
+    .part = format_part_pretty,
+};
+
 static const notmuch_show_format_t format_mbox = {
     .new_sprinter = sprinter_text_create,
     .part = format_part_mbox,
@@ -1225,6 +1277,7 @@ static const notmuch_show_format_t *formatters[] = {
     [NOTMUCH_FORMAT_JSON] = &format_json,
     [NOTMUCH_FORMAT_SEXP] = &format_sexp,
     [NOTMUCH_FORMAT_TEXT] = &format_text,
+    [NOTMUCH_FORMAT_PRETTY] = &format_pretty,
     [NOTMUCH_FORMAT_MBOX] = &format_mbox,
     [NOTMUCH_FORMAT_RAW] = &format_raw,
 };
@@ -1259,6 +1312,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	{ .opt_keyword = &format, .name = "format", .keywords =
 	      (notmuch_keyword_t []){ { "json", NOTMUCH_FORMAT_JSON },
 				      { "text", NOTMUCH_FORMAT_TEXT },
+				      { "pretty", NOTMUCH_FORMAT_PRETTY },
 				      { "sexp", NOTMUCH_FORMAT_SEXP },
 				      { "mbox", NOTMUCH_FORMAT_MBOX },
 				      { "raw", NOTMUCH_FORMAT_RAW },
diff --git a/test/T520-show.sh b/test/T520-show.sh
index 6f42ca12..c9bcf81d 100755
--- a/test/T520-show.sh
+++ b/test/T520-show.sh
@@ -27,4 +27,36 @@ notmuch show --entire-thread=true --sort=newest-first $QUERY > EXPECTED
 notmuch show --entire-thread=true --sort=oldest-first $QUERY > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "--format=pretty"
+output=$(notmuch show --format=pretty id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com 2>&1 && echo OK)
+test_expect_equal "$output" "Subject: [notmuch] preliminary FreeBSD support
+From: Alex Botero-Lowry <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 11:36:14 -0800
+Message-ID: <cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com>
+
+I saw the announcement this morning, and was very excited, as I had been
+hoping sup would be turned into a library,
+since I like the concept more than the UI (I'd rather an emacs interface).
+
+I did a preliminary compile which worked out fine, but
+sysconf(_SC_SC_GETPW_R_SIZE_MAX) returns -1 on
+FreeBSD, so notmuch_config_open segfaulted.
+
+Attached is a patch that supplies a default buffer size of 64 in cases where
+-1 is returned.
+
+http://www.opengroup.org/austin/docs/austin_328.txt - seems to indicate this
+is acceptable behavior,
+and http://mail-index.netbsd.org/pkgsrc-bugs/2006/06/07/msg016808.htmlspecifically
+uses 64 as the
+buffer size.
+
+_______________________________________________
+notmuch mailing list
+notmuch@notmuchmail.org
+http://notmuchmail.org/mailman/listinfo/notmuch
+
+OK"
+
 test_done
-- 
2.32.0

[PATCH v3 2/2] cli/show: add color for --format=pretty

[Hannu Hartikainen]

The arguments `--color` and `--no-color` allow setting color explicitly
on or off when using `--format=pretty`. The default is to use color iff
stdout is a TTY.
---
 NEWS                      |  4 +++-
 doc/man1/notmuch-show.rst |  9 ++++++++-
 notmuch-client.h          |  1 +
 notmuch-show.c            | 15 +++++++++++++++
 4 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 57826734..ac6fdddd 100644
--- a/NEWS
+++ b/NEWS
@@ -6,7 +6,9 @@ CLI
 
 `notmuch show` now has `--format=pretty`, optimized for reading plain
 text emails on the command line. It only shows the most important
-headers and plain text parts.
+headers and plain text parts. The output is colored if the output is a
+TTY or.  Alternatively, `--color` and `--no-color` can be used to set
+color explicitly on or off.
 
 Emacs
 -----
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index c68f0b2b..d1faf8fe 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -49,7 +49,8 @@ Supported options for **show** include
    **pretty**
      The plain-text parts of all matching messages are printed in a
      format optimized for readability. Only the most important
-     headers are displayed.
+     headers are displayed. Color is used if the output is to a TTY
+     unless ``--color`` or ``--no-color`` is used explicitly.
 
    **json**
      The output is formatted with Javascript Object Notation
@@ -221,6 +222,12 @@ Supported options for **show** include
    "text/html" parts, no part with content type "text/html" is included
    in the output.
 
+.. option:: --color
+
+   Use colored output for formats that support that (currently only
+   ``--format=pretty``). By default, color is used if the output
+   goes to a TTY. To disable color, use ``--no-color``.
+
 A common use of **notmuch show** is to display a single thread of
 email messages. For this, use a search term of "thread:<thread-id>" as
 can be seen in the first column of output from the
diff --git a/notmuch-client.h b/notmuch-client.h
index 8227fea4..c4f7cfd6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -78,6 +78,7 @@ typedef struct notmuch_show_params {
     int part;
     _notmuch_crypto_t crypto;
     bool include_html;
+    bool color;
     GMimeStream *out_stream;
 } notmuch_show_params_t;
 
diff --git a/notmuch-show.c b/notmuch-show.c
index 69b1e697..a67bd698 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -617,14 +617,21 @@ format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
     GMimeContentType *content_type = g_mime_object_get_content_type (meta);
     GMimeStream *stream = params->out_stream;
     int i;
+    bool color = params->color;
 
     if (GMIME_IS_MESSAGE (node->part)) {
 	GMimeMessage *message = GMIME_MESSAGE (node->part);
 	char *recipients_string;
 	char *date_string;
 
+	if (color)
+	    g_mime_stream_printf (stream, "\e[36m");
 	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[33m");
 	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[31m");
 	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
 	if (recipients_string)
 	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
@@ -634,9 +641,15 @@ format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
 	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
 	g_free (recipients_string);
 	date_string = g_mime_message_get_date_string (node, message);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[35m");
 	g_mime_stream_printf (stream, "Date: %s\n", date_string);
+	if (color)
+	    g_mime_stream_printf (stream, "\e[32m");
 	g_mime_stream_printf (stream, "Message-ID: <%s>\n\n", g_mime_message_get_message_id (
 				  message));
+	if (color)
+	    g_mime_stream_printf (stream, "\e[0m");
     }
 
     if (GMIME_IS_PART (node->part) &&
@@ -1294,6 +1307,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	.part = -1,
 	.omit_excluded = true,
 	.output_body = true,
+	.color = isatty (fileno (stdout)),
 	.crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
     };
     int format = NOTMUCH_FORMAT_NOT_SPECIFIED;
@@ -1333,6 +1347,7 @@ notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[])
 	{ .opt_bool = &params.crypto.verify, .name = "verify" },
 	{ .opt_bool = &params.output_body, .name = "body" },
 	{ .opt_bool = &params.include_html, .name = "include-html" },
+	{ .opt_bool = &params.color, .name = "color" },
 	{ .opt_inherit = notmuch_shared_options },
 	{ }
     };
-- 
2.32.0

Re: Yet another revision of --format=pretty

[David Bremner]

Hannu Hartikainen <hannu@hrtk.in> writes:

>
> In my personal opinion this output format is better than `--format=text`
> both for humans and machines. I might consider deprecating `text` now,
> making `pretty` the default later, and then much later removing `text`
> altogether. But I don't know the users and if there would be politics
> involved in such a change.

The main thing I was worried about was the vim interface and Felipe
confirmed that it does not use text format at all. There are no doubt
user scripts out there that will have to be adjusted, so it would have
to be a long-ish deprecation, but we've done that before.

I guess the question for you is if the format will still be useful for
you if we need to include _some_ metadata in order to replace use cases
for text format? I'm thinking of potentially a couple of X-Notmuch-*
headers. But that in itself is another design discussion.

> As I said before, if this is a non-goal for notmuch, I can understand
> this change being unwanted. Still, I'd like some discussion on the
> matter. But maybe most readers of this list don't care either way?

I think having some quick mail pager for search results makes sense. I'm
not sure how far in the direction of MH we should go, but we can look at
each potential change as it arrives.

I'll reply with actual code review in a separate message.

Re: [PATCH v3 1/2] cli/show: add --format=pretty

[David Bremner]

Hannu Hartikainen <hannu@hrtk.in> writes:


> +static notmuch_status_t
> +format_part_pretty (const void *ctx, sprinter_t *sp, mime_node_t *node,
> +		    int indent, const notmuch_show_params_t *params)
> +{
> +    /* The disposition and content-type metadata are associated with
> +     * the envelope for message parts */
> +    GMimeObject *meta = node->envelope_part ? (
> +	GMIME_OBJECT (node->envelope_part) ) : node->part;
> +    GMimeContentType *content_type = g_mime_object_get_content_type (meta);
> +    GMimeStream *stream = params->out_stream;
> +    int i;
> +
> +    if (GMIME_IS_MESSAGE (node->part)) {
> +	GMimeMessage *message = GMIME_MESSAGE (node->part);
> +	char *recipients_string;
> +	char *date_string;
> +
> +	g_mime_stream_printf (stream, "Subject: %s\n", g_mime_message_get_subject (message));
> +	g_mime_stream_printf (stream, "From: %s\n", g_mime_message_get_from_string (message));
> +	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_TO);
> +	if (recipients_string)
> +	    g_mime_stream_printf (stream, "To: %s\n", recipients_string);
> +	g_free (recipients_string);
> +	recipients_string = g_mime_message_get_address_string (message, GMIME_ADDRESS_TYPE_CC);
> +	if (recipients_string)
> +	    g_mime_stream_printf (stream, "Cc: %s\n", recipients_string);
> +	g_free (recipients_string);
> +	date_string = g_mime_message_get_date_string (node, message);
> +	g_mime_stream_printf (stream, "Date: %s\n", date_string);
> +	g_mime_stream_printf (stream, "Message-ID: <%s>\n\n", g_mime_message_get_message_id (
> +				  message));

I don't know what g_mime_message_get_message_id will return if there is
no message-id, but that case can and does arise. Notmuch constructs a
message-id by sha1 hashing the message. That message-id is in the
database, so I think you should probably print out the message-id from
the database.

> +static const notmuch_show_format_t format_pretty = {
> +    .new_sprinter = sprinter_text_create,
> +    .part = format_part_pretty,
> +};

I think this deserves a comment to explain briefly why it works.

> diff --git a/test/T520-show.sh b/test/T520-show.sh
> index 6f42ca12..c9bcf81d 100755
> --- a/test/T520-show.sh
> +++ b/test/T520-show.sh
> @@ -27,4 +27,36 @@ notmuch show --entire-thread=true --sort=newest-first $QUERY > EXPECTED
>  notmuch show --entire-thread=true --sort=oldest-first $QUERY > OUTPUT
>  test_expect_equal_file EXPECTED OUTPUT
>  
> +test_begin_subtest "--format=pretty"
> +output=$(notmuch show --format=pretty id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com 2>&1 && echo OK)
> +test_expect_equal "$output" "Subject: [notmuch] preliminary FreeBSD support
> +From: Alex Botero-Lowry <alex.boterolowry@gmail.com>
> +To: notmuch@notmuchmail.org
> +Date: Tue, 17 Nov 2009 11:36:14 -0800
> +Message-ID: <cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com>
> +

I would prefer the
cat <<EOF > EXPECTED
...
EOF

style here rather then the multiline strings, finishing with
"test_expect_equal_file EXPECTED OUTPUT".

Re: [PATCH v3 1/2] cli/show: add --format=pretty

[David Bremner]

David Bremner <david@tethera.net> writes:

>
> I don't know what g_mime_message_get_message_id will return if there is
> no message-id, but that case can and does arise. Notmuch constructs a
> message-id by sha1 hashing the message. That message-id is in the
> database, so I think you should probably print out the message-id from
> the database.
>

I expect that would require somehow making the notmuch_database_t object
available inside the sprinter. We want this for at least one other
feature request (configurable headers in json / sexpr output), so it's
not as much of a "waste" of effort as it might seem at first.  The
options are adding extra arguments to functions and stashing a copy of
the pointer in some struct (perhaps the sprinter struct). I think David
E has a prototype of the former. My own instincts is that the latter
will be more general.  The previous consensus was that we did not want
to add static (file scope) variables, even though that would be the
quick and dirty solution.

d

Re: [PATCH v3 1/2] cli/show: add --format=pretty

[Hannu Hartikainen]

Thanks for reviewing the patch!

On Tue, 13 Jul 2021 07:24:43 -0300, David Bremner <david@tethera.net> wrote:
> David Bremner <david@tethera.net> writes:
> > I don't know what g_mime_message_get_message_id will return if there is
> > no message-id, but that case can and does arise. 

It returns null and the printing function prints out `(null)` IIRC so
it's not a crash (I checked that), but good point nonetheless.

> I expect that would require somehow making the notmuch_database_t object
> available inside the sprinter. We want this for at least one other
> feature request (configurable headers in json / sexpr output), so it's
> not as much of a "waste" of effort as it might seem at first.

That's good to know. I was afraid of making big changes for no reason. I
think I had a version that added a function parameter but thought it's
too ugly and complicated to submit.

> The options are adding extra arguments to functions and stashing a
> copy of the pointer in some struct (perhaps the sprinter struct).

I'll see if I find a good way to do the latter when I have time. Thanks
for the pointers!

Hannu

Re: Yet another revision of --format=pretty

[Hannu Hartikainen]

On Mon, 12 Jul 2021 20:19:37 -0300, David Bremner <david@tethera.net> wrote:
> The main thing I was worried about was the vim interface and Felipe
> confirmed that it does not use text format at all. There are no doubt
> user scripts out there that will have to be adjusted, so it would have
> to be a long-ish deprecation, but we've done that before.

Cool!

> I guess the question for you is if the format will still be useful for
> you if we need to include _some_ metadata in order to replace use cases
> for text format? I'm thinking of potentially a couple of X-Notmuch-*
> headers. But that in itself is another design discussion.

I'm fine with that. There also *could* be a way to configure which
headers are displayed; Tomi suggested that the colors should be
configurable, and I think configuring pairs of header+color would be one
option of implementing that. But I need to dogfood some more before I
can confidently say what I like best.

> I think having some quick mail pager for search results makes sense. I'm
> not sure how far in the direction of MH we should go, but we can look at
> each potential change as it arrives.

I'd never heard of MH before. I'll need to try it out :) (For those not
familiar, it can be found at https://www.nongnu.org/nmh/ .) It sounds
philosophically like what I want from a CLI mail client. But then again
I really don't need much. I'm writing this message with this shell
function (and Neovim) and I think it's good enough. :)

    nr() {
	draft=$(mktemp $HOME/draft-XXXX)
	$notmuch reply $1 > "$draft" || return
	nvim -c 'set filetype=mail' "$draft" || return
	cat "$draft"
	echo
	echo "Press ENTER to send (Ctrl-C to cancel)"
	read || return
	cat "$draft" | msmtp -t || return
	rm "$draft"
    }

Hannu