[RFC PATCH] show: Add support for -format=raw -body=false

[RFC PATCH] show: Add support for -format=raw -body=false

[David Edmondson]

Similar to other formats, allow the body to be omitted when outputting
raw messages.

This can be used by UI code to get the full headers of a message
without the need to consume the body, which may be large.
---
 notmuch-show.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index dd836add..f52e6a40 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -888,6 +888,8 @@ format_part_raw (unused (const void *ctx), unused (sprinter_t *sp),
 	ssize_t ssize;
 	char buf[4096];
 	notmuch_status_t ret = NOTMUCH_STATUS_FILE_ERROR;
+	unsigned int cr_count = 0;
+	notmuch_bool_t done = false;
 
 	filename = notmuch_message_get_filename (node->envelope_file);
 	if (filename == NULL) {
@@ -901,13 +903,32 @@ format_part_raw (unused (const void *ctx), unused (sprinter_t *sp),
 	    goto DONE;
 	}
 
-	while (! g_mime_stream_eos (stream)) {
+	while (! done && ! g_mime_stream_eos (stream)) {
 	    ssize = g_mime_stream_read (stream, buf, sizeof (buf));
 	    if (ssize < 0) {
 		fprintf (stderr, "Error: Read failed from %s\n", filename);
 		goto DONE;
 	    }
 
+	    if (! params->output_body) {
+		/*
+		 * Look for two adjacent newlines, as they mark the
+		 * separation of the headers from the body.
+		 */
+		for (ssize_t off = 0; off < ssize; off++) {
+		    if (buf[off] == '\n') {
+			cr_count++;
+		    } else {
+			cr_count = 0;
+		    }
+		    if (cr_count == 2) {
+			ssize = off;
+			done = true;
+			break;
+		    }
+		}
+	    }
+
 	    if (ssize > 0 && fwrite (buf, ssize, 1, stdout) != 1) {
 		fprintf (stderr, "Error: Write %ld chars to stdout failed\n", ssize);
 		goto DONE;
@@ -1310,9 +1331,10 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[])
 	} else {
 	    if (format != NOTMUCH_FORMAT_TEXT &&
 		format != NOTMUCH_FORMAT_JSON &&
-		format != NOTMUCH_FORMAT_SEXP)
+		format != NOTMUCH_FORMAT_SEXP &&
+		format != NOTMUCH_FORMAT_RAW)
 		fprintf (stderr,
-			 "Warning: --body=false only implemented for format=text, format=json and format=sexp\n");
+			 "Warning: --body=false is not implemented for format=mbox\n");
 	}
     }
 
-- 
2.20.1

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Bremner]

David Edmondson <dme@dme.org> writes:

> Similar to other formats, allow the body to be omitted when outputting
> raw messages.
>
> This can be used by UI code to get the full headers of a message
> without the need to consume the body, which may be large.

I guess I'm a bit confused why this is for "raw" format, and not
"text". Are you proposing that we have two ad-hoc output formats for
consuming by scripts and other tools that don't want JSON or SEXP?

d

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Edmondson]

On Monday, 2021-01-18 at 13:37:08 -04, David Bremner wrote:

> David Edmondson <dme@dme.org> writes:
>
>> Similar to other formats, allow the body to be omitted when outputting
>> raw messages.
>>
>> This can be used by UI code to get the full headers of a message
>> without the need to consume the body, which may be large.
>
> I guess I'm a bit confused why this is for "raw" format, and not
> "text". Are you proposing that we have two ad-hoc output formats for
> consuming by scripts and other tools that don't want JSON or SEXP?

"text" is a strange archaic output format, the rationale for which seems
lost in time :-)

"notmuch show --format=text --body=false id:foo" already works, but
doesn't provide a complete set of headers - it's limited to subject,
from, to, date, *cc, ...

I wanted to get the Autocrypt or Face headers, so ended up using
"--format=raw", but then end up with the entire message, which can be
megabytes large. With the emacs UI, that means pulling all of that data
into a buffer, just to ignore it.

Now, it would be nice to be able to say something like...

notmuch show --format=sexp --body=false \
  --headers=received-by,face,x-face \
  id:foo

...to get some extra headers in the sexp output, but I didn't look at it
yet.

The proposed change seems like a simple, obvious improvement that is
entirely in-line with the existing interface and implementation. Is
there any reason not to do it?

dme.
-- 
Don't you know you're never going to get to France.

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Bremner]

David Edmondson <dme@dme.org> writes:

> On Monday, 2021-01-18 at 13:37:08 -04, David Bremner wrote:
>
>> David Edmondson <dme@dme.org> writes:
>>
>>> Similar to other formats, allow the body to be omitted when outputting
>>> raw messages.
>>>
>>> This can be used by UI code to get the full headers of a message
>>> without the need to consume the body, which may be large.
>>
>> I guess I'm a bit confused why this is for "raw" format, and not
>> "text". Are you proposing that we have two ad-hoc output formats for
>> consuming by scripts and other tools that don't want JSON or SEXP?
>
> "text" is a strange archaic output format, the rationale for which seems
> lost in time :-)

Yeah, at one time I wanted to get rid of it, but inter alia the vim
bindings use(d?) it.

> I wanted to get the Autocrypt or Face headers, so ended up using
> "--format=raw", but then end up with the entire message, which can be
> megabytes large. With the emacs UI, that means pulling all of that data
> into a buffer, just to ignore it.

Ok thanks for explaining the use case.


> Now, it would be nice to be able to say something like...
>
> notmuch show --format=sexp --body=false \
>   --headers=received-by,face,x-face \
>   id:foo
>
> ...to get some extra headers in the sexp output, but I didn't look at it
> yet.

There was a series at id:20191122230730.35712-2-johan.parin@gmail.com
that got stalled due to (I think) some issues with passing around
configuration information.
>
> The proposed change seems like a simple, obvious improvement that is
> entirely in-line with the existing interface and implementation. Is
> there any reason not to do it?

I'm just nervous about more legacy formats. I think we will get the
arbitrary headers thing working eventually, but I can see it might take
some time.

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Edmondson]

On Monday, 2021-01-18 at 14:28:02 -04, David Bremner wrote:

> David Edmondson <dme@dme.org> writes:
>> Now, it would be nice to be able to say something like...
>>
>> notmuch show --format=sexp --body=false \
>>   --headers=received-by,face,x-face \
>>   id:foo
>>
>> ...to get some extra headers in the sexp output, but I didn't look at it
>> yet.
>
> There was a series at id:20191122230730.35712-2-johan.parin@gmail.com
> that got stalled due to (I think) some issues with passing around
> configuration information.

I will go and look at this.

>> The proposed change seems like a simple, obvious improvement that is
>> entirely in-line with the existing interface and implementation. Is
>> there any reason not to do it?
>
> I'm just nervous about more legacy formats. I think we will get the
> arbitrary headers thing working eventually, but I can see it might take
> some time.

Are we ever likely to get rid of the raw format? I use it quite a bit,
and it seems to be relatively low complexity and simple to support.

dme.
-- 
They must have taken my marbles away.

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[Tomi Ollila]

On Mon, Jan 18 2021, David Edmondson wrote:

> On Monday, 2021-01-18 at 14:28:02 -04, David Bremner wrote:
>
>> David Edmondson <dme@dme.org> writes:
>>> Now, it would be nice to be able to say something like...
>>>
>>> notmuch show --format=sexp --body=false \
>>>   --headers=received-by,face,x-face \
>>>   id:foo
>>>
>>> ...to get some extra headers in the sexp output, but I didn't look at it
>>> yet.
>>
>> There was a series at id:20191122230730.35712-2-johan.parin@gmail.com
>> that got stalled due to (I think) some issues with passing around
>> configuration information.
>
> I will go and look at this.
>
>>> The proposed change seems like a simple, obvious improvement that is
>>> entirely in-line with the existing interface and implementation. Is
>>> there any reason not to do it?
>>
>> I'm just nervous about more legacy formats. I think we will get the
>> arbitrary headers thing working eventually, but I can see it might take
>> some time.
>
> Are we ever likely to get rid of the raw format? I use it quite a bit,
> and it seems to be relatively low complexity and simple to support.

Now that I tried I kinda like it (and seems useful, which I don't know
whether 'text' is). It would also be nice that with raw format more
than one message could match (just like text). 

Also, this --body=false sound good. I looked the patch a bit yesterday,
and wondered whether it could be made simpler (or whether it is consistent
how that is done in other formats...)

Tomi

>
> dme.
> -- 
> They must have taken my marbles away.

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Bremner]

David Edmondson <dme@dme.org> writes:

>> I'm just nervous about more legacy formats. I think we will get the
>> arbitrary headers thing working eventually, but I can see it might take
>> some time.
>
> Are we ever likely to get rid of the raw format? I use it quite a bit,
> and it seems to be relatively low complexity and simple to support.

It isn't completely without maintenance overhead, but I agree it's not
going away. I can live with the body=false option if you think it's
better / more feasible than the arbitrary headers series.

d

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Edmondson]

On Thursday, 2021-02-04 at 22:33:06 -04, David Bremner wrote:

> David Edmondson <dme@dme.org> writes:
>
>>> I'm just nervous about more legacy formats. I think we will get the
>>> arbitrary headers thing working eventually, but I can see it might take
>>> some time.
>>
>> Are we ever likely to get rid of the raw format? I use it quite a bit,
>> and it seems to be relatively low complexity and simple to support.
>
> It isn't completely without maintenance overhead, but I agree it's not
> going away. I can live with the body=false option if you think it's
> better / more feasible than the arbitrary headers series.

At the moment I'm using the former because the latter is missing, but in
principle I think that they might fill different needs. Could we have
both?

dme.
-- 
I just bite it, it's for the look I don't light it.

Re: [RFC PATCH] show: Add support for -format=raw -body=false

[David Bremner]

David Edmondson <dme@dme.org> writes:

> On Thursday, 2021-02-04 at 22:33:06 -04, David Bremner wrote:
>
>> David Edmondson <dme@dme.org> writes:
>>
>>>> I'm just nervous about more legacy formats. I think we will get the
>>>> arbitrary headers thing working eventually, but I can see it might take
>>>> some time.
>>>
>>> Are we ever likely to get rid of the raw format? I use it quite a bit,
>>> and it seems to be relatively low complexity and simple to support.
>>
>> It isn't completely without maintenance overhead, but I agree it's not
>> going away. I can live with the body=false option if you think it's
>> better / more feasible than the arbitrary headers series.
>
> At the moment I'm using the former because the latter is missing, but in
> principle I think that they might fill different needs. Could we have
> both?

On the scale of things, the extra code is not a big deal, so sure.

d