Getting the right root mail of the thread

Getting the right root mail of the thread

[Felipe Contreras]

Hi,

I often wondered why when browsing threads, I often see a message that doesn't
describe at all the thread, for example:

  notmuch search ...
  thread:000000000000826a 18 mins. ago [45/45] Felipe Contreras; [PATCH/TEST 44/44]

I can reverse the order of the search:

  notmuch search --sort=oldest-first
  thread:000000000000826a 23 mins. ago [45/45] Felipe Contreras; [PATCH/TEST 00/44]

Then I get the correct summary, but now the order of the search is the other
way around, and there doesn't seem to be a way to specify the order of the
messages independently of the order of the threads.

Either way this doesn't make any sense to me. Each thread has a single origin
mail, why would anybody would like to show a message other than that while
displaying the summary of the tread? Even more, why isn't there an option to
fetch that information easily?

I am forced to get the list of messages, and then grab the first one, even
though it's not efficient, probably unnecessary, and potentiall wrong. For
example, it's possible that multiple mails arrived at the same time, then how
am I supposed to display the one that originated the thread?

I think there should be a way to get the root mail of a thread, irrespective of
the search order.

Cheers.

-- 
Felipe Contreras

Re: Getting the right root mail of the thread

[Jani Nikula]

On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
> I often wondered why when browsing threads, I often see a message that doesn't
> describe at all the thread, for example:
>
>   notmuch search ...
>   thread:000000000000826a 18 mins. ago [45/45] Felipe Contreras; [PATCH/TEST 44/44]
>
> I can reverse the order of the search:
>
>   notmuch search --sort=oldest-first
>   thread:000000000000826a 23 mins. ago [45/45] Felipe Contreras; [PATCH/TEST 00/44]
>
> Then I get the correct summary, but now the order of the search is the other
> way around, and there doesn't seem to be a way to specify the order of the
> messages independently of the order of the threads.

I think it's actually worse than what your example demonstrates. It's
the subject of the newest/oldest *matching* message that gets used. In
your example, the first/last messages in the thread apparently match
your query.

> Either way this doesn't make any sense to me. Each thread has a single origin
> mail, why would anybody would like to show a message other than that while
> displaying the summary of the tread? Even more, why isn't there an option to
> fetch that information easily?
>
> I am forced to get the list of messages, and then grab the first one, even
> though it's not efficient, probably unnecessary, and potentiall wrong. For
> example, it's possible that multiple mails arrived at the same time, then how
> am I supposed to display the one that originated the thread?
>
> I think there should be a way to get the root mail of a thread,
> irrespective of the search order.

Largely agreed. It's just that nobody's gotten around to doing this
yet. At the cli level I think the consensus is that the structured
(sexp/json) output format should contain multiple (or all) subjects.


BR,
Jani.

Re: Getting the right root mail of the thread

[Felipe Contreras]

On Sat, Nov 2, 2013 at 7:50 AM, Jani Nikula <jani@nikula.org> wrote:
> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:

>> I think there should be a way to get the root mail of a thread,
>> irrespective of the search order.
>
> Largely agreed. It's just that nobody's gotten around to doing this
> yet. At the cli level I think the consensus is that the structured
> (sexp/json) output format should contain multiple (or all) subjects.

What about the default? (--format=text). What about user-interfaces
that must display a summary of a thread?

-- 
Felipe Contreras

Re: Getting the right root mail of the thread

[Jani Nikula]

On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
> On Sat, Nov 2, 2013 at 7:50 AM, Jani Nikula <jani@nikula.org> wrote:
>> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
>
>>> I think there should be a way to get the root mail of a thread,
>>> irrespective of the search order.
>>
>> Largely agreed. It's just that nobody's gotten around to doing this
>> yet. At the cli level I think the consensus is that the structured
>> (sexp/json) output format should contain multiple (or all) subjects.
>
> What about the default? (--format=text). What about user-interfaces
> that must display a summary of a thread?

To be honest, we haven't had much interest in adding new content or
features to the text output formats. It's just much easier to do this in
a backwards compatible way in the structured output formats, for which
we also have the format versioning. The emacs ui uses the sexp format
for the thread summaries.

That said, if someone were interested in amending notmuch search
--format=text --output=summary, I don't think we'd have anything against
it. I'd say a new option to specify a format string would be the way to
go.


BR,
Jani.

Re: Getting the right root mail of the thread

[Daniel Kahn Gillmor]

On 11/02/2013 09:08 AM, Felipe Contreras wrote:
> Either way this doesn't make any sense to me. Each thread has a single origin
> mail, why would anybody would like to show a message other than that while
> displaying the summary of the tread? Even more, why isn't there an option to
> fetch that information easily?

It's entirely possible to make a message that has a References: header 
that includes message IDs from two different threads.  For a message 
like this, there is not a single "origin mail", right?  For threads that 
derive from this message, the whole thread has more than one origin.

	--dkg

Re: Getting the right root mail of the thread

[Felipe Contreras]

On Sat, Nov 2, 2013 at 8:58 AM, Jani Nikula <jani@nikula.org> wrote:
> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
>> On Sat, Nov 2, 2013 at 7:50 AM, Jani Nikula <jani@nikula.org> wrote:
>>> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
>>
>>>> I think there should be a way to get the root mail of a thread,
>>>> irrespective of the search order.
>>>
>>> Largely agreed. It's just that nobody's gotten around to doing this
>>> yet. At the cli level I think the consensus is that the structured
>>> (sexp/json) output format should contain multiple (or all) subjects.
>>
>> What about the default? (--format=text). What about user-interfaces
>> that must display a summary of a thread?
>
> To be honest, we haven't had much interest in adding new content or
> features to the text output formats. It's just much easier to do this in
> a backwards compatible way in the structured output formats, for which
> we also have the format versioning. The emacs ui uses the sexp format
> for the thread summaries.

How is this any better?

(:thread "000000000000826a" :timestamp 1383396519 :date_relative
"Yest. 06:48" :matched 45 :total 45 :authors "Felipe Contreras"
:subject "[PATCH/TEST 44/44] request-pull: rewrite to C" :tags ("sent"
"to-me"))

-- 
Felipe Contreras

Re: Getting the right root mail of the thread

[Felipe Contreras]

On Sun, Nov 3, 2013 at 1:22 PM, Daniel Kahn Gillmor
<dkg@fifthhorseman.net> wrote:
> On 11/02/2013 09:08 AM, Felipe Contreras wrote:
>>
>> Either way this doesn't make any sense to me. Each thread has a single
>> origin
>> mail, why would anybody would like to show a message other than that while
>> displaying the summary of the tread? Even more, why isn't there an option
>> to
>> fetch that information easily?
>
>
> It's entirely possible to make a message that has a References: header that
> includes message IDs from two different threads.

Right, there could be two or more roots. But we don't get either in
the summary, we can only get the first or the last one.

I say the summary should contain the earliest root, always. Or at
least have an option to get it.

-- 
Felipe Contreras

Re: Getting the right root mail of the thread

[Jani Nikula]

On Sun, 03 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
> On Sat, Nov 2, 2013 at 8:58 AM, Jani Nikula <jani@nikula.org> wrote:
>> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
>>> On Sat, Nov 2, 2013 at 7:50 AM, Jani Nikula <jani@nikula.org> wrote:
>>>> On Sat, 02 Nov 2013, Felipe Contreras <felipe.contreras@gmail.com> wrote:
>>>
>>>>> I think there should be a way to get the root mail of a thread,
>>>>> irrespective of the search order.
>>>>
>>>> Largely agreed. It's just that nobody's gotten around to doing this
>>>> yet. At the cli level I think the consensus is that the structured
>>>> (sexp/json) output format should contain multiple (or all) subjects.
>>>
>>> What about the default? (--format=text). What about user-interfaces
>>> that must display a summary of a thread?
>>
>> To be honest, we haven't had much interest in adding new content or
>> features to the text output formats. It's just much easier to do this in
>> a backwards compatible way in the structured output formats, for which
>> we also have the format versioning. The emacs ui uses the sexp format
>> for the thread summaries.
>
> How is this any better?
>
> (:thread "000000000000826a" :timestamp 1383396519 :date_relative
> "Yest. 06:48" :matched 45 :total 45 :authors "Felipe Contreras"
> :subject "[PATCH/TEST 44/44] request-pull: rewrite to C" :tags ("sent"
> "to-me"))

I made no claims it would contain more or better information *now*. But
if someone were to add new things, the structured output is where it
would likely be added. We could add :subject_first, :subject_oldest, or
whatever *in addition* to :subject.

BR,
Jani.

Re: Getting the right root mail of the thread

[Jesse Rosenthal]

Jani Nikula <jani@nikula.org> writes:
> I think it's actually worse than what your example demonstrates. It's
> the subject of the newest/oldest *matching* message that gets used. In
> your example, the first/last messages in the thread apparently match
> your query.

The behavior is there because subjects frequently change in long
threads, and this allows the subject to refer to the most recent unread
message (if we're sorting in the default order). The 
reason I requested and wrote in this behavior five years ago or so (my
only c contribution ever) was that numerous business associates would
keep email lists by replying and changing the subject. This is *very*
common outside of programming circles. Even in programming circles,
subjects frequently change on mailing list (with a "[was: ...]"
appended). 

The current behavior can be annoying, but the old behavior could make
the MUA quite unusable in a number of circumstances. (And yes, an MUA
that fails on reading mail from senders with bad emailing practices is
unusable for me.)

Maybe there should be a "show original subject" toggle? That wouldn't be
too hard, though it would require another call to the library and
regenerating the search results.

Re: Getting the right root mail of the thread

[Jani Nikula]

On Sun, 03 Nov 2013, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> The current behavior can be annoying, but the old behavior could make
> the MUA quite unusable in a number of circumstances. (And yes, an MUA
> that fails on reading mail from senders with bad emailing practices is
> unusable for me.)

I wouldn't have suggested changing the current default behaviour anyway,
but thanks for reminding us there are users depending on it!

> Maybe there should be a "show original subject" toggle? That wouldn't be
> too hard, though it would require another call to the library and
> regenerating the search results.

IMO the best approach would be including the first (or oldest, or
original) subject in the structured thread summary output, in addition
to the first matching subject that we have now.

BR,
Jani.

Re: Getting the right root mail of the thread

[Felipe Contreras]

On Sun, Nov 3, 2013 at 2:35 PM, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> Jani Nikula <jani@nikula.org> writes:
>> I think it's actually worse than what your example demonstrates. It's
>> the subject of the newest/oldest *matching* message that gets used. In
>> your example, the first/last messages in the thread apparently match
>> your query.
>
> The behavior is there because subjects frequently change in long
> threads, and this allows the subject to refer to the most recent unread
> message (if we're sorting in the default order). The
> reason I requested and wrote in this behavior five years ago or so (my
> only c contribution ever) was that numerous business associates would
> keep email lists by replying and changing the subject. This is *very*
> common outside of programming circles. Even in programming circles,
> subjects frequently change on mailing list (with a "[was: ...]"
> appended).

Yes but how important is it to keep track of that?

I say it is much more important to track threads like this properly:

  foobar patch 0 (usually a summary/overview)
  +-foobar patch 1
  | +-comment on patch 1
  +-foobar patch 2
  +-foobar patch 3
  | +-comment on patch 3
  +-foobar patch 4
  +-foobar patch 5

But fine, let's concentrate on the common user scenario (which is not
common for notmuch users at all). We can have a thread like this:

  No work on Friday
  + Shall we go for some beers? (was: No work on Friday)
  + What about project X? (was: No work on Friday)

So which is the correct summary of the thread? The fact of the matter
is that we are talking about three threads now.

Gmail does this correctly. Each time the subject is changed, it's
considered a new thread.

> The current behavior can be annoying, but the old behavior could make
> the MUA quite unusable in a number of circumstances. (And yes, an MUA
> that fails on reading mail from senders with bad emailing practices is
> unusable for me.)

This is rhetorical warfare. It's wouldn't be "failing on reading mail".

If displaying the original subject is failing, then we might be
failing when searching with older first. Why would the order of the
search affect the thread summary?

> Maybe there should be a "show original subject" toggle? That wouldn't be
> too hard, though it would require another call to the library and
> regenerating the search results.

Yes. I say it should be a property of the query. I don't see why
anybody would want it any other way, but it wouldn't hurt to make it
an option.

-- 
Felipe Contreras