Recent (and forthcoming) improvements to the emacs interface

Recent (and forthcoming) improvements to the emacs interface

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 6264 bytes --]

I just pushed out a nice set of changes to the emacs interface. Here's a
quick summary of what you can expect to get when you next update:

  * Much nicer looking presentation, (no more ugly reverse-video or
    underlines on the message summary line).

  * More reliable message-visibility buttons, (using RET in the first
    column of a message-summary line now works).

  * Space bar fixed to advance to next open message, (it was originally
    written this way, but has been broken since we changed from global
    to local toggling of hidden message parts).

  * Showing a thread where the search matches only a subset of the
    thread now opens only the matched messages (in addition to unread
    messages).

This last feature is the big one---the rest all just happened to come
along at the same time. One thing that I often do is read some giant
thread and then tag a single message deep in that thread for dealing
with later. And previously, doing a search for that tag would bring back
the entire thread. Now, it opens only the message I'm actually looking
for. So this is a very welcome change

And thanks to Bart Trojanowski for the groundwork for this change. I
think the vim interface has had this feature for a while, (or would have
if I had pushed all of Bart's changes earlier).

Meanwhile, Keith and Eric gave me some helpful feedback about the
notmuch user-interface over lunch today, and in particular about the
handling of the "unread" tag. Here are some of the things discussed,
along with some things I'd like to change in response.

I'm open to suggestions on all of these, and most importantly, wanted to
let people know about some upcoming user-interface changes before they
were in place and potentially surprising.

  * The magic space bar is too magic. Threads are separate conversations
    so one key for paging through the current conversation shouldn't
    also switch to the next conversation, (particularly when the
    complementary key DEL doesn't reverse this behavior of SPACE).

    Recommendation: Make SPACE only page the current message. Recommend
    that user use 'a' to advance to next thread, (or 'x' to exit back to
    search results).

  * The unread tag is not handled transparently enough. Both Keith and
    Eric complained of frequently being presented with messages as
    "unread" that they had read before. (And I don't want to ever have
    to manually think about whether to remove a thread as "unread".)

    Recommendation: Drop the 'A' and 'X' keybindings and make 'a' and
    'x' mark remove the "unread" tag from all messages in the current
    thread (as well as the "inbox" tag as currently). Also make 'n' and
    'p' remove the "unread tag as well.

    Followup: This frees up 'N' and 'P', so I'd like to use the for
    "next message" and "previous message" and make 'n' and 'p' do "next
    open message" and "previous open message".

  * Opening up unread messages in notmuch-show mode is not
    helpful. Keith reads a lot of high-volume mailing lists by reading
    the subject lines in search mode and then doing "* -inbox". He likes
    that notmuch remembers that these messages are still unread, but if
    he later searches for a single message that happens to be in a giant
    thread of unread messages, then he wants to see just than one
    message, not all of them.

    Recommendation: Make notmuch-show-mode open *only* messages that
    match the search---not unread messages as well. At this point the
    unread tag becomes just a hint to the user and won't be explicitly
    handled differently by the interface, (other than that various
    commands will remove the unread tag if present). The unread tag is
    still useful for when searching for something like "I know I read
    this message recently".

    Followup: I wonder if I would miss one feature here. If I'm
    interrupted after reading part of a giant thread, currently I can
    quite and when I come back notmuch will remember right where I was
    while reading. One way to get this behavior back would be to make
    SPACE remove the inbox tag from each message its scrolled off. I'll
    have to think about that.

  * The current 'a' key in search-mode is unreliable. It seemed like a
    good idea to make 'a' only archive messages that match the search,
    but it's a flawed idea. Imagine the following scenario: Eric is
    reading his inbox and sees some threads related to a boring
    topic. He filters down to these with "f tag:boring". He's satisfied
    with the search results, and hits 'a' on each thread and even sees
    the "inbox" tag disappear from the presentation. But then when he
    returns to his inbox search and refreshes, the boring threads
    re-appear and have the inbox tag again. Ugh. The presentation is
    inconsistent and things just feel unreliable and broken.

    And a related issue:

  * The '*' key in search-mode doesn't provide any feedback that it has
    actually done anything, (none of the added/removed tags are changed
    in the presentation). And hitting '=' isn't necessarily ideal since
    it can make things irretrievably disappear, ('a' is different since
    it allows the user to confirm that things are good before making
    results disappear with '='). [*]

    Recommendation: Revert 'a' to act on all messages in a thread---not
    only those that match the search results. Then change '*' to work by
    walking the list and explicitly calling the same action as 'a' on
    each line. This will provide the desired feedback and should be
    plenty fast.

    Note: There are still use cases where the user might want to modify
    the tags only on messages matching the search, (think, "remove from
    inbox all messages from:someone"). So I'm aware that there's still a
    hole in functionality here, but I really want to fix the current
    inconsistency in the presentation. And I'm open to further
    suggestions here.

Let me know if any of the above seems crazy,

-Carl

PS. We also talked about new support for efficiently detecting file
addition, deletion, and renames. More on that when it becomes real.

[*] Yes, this is just a lame standin for a real undo feature. But until
we do have undo, it's an important standin.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Recent (and forthcoming) improvements to the emacs interface

[Marten Veldthuis]

On Thu, 03 Dec 2009 14:15:07 -0800, Carl Worth <cworth@cworth.org> wrote:
>   * Much nicer looking presentation, (no more ugly reverse-video or
>     underlines on the message summary line).
> 
>   * More reliable message-visibility buttons, (using RET in the first
>     column of a message-summary line now works).

Not sure what happened, but:

    Subject: [notmuch] [PATCH 4/9] Factor out message buffer mgmt from notmuch-show-view-all-mime-parts
    From: camalot@picnicpark.org
    To: notmuch@notmuchmail.org
    Cc: Keith Amidon <keith@nicira.com>
    Bcc: 
    Date: Fri, 27 Nov 2009 05:30:10 -0800

now collapses into:


    Subject: [notmuch] [PATCH 4/9] Factor out message buffer mgmt from notmuch-show-view-all-mime-parts
    From: Keith Amidon <keith@nicira.com>

on my system (notice the From: header).

-- 
- Marten

Re: Recent (and forthcoming) improvements to the emacs interface

[Aneesh Kumar K. V]

On Thu, 03 Dec 2009 14:15:07 -0800, Carl Worth <cworth@cworth.org> wrote:

.....

> 
>   * The magic space bar is too magic. Threads are separate conversations
>     so one key for paging through the current conversation shouldn't
>     also switch to the next conversation, (particularly when the
>     complementary key DEL doesn't reverse this behavior of SPACE).

agreed

> 
>     Recommendation: Make SPACE only page the current message. Recommend
>     that user use 'a' to advance to next thread, (or 'x' to exit back to
>     search results).

Later you mention 'N' and 'n' to do the same task. Or are you suggesting
that 'a' would move to the next task after marking the current task read ?

> 
>   * The unread tag is not handled transparently enough. Both Keith and
>     Eric complained of frequently being presented with messages as
>     "unread" that they had read before. (And I don't want to ever have
>     to manually think about whether to remove a thread as "unread".)
> 
>     Recommendation: Drop the 'A' and 'X' keybindings and make 'a' and
>     'x' mark remove the "unread" tag from all messages in the current
>     thread (as well as the "inbox" tag as currently). Also make 'n' and
>     'p' remove the "unread tag as well.


ok that explains. But with Xapian ticket 250 we would definitely want
some keybinding that move to the next mail without updating tags.


> 
>     Followup: This frees up 'N' and 'P', so I'd like to use the for
>     "next message" and "previous message" and make 'n' and 'p' do "next
>     open message" and "previous open message".
> 
>   * Opening up unread messages in notmuch-show mode is not
>     helpful. Keith reads a lot of high-volume mailing lists by reading
>     the subject lines in search mode and then doing "* -inbox". He likes
>     that notmuch remembers that these messages are still unread, but if
>     he later searches for a single message that happens to be in a giant
>     thread of unread messages, then he wants to see just than one
>     message, not all of them.
> 
>     Recommendation: Make notmuch-show-mode open *only* messages that
>     match the search---not unread messages as well. At this point the
>     unread tag becomes just a hint to the user and won't be explicitly
>     handled differently by the interface, (other than that various
>     commands will remove the unread tag if present). The unread tag is
>     still useful for when searching for something like "I know I read
>     this message recently".
> 
>     Followup: I wonder if I would miss one feature here. If I'm
>     interrupted after reading part of a giant thread, currently I can
>     quite and when I come back notmuch will remember right where I was
>     while reading. One way to get this behavior back would be to make
>     SPACE remove the inbox tag from each message its scrolled off. I'll
>     have to think about that.
> 
>   * The current 'a' key in search-mode is unreliable. It seemed like a
>     good idea to make 'a' only archive messages that match the search,
>     but it's a flawed idea. Imagine the following scenario: Eric is
>     reading his inbox and sees some threads related to a boring
>     topic. He filters down to these with "f tag:boring". He's satisfied
>     with the search results, and hits 'a' on each thread and even sees
>     the "inbox" tag disappear from the presentation. But then when he
>     returns to his inbox search and refreshes, the boring threads
>     re-appear and have the inbox tag again. Ugh. The presentation is
>     inconsistent and things just feel unreliable and broken.
> 
>     And a related issue:
> 
>   * The '*' key in search-mode doesn't provide any feedback that it has
>     actually done anything, (none of the added/removed tags are changed
>     in the presentation). And hitting '=' isn't necessarily ideal since
>     it can make things irretrievably disappear, ('a' is different since
>     it allows the user to confirm that things are good before making
>     results disappear with '='). [*]
> 
>     Recommendation: Revert 'a' to act on all messages in a thread---not
>     only those that match the search results. Then change '*' to work by
>     walking the list and explicitly calling the same action as 'a' on
>     each line. This will provide the desired feedback and should be
>     plenty fast.

With xapian ticket 250 doing a tag update per thread is going to be
really slow right ?


> 
>     Note: There are still use cases where the user might want to modify
>     the tags only on messages matching the search, (think, "remove from
>     inbox all messages from:someone"). So I'm aware that there's still a
>     hole in functionality here, but I really want to fix the current
>     inconsistency in the presentation. And I'm open to further
>     suggestions here.
> 
> Let me know if any of the above seems crazy,
> 

-aneesh

Re: Recent (and forthcoming) improvements to the emacs interface

[Aneesh Kumar K. V]

On Thu, 03 Dec 2009 14:15:07 -0800, Carl Worth <cworth@cworth.org> wrote:
> I just pushed out a nice set of changes to the emacs interface. Here's a
> quick summary of what you can expect to get when you next update:
> 
>   * Much nicer looking presentation, (no more ugly reverse-video or
>     underlines on the message summary line).
> 
>   * More reliable message-visibility buttons, (using RET in the first
>     column of a message-summary line now works).
> 
>   * Space bar fixed to advance to next open message, (it was originally
>     written this way, but has been broken since we changed from global
>     to local toggling of hidden message parts).
> 
>   * Showing a thread where the search matches only a subset of the
>     thread now opens only the matched messages (in addition to unread
>     messages).
> 
> This last feature is the big one---the rest all just happened to come
> along at the same time. One thing that I often do is read some giant
> thread and then tag a single message deep in that thread for dealing
> with later. And previously, doing a search for that tag would bring back
> the entire thread. Now, it opens only the message I'm actually looking
> for. So this is a very welcome change
> 
> And thanks to Bart Trojanowski for the groundwork for this change. I
> think the vim interface has had this feature for a while, (or would have
> if I had pushed all of Bart's changes earlier).
> 
> Meanwhile, Keith and Eric gave me some helpful feedback about the
> notmuch user-interface over lunch today, and in particular about the
> handling of the "unread" tag. Here are some of the things discussed,
> along with some things I'd like to change in response.
> 
> I'm open to suggestions on all of these, and most importantly, wanted to
> let people know about some upcoming user-interface changes before they
> were in place and potentially surprising.
> 
>   * The magic space bar is too magic. Threads are separate conversations
>     so one key for paging through the current conversation shouldn't
>     also switch to the next conversation, (particularly when the
>     complementary key DEL doesn't reverse this behavior of SPACE).
> 
>     Recommendation: Make SPACE only page the current message. Recommend
>     that user use 'a' to advance to next thread, (or 'x' to exit back to
>     search results).
> 
>   * The unread tag is not handled transparently enough. Both Keith and
>     Eric complained of frequently being presented with messages as
>     "unread" that they had read before. (And I don't want to ever have
>     to manually think about whether to remove a thread as "unread".)
> 
>     Recommendation: Drop the 'A' and 'X' keybindings and make 'a' and
>     'x' mark remove the "unread" tag from all messages in the current
>     thread (as well as the "inbox" tag as currently). Also make 'n' and
>     'p' remove the "unread tag as well.
> 
>     Followup: This frees up 'N' and 'P', so I'd like to use the for
>     "next message" and "previous message" and make 'n' and 'p' do "next
>     open message" and "previous open message".
> 
>   * Opening up unread messages in notmuch-show mode is not
>     helpful. Keith reads a lot of high-volume mailing lists by reading
>     the subject lines in search mode and then doing "* -inbox". He likes
>     that notmuch remembers that these messages are still unread, but if
>     he later searches for a single message that happens to be in a giant
>     thread of unread messages, then he wants to see just than one
>     message, not all of them.
> 
>     Recommendation: Make notmuch-show-mode open *only* messages that
>     match the search---not unread messages as well. At this point the
>     unread tag becomes just a hint to the user and won't be explicitly
>     handled differently by the interface, (other than that various
>     commands will remove the unread tag if present). The unread tag is
>     still useful for when searching for something like "I know I read
>     this message recently".
> 
>     Followup: I wonder if I would miss one feature here. If I'm
>     interrupted after reading part of a giant thread, currently I can
>     quite and when I come back notmuch will remember right where I was
>     while reading. One way to get this behavior back would be to make
>     SPACE remove the inbox tag from each message its scrolled off. I'll
>     have to think about that.
> 
>   * The current 'a' key in search-mode is unreliable. It seemed like a
>     good idea to make 'a' only archive messages that match the search,
>     but it's a flawed idea. Imagine the following scenario: Eric is
>     reading his inbox and sees some threads related to a boring
>     topic. He filters down to these with "f tag:boring". He's satisfied
>     with the search results, and hits 'a' on each thread and even sees
>     the "inbox" tag disappear from the presentation. But then when he
>     returns to his inbox search and refreshes, the boring threads
>     re-appear and have the inbox tag again. Ugh. The presentation is
>     inconsistent and things just feel unreliable and broken.
> 
>     And a related issue:
> 
>   * The '*' key in search-mode doesn't provide any feedback that it has
>     actually done anything, (none of the added/removed tags are changed
>     in the presentation). And hitting '=' isn't necessarily ideal since
>     it can make things irretrievably disappear, ('a' is different since
>     it allows the user to confirm that things are good before making
>     results disappear with '='). [*]
> 
>     Recommendation: Revert 'a' to act on all messages in a thread---not
>     only those that match the search results. Then change '*' to work by
>     walking the list and explicitly calling the same action as 'a' on
>     each line. This will provide the desired feedback and should be
>     plenty fast.

Can we also get a facility to temporarily mark a message and apply tags
on all marked message. In mutt terminology it is called 'tag'. Here is
the use case:
    When reading mailing list with high volume we would really want to
take a break in between reading mails or switch "folders". So if we do
a * +done that would mean we add "done" tag on all the mails. But what we
wanted was to do add done tag on the mails i looked till now. These marks
should not go to database so there is no I/O involved when selecting
the thread/mail.

-aneesh

Re: Recent (and forthcoming) improvements to the emacs interface

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1569 bytes --]

Hi Aneesh,

Thanks for the followup.

On Fri, 04 Dec 2009 14:08:52 +0530, "Aneesh Kumar K. V" <aneesh.kumar@linux.vnet.ibm.com> wrote:
> >     Recommendation: Make SPACE only page the current message. Recommend
> >     that user use 'a' to advance to next thread, (or 'x' to exit back to
> >     search results).
> 
> Later you mention 'N' and 'n' to do the same task. Or are you suggesting
> that 'a' would move to the next task after marking the current task
> read ?

Sorry, I meant for 'N' and 'P' to move between messages in a thread.

But it would make sense to also have commands to navigate to the next
and previous threads. So many actions and so few keys... :-}

> ok that explains. But with Xapian ticket 250 we would definitely want
> some keybinding that move to the next mail without updating tags.

I don't want to let a current bug shape the interface we want. But, yes,
that's a current reality.

> >     Recommendation: Revert 'a' to act on all messages in a thread---not
> >     only those that match the search results. Then change '*' to work by
> >     walking the list and explicitly calling the same action as 'a' on
> >     each line. This will provide the desired feedback and should be
> >     plenty fast.
> 
> With xapian ticket 250 doing a tag update per thread is going to be
> really slow right ?

Yes, but that's already the case with '*'. The Xapian work involved
should be the same whether calling "notmuch tag" once with the whole
search string, or several times, (once for each thread).

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Recent (and forthcoming) improvements to the emacs interface

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 371 bytes --]

On Fri, 04 Dec 2009 14:28:22 +0530, "Aneesh Kumar K. V" <aneesh.kumar@linux.vnet.ibm.com> wrote:
> Can we also get a facility to temporarily mark a message and apply tags
> on all marked message. In mutt terminology it is called 'tag'.

Would be nice, yes. Another thing we probably want is a way to modify
the tags on all threads within the current emacs region.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Recent (and forthcoming) improvements to the emacs interface

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 780 bytes --]

On Fri, 04 Dec 2009 03:06:32 +0100, Marten Veldthuis <marten@veldthuis.com> wrote:
> Not sure what happened, but:
> 
>     Subject: [notmuch] [PATCH 4/9] Factor out message buffer mgmt from notmuch-show-view-all-mime-parts
>     From: camalot@picnicpark.org
...
> now collapses into:
> 
>     Subject: [notmuch] [PATCH 4/9] Factor out message buffer mgmt from notmuch-show-view-all-mime-parts
>     From: Keith Amidon <keith@nicira.com>
> 
> on my system (notice the From: header).

I think that's actual text in the message, (not a header). Does that
look to you like what's happening?

And any ideas for making this kind of thing more clear? On my system,
the "From:" line that is in the message at least isn't colorized like
the actual headers are.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Emacs: Problem viewing a thread after reading it once interface

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1830 bytes --]

On Thu, 03 Dec 2009 14:15:07 -0800, Carl Worth <cworth@cworth.org> wrote:
>   * Showing a thread where the search matches only a subset of the
>     thread now opens only the matched messages (in addition to unread
>     messages).
> 
> This last feature is the big one---the rest all just happened to come
> along at the same time. One thing that I often do is read some giant
> thread and then tag a single message deep in that thread for dealing
> with later. And previously, doing a search for that tag would bring back
> the entire thread. Now, it opens only the message I'm actually looking
> for. So this is a very welcome change

Unfortunately, this change also introduces a major bug.

After I do a search such as tag:inbox and then view a resulting thread,
(and read it and archive it), then my search results still show that
thread result until I manually update the search view.

But, if I actually try to view that thread again from the search view,
it doesn't work. It previously worked since it would call "notmuch show"
with a query string of "thread:foo" but now calls it with "thread:foo
and tag:inbox" which now matches no messages.

The additional search term wasn't intended to change the returned
messages, (since we're passing the --entire-thread option to see all the
messages in the thread). It was only intended to restrict which messages
get the "match:1" marker added to them.

So maybe we need "notmuch show" to accept a second query string
to do something like:

	notmuch show tag:foo --matching tag:inbox

which will display all threads with messages matching "tag:foo" but then
mark only the messages matching "tag:inbox" with the "match:1" marker
for the UI to use.

What do you think, Bart? Did you run into a similar issue with the vim
interface?

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Emacs: Problem viewing a thread after reading it once interface

[Bart Trojanowski]

* Carl Worth <cworth@cworth.org> [091204 13:55]:
> So maybe we need "notmuch show" to accept a second query string
> to do something like:
> 
> 	notmuch show tag:foo --matching tag:inbox
> 
> which will display all threads with messages matching "tag:foo" but then
> mark only the messages matching "tag:inbox" with the "match:1" marker
> for the UI to use.

Right.  That would make sense.
thread ID, like say:

> What do you think, Bart? Did you run into a similar issue with the vim
> interface?

While I have not noticed before, yes... it's there.

-Bart



-- 
				WebSig: http://www.jukie.net/~bart/sig/