[PATCH] emacs: Make the queries used in the all-tags section

[PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1.1: Type: text/plain, Size: 565 bytes --]


From the commit message:

    emacs: Make queries used in the all-tags section configurable
    
    This patch adds a customization variable that controls what queries
    are used to construct the all-tags section in notmuch-hello. It allows
    the user to specify a function to construct the query given a tag. It
    also allows hiding tags by returning nil.

Possible uses would be things like displaying the unread count for
tags instead of all messages with that tag and/or hiding uninteresting
tags like "signed" or "encrypted".

-- Daniel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-emacs-Make-queries-used-in-the-all-tags-section-conf.patch --]
[-- Type: text/x-diff, Size: 2763 bytes --]

From bc4e046537518255ad02254c88e7b5bedd347c2b Mon Sep 17 00:00:00 2001
From: Daniel Schoepe <daniel.schoepe@googlemail.com>
Date: Fri, 20 May 2011 00:53:50 +0200
Subject: [PATCH] emacs: Make queries used in the all-tags section
 configurable

This patch adds a customization variable that controls what queries
are used to construct the all-tags section in notmuch-hello. It allows
the user to specify a function to construct the query given a tag. It
also allows hiding tags by returning nil.

Signed-off-by: Daniel Schoepe <daniel.schoepe@googlemail.com>
---
 emacs/notmuch-hello.el |   25 ++++++++++++++++++++++---
 1 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index e58dd24..0b8c0b9 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -55,6 +55,17 @@
   :type 'boolean
   :group 'notmuch)
 
+(defcustom notmuch-hello-tag-list-make-query nil
+  "Function to generate queries for the all tags list.
+
+This variable controls which query results are shown for each tag
+in the \"all tags\" list. If this is nil, it uses the default of
+\"tag:TAG\", otherwise it should be a function of one argument,
+the tag, that should return the query to be used for that tag, or
+nil, in which case the tag will be hidden."
+  :type '(choice function (const nil :tag "tag:TAG"))
+  :group 'notmuch)
+
 (defface notmuch-hello-logo-background
   '((((class color)
       (background dark))
@@ -318,6 +329,16 @@ Complete list of currently available key bindings:
  ;;(setq buffer-read-only t)
 )
 
+(defun notmuch-hello-generate-tag-alist ()
+  "Return an alist from tags to queries to display in the all-tags section."
+  (filter 'cdr
+	  (mapcar '(lambda (tag)
+		     (cons tag
+			   (if notmuch-hello-tag-list-make-query
+			       (funcall notmuch-hello-tag-list-make-query tag)
+			     (concat "tag:" tag))))
+		  (process-lines notmuch-command "search-tags"))))
+
 ;;;###autoload
 (defun notmuch-hello (&optional no-display)
   "Run notmuch and display saved searches, known tags, etc."
@@ -396,9 +417,7 @@ Complete list of currently available key bindings:
 		      if (> (string-to-number (notmuch-saved-search-count (cdr elem))) 0)
 		      collect elem)))
 	     (saved-widest (notmuch-hello-longest-label saved-alist))
-	     (alltags-alist (if notmuch-show-all-tags-list
-				(mapcar '(lambda (tag) (cons tag (concat "tag:" tag)))
-					(process-lines notmuch-command "search-tags"))))
+	     (alltags-alist (if notmuch-show-all-tags-list (notmuch-hello-generate-tag-alist)))
 	     (alltags-widest (notmuch-hello-longest-label alltags-alist))
 	     (widest (max saved-widest alltags-widest)))
 
-- 
1.7.5.1


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]

On Fri, 20 May 2011 01:18:35 +0200, Daniel Schoepe <daniel.schoepe@googlemail.com> wrote:
> From the commit message:
> 
>     emacs: Make queries used in the all-tags section configurable
>     
>     This patch adds a customization variable that controls what queries
>     are used to construct the all-tags section in notmuch-hello. It allows
>     the user to specify a function to construct the query given a tag. It
>     also allows hiding tags by returning nil.

This seems like a useful feature, but perhaps it's a little too general?

I'm imagining a user wanting to use this functionality but not knowing
anything about writing an emacs-lisp function. For such a user, this
variable won't provide much of a feature.

I think that might be an argument for dropping this variable from the
notmuch customization group. That customization page is starting to get
crowded, and if there are things there that can't be easily manipulated
with the available controls, I think they're mostly just clutter.

Perhaps this could be addressed by allowing this variable to be an alist
instead of (or in addition to) a function. What do you think?

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

On Tue, 24 May 2011 13:39:44 -0700, Carl Worth <cworth@cworth.org> wrote:
> This seems like a useful feature, but perhaps it's a little too general?
> 
> I'm imagining a user wanting to use this functionality but not knowing
> anything about writing an emacs-lisp function. For such a user, this
> variable won't provide much of a feature.
>
> I think that might be an argument for dropping this variable from the
> notmuch customization group. That customization page is starting to get
> crowded, and if there are things there that can't be easily manipulated
> with the available controls, I think they're mostly just clutter.
> 
> Perhaps this could be addressed by allowing this variable to be an alist
> instead of (or in addition to) a function. What do you think?

Yes, allowing both, an alist or a function for this variable seems
better than forcing the user to write a function, but how should the
alist control the behaviour? I don't think people would like having to
specify a query for each of their tags, and since, by assumption, they
cannot write a function to automate this, they probably wouldn't be much
better off.  (Except of course, if they only wanted to change it for a
few individual tags).

Another option would be to also allow various symbols like 'unread (meaning
"tag:TAG and tag:unread") for "popular" queries in addition to a function.

Cheers,
Daniel

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

On Tue, 24 May 2011 23:01:16 +0200, Daniel Schoepe <daniel.schoepe@googlemail.com> wrote:
> Another option would be to also allow various symbols like 'unread (meaning
> "tag:TAG and tag:unread") for "popular" queries in addition to a
> function.

Or, for a case like this the variable could accept a simple string to
append:

	"and tag:unread"

That would be easy to use and still quite flexible.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Austin Clements]

Out of curiosity, what use cases do you envision for this?  So far
I've only heard two, both of which seem like great ideas, but neither
of which require such a heavy-handed solution: displaying unread
counts for tags rather than total counts, and hiding unused tags.

I would argue that we *always* want to display unread counts (maybe in
addition to total counts, or maybe not).  In fact, I'm generally
surprised by how little notmuch treats the "unread" specially, given
how important that tag is to the user.  For example, I would similarly
find unread counts in the search results far more useful than the
count of messages matching the query.

Hiding unused tags also seems like a genuinely useful feature, and
could be accomplished with a very simple customization UI (perhaps
even linked directly from the hello buffer).

It seems to me like anything more sophisticated is better suited to a
saved search.

On Thu, May 19, 2011 at 7:18 PM, Daniel Schoepe
<daniel.schoepe@googlemail.com> wrote:
>
> From the commit message:
>
>    emacs: Make queries used in the all-tags section configurable
>
>    This patch adds a customization variable that controls what queries
>    are used to construct the all-tags section in notmuch-hello. It allows
>    the user to specify a function to construct the query given a tag. It
>    also allows hiding tags by returning nil.
>
> Possible uses would be things like displaying the unread count for
> tags instead of all messages with that tag and/or hiding uninteresting
> tags like "signed" or "encrypted".
>
> -- Daniel

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 2010 bytes --]

On Wed, 25 May 2011 00:10:43 -0400, Austin Clements <amdragon@mit.edu> wrote:
> Out of curiosity, what use cases do you envision for this?  So far
> I've only heard two, both of which seem like great ideas, but neither
> of which require such a heavy-handed solution: displaying unread
> counts for tags rather than total counts, and hiding unused tags.

Another thing I use this for, is to hide messages/threads with a
"killed"-tag.

Anyway, I don't think this solution is very "heavy-handed", since it
doesn't add any significant complexity to the UI code and for people who
know elisp, writing such a query-construction function is not something
"heavy-weight" either, so I don't think giving up the extra flexibility
is really worth it.

The only problem I see is what Carl Worth mentioned: Users unaccustomed
to elisp can't do much with this configuration variable.

> I would argue that we *always* want to display unread counts (maybe in
> addition to total counts, or maybe not).  In fact, I'm generally
> surprised by how little notmuch treats the "unread" specially, given
> how important that tag is to the user.  For example, I would similarly
> find unread counts in the search results far more useful than the
> count of messages matching the query.

I think most users would agree with this, but I still think we
should keep/make it configurable.

> Hiding unused tags also seems like a genuinely useful feature, and
> could be accomplished with a very simple customization UI (perhaps
> even linked directly from the hello buffer).
> 
> It seems to me like anything more sophisticated is better suited to a
> saved search.

I think a sensible compromise would be to allow either a function or a
string that is appended (which people could set to "and tag:unread") for
the proposed configuration variable and additionally to add a
variable that lists tags that should be hidden (which would also be
easily modifiable in M-x customize).

Cheers,
Daniel

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Austin Clements]

On Wed, May 25, 2011 at 6:04 AM, Daniel Schoepe
<daniel.schoepe@googlemail.com> wrote:
> On Wed, 25 May 2011 00:10:43 -0400, Austin Clements <amdragon@mit.edu> wrote:
>> Out of curiosity, what use cases do you envision for this?  So far
>> I've only heard two, both of which seem like great ideas, but neither
>> of which require such a heavy-handed solution: displaying unread
>> counts for tags rather than total counts, and hiding unused tags.
>
> Another thing I use this for, is to hide messages/threads with a
> "killed"-tag.

Ah, interesting.

> I think a sensible compromise would be to allow either a function or a
> string that is appended (which people could set to "and tag:unread") for
> the proposed configuration variable and additionally to add a
> variable that lists tags that should be hidden (which would also be
> easily modifiable in M-x customize).

In principle, I completely agree, but the little parser in my head
screams "parse error!  parse error!" when I feed it "and tag:unread"
and that bothers me.

May I suggest a slightly different way of looking at this that will
quell my inner parser?  Instead of configuring a weird "query
fragment" like "and tag:unread" to be string-concatenated with the tag
query, configure a *filter* query like merely "tag:unread" that
narrows down what you'd like to be counted within the scope of a tag.
The implementations are hardly different---simply generate the query
"tag:<tag> and ( <filter> )"---but a filter is a well-formed query,
not some string fragment.  Furthermore, the user can't get bitten by
precedence and wind up with a query that counts messages that don't
even have that tag.

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1.1: Type: text/plain, Size: 790 bytes --]

On Wed, 25 May 2011 10:44:51 -0400, Austin Clements <amdragon@mit.edu> wrote:
> May I suggest a slightly different way of looking at this that will
> quell my inner parser?  Instead of configuring a weird "query
> fragment" like "and tag:unread" to be string-concatenated with the tag
> query, configure a *filter* query like merely "tag:unread" that
> narrows down what you'd like to be counted within the scope of a tag.
> The implementations are hardly different---simply generate the query
> "tag:<tag> and ( <filter> )"---but a filter is a well-formed query,
> not some string fragment.  Furthermore, the user can't get bitten by
> precedence and wind up with a query that counts messages that don't
> even have that tag.

Good point, I attached an updated patch that implements this.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-emacs-Make-queries-used-in-the-all-tags-section-conf.patch --]
[-- Type: text/x-diff, Size: 3394 bytes --]

From 6a4a5fd3e2a1baa25f86f85d6c3324db26582b82 Mon Sep 17 00:00:00 2001
From: Daniel Schoepe <daniel.schoepe@googlemail.com>
Date: Fri, 20 May 2011 00:53:50 +0200
Subject: [PATCH] emacs: Make queries used in the all-tags section
 configurable

This patch adds a customization variable that controls what queries
are used to construct the all-tags section in notmuch-hello. It allows
the user to specify a function to construct the query given a tag or
a string that is used as a filter for each tag.
It also adds a variable to hide various tags from the all-tags section.

Signed-off-by: Daniel Schoepe <daniel.schoepe@googlemail.com>
---
 emacs/notmuch-hello.el |   37 ++++++++++++++++++++++++++++++++++---
 1 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index e58dd24..a513bb0 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -55,6 +55,24 @@
   :type 'boolean
   :group 'notmuch)
 
+(defcustom notmuch-hello-tag-list-make-query nil
+  "Function or string to generate queries for the all tags list.
+
+This variable controls which query results are shown for each tag
+in the \"all tags\" list. It can be nil (for default behaviour,
+displaying all messages for a tag), a string that is used as a
+filter for messages having that tag (equivalent to \"tag:TAG
+and (THIS-VARIABLE)\", or a function that is given a tag and
+should return the query that is to be used for the tag. If it
+returns nil, the corresponding tag will be hidden."
+  :type '(choice (const nil :tag "tag:TAG") string function)
+  :group 'notmuch)
+
+(defcustom notmuch-hello-hide-tags nil
+  "List of tags to be hidden in the \"all tags\"-section."
+  :type '(repeat string)
+  :group 'notmuch)
+
 (defface notmuch-hello-logo-background
   '((((class color)
       (background dark))
@@ -318,6 +336,21 @@ Complete list of currently available key bindings:
  ;;(setq buffer-read-only t)
 )
 
+(defun notmuch-hello-generate-tag-alist ()
+  "Return an alist from tags to queries to display in the all-tags section."
+  (filter 'cdr
+	  (mapcar '(lambda (tag)
+		     (cons tag
+			   (cond
+			    ((functionp notmuch-hello-tag-list-make-query)
+			     (funcall notmuch-hello-tag-list-make-query tag))
+			    ((stringp notmuch-hello-tag-list-make-query)
+			     (concat "tag:" tag " and ("
+				     notmuch-hello-tag-list-make-query ")"))
+			    (t (concat "tag:" tag)))))
+		  (filter (lambda (tag) (not (member tag notmuch-hello-hide-tags)))
+			  (process-lines notmuch-command "search-tags")))))
+
 ;;;###autoload
 (defun notmuch-hello (&optional no-display)
   "Run notmuch and display saved searches, known tags, etc."
@@ -396,9 +429,7 @@ Complete list of currently available key bindings:
 		      if (> (string-to-number (notmuch-saved-search-count (cdr elem))) 0)
 		      collect elem)))
 	     (saved-widest (notmuch-hello-longest-label saved-alist))
-	     (alltags-alist (if notmuch-show-all-tags-list
-				(mapcar '(lambda (tag) (cons tag (concat "tag:" tag)))
-					(process-lines notmuch-command "search-tags"))))
+	     (alltags-alist (if notmuch-show-all-tags-list (notmuch-hello-generate-tag-alist)))
 	     (alltags-widest (notmuch-hello-longest-label alltags-alist))
 	     (widest (max saved-widest alltags-widest)))
 
-- 
1.7.5.1


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1.1: Type: text/plain, Size: 134 bytes --]


I accidentally used `filter' in the previous patch which isn't defined
by default during runtime, updated version in the attachment.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-emacs-Make-queries-used-in-the-all-tags-section-conf.patch --]
[-- Type: text/x-diff, Size: 4576 bytes --]

From e8f5e997b7082a50f75d7329a69fccc126a21e5e Mon Sep 17 00:00:00 2001
From: Daniel Schoepe <daniel.schoepe@googlemail.com>
Date: Fri, 20 May 2011 00:53:50 +0200
Subject: [PATCH] emacs: Make queries used in the all-tags section
 configurable

This patch adds a customization variable that controls what queries
are used to construct the all-tags section in notmuch-hello. It allows
the user to specify a function to construct the query given a tag or
a string that is used as a filter for each tag.
It also adds a variable to hide various tags from the all-tags section.

Signed-off-by: Daniel Schoepe <daniel.schoepe@googlemail.com>
---
 emacs/notmuch-hello.el |   38 +++++++++++++++++++++++++++++++++++---
 emacs/notmuch-lib.el   |   12 ++++++++++++
 2 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index e58dd24..b3f133c 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -55,6 +55,24 @@
   :type 'boolean
   :group 'notmuch)
 
+(defcustom notmuch-hello-tag-list-make-query nil
+  "Function or string to generate queries for the all tags list.
+
+This variable controls which query results are shown for each tag
+in the \"all tags\" list. It can be nil (for default behaviour,
+displaying all messages for a tag), a string that is used as a
+filter for messages having that tag (equivalent to \"tag:TAG
+and (THIS-VARIABLE)\", or a function that is given a tag and
+should return the query that is to be used for the tag. If it
+returns nil, the corresponding tag will be hidden."
+  :type '(choice (const nil :tag "tag:TAG") string function)
+  :group 'notmuch)
+
+(defcustom notmuch-hello-hide-tags nil
+  "List of tags to be hidden in the \"all tags\"-section."
+  :type '(repeat string)
+  :group 'notmuch)
+
 (defface notmuch-hello-logo-background
   '((((class color)
       (background dark))
@@ -318,6 +336,22 @@ Complete list of currently available key bindings:
  ;;(setq buffer-read-only t)
 )
 
+(defun notmuch-hello-generate-tag-alist ()
+  "Return an alist from tags to queries to display in the all-tags section."
+  (notmuch-filter
+   'cdr
+   (mapcar '(lambda (tag)
+	      (cons tag
+		    (cond
+		     ((functionp notmuch-hello-tag-list-make-query)
+		      (funcall notmuch-hello-tag-list-make-query tag))
+		     ((stringp notmuch-hello-tag-list-make-query)
+		      (concat "tag:" tag " and ("
+			      notmuch-hello-tag-list-make-query ")"))
+		     (t (concat "tag:" tag)))))
+	   (notmuch-filter (lambda (tag) (not (member tag notmuch-hello-hide-tags)))
+			   (process-lines notmuch-command "search-tags")))))
+
 ;;;###autoload
 (defun notmuch-hello (&optional no-display)
   "Run notmuch and display saved searches, known tags, etc."
@@ -396,9 +430,7 @@ Complete list of currently available key bindings:
 		      if (> (string-to-number (notmuch-saved-search-count (cdr elem))) 0)
 		      collect elem)))
 	     (saved-widest (notmuch-hello-longest-label saved-alist))
-	     (alltags-alist (if notmuch-show-all-tags-list
-				(mapcar '(lambda (tag) (cons tag (concat "tag:" tag)))
-					(process-lines notmuch-command "search-tags"))))
+	     (alltags-alist (if notmuch-show-all-tags-list (notmuch-hello-generate-tag-alist)))
 	     (alltags-widest (notmuch-hello-longest-label alltags-alist))
 	     (widest (max saved-widest alltags-widest)))
 
diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index cc80fb2..a3195ff 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -120,6 +120,18 @@ within the current window."
       (or (memq prop buffer-invisibility-spec)
 	  (assq prop buffer-invisibility-spec)))))
 
+;; notmuch- prefix to avoid clashes with user-defined filter-functions:
+;; For instance, the filter-function suggested in the elisp cookbook
+;; on the emacswiki always filters out elements that are nil
+;; even if the predicate returns t for nil; hence this has slightly
+;; different semantics than what some users have defined for themselves.
+(defun notmuch-filter (pred lst)
+  "Return a list containing all elements from LST that satisfy PRED."
+  (and lst
+     (if (funcall pred (car lst))
+	 (cons (car lst) (notmuch-filter pred (cdr lst)))
+       (notmuch-filter pred (cdr lst)))))
+
 ; This lets us avoid compiling these replacement functions when emacs
 ; is sufficiently new enough to supply them alone. We do the macro
 ; treatment rather than just wrapping our defun calls in a when form
-- 
1.7.5.1


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Austin Clements]

On Wed, May 25, 2011 at 1:56 PM, Daniel Schoepe
<daniel.schoepe@googlemail.com> wrote:
> I accidentally used `filter' in the previous patch which isn't defined
> by default during runtime, updated version in the attachment.

Cool.  My inner parser is happy.

A few comments on the code:

> +(defcustom notmuch-hello-tag-list-make-query nil
> +  "Function or string to generate queries for the all tags list.
> +
> +This variable controls which query results are shown for each tag
> +in the \"all tags\" list. It can be nil (for default behaviour,
> +displaying all messages for a tag), a string that is used as a
> +filter for messages having that tag (equivalent to \"tag:TAG
> +and (THIS-VARIABLE)\", or a function that is given a tag and
> +should return the query that is to be used for the tag. If it
> +returns nil, the corresponding tag will be hidden."
> +  :type '(choice (const nil :tag "tag:TAG") string function)
> +  :group 'notmuch)

At least in Emacs 23.3.1, it has to be (const :tag "tag:TAG" nil).  I
didn't think the order mattered, but the tag didn't display otherwise.
 It would also be good to give descriptive tags to the other choices.

It would be more consistent if the function form of this also returned
a filter expression, rather than a whole expression.  This also
simplifies the documentation.

So, perhaps something like

(defcustom notmuch-hello-tag-list-counts nil
  "Method for generating counts displayed in the all tags list.

This variable controls the query used to generate counts for each
tag in the \"all tags\" list.  If nil, the tag list will count
all messages with each tag.  This can be a query string that will
filter the messages counted for each tag.  Finally, this can be a
function that will be called for each tag and should return a
filter query for that tag, or nil to hide the tag."
  :type '(choice (const :tag "Count all messages" nil)
                 (const :tag "Count unread messages" "tag:unread")
                 (const :tag "Custom filter" string)
                 (const :tag "Custom filter function" function))
  :group 'notmuch)

> +(defun notmuch-hello-generate-tag-alist ()
> +  "Return an alist from tags to queries to display in the all-tags section."
> +  (notmuch-filter
> +   'cdr
> +   (mapcar '(lambda (tag)

You don't need the quote before a lambda form.  (You can also change
'cdr to #'cdr to further hint the compiler, though it appears to not
matter in this case.)

> +(defun notmuch-filter (pred lst)
> +  "Return a list containing all elements from LST that satisfy PRED."

notmuch-remove-if-not would be more canonical (yeah, it's unwieldy,
blame Common Lisp).  Also, since Elisp doesn't do tail-recursion, the
standard way to define a remove-if-not function is

(defun notmuch-remove-if-not (predicate list)
  "Return a copy of LIST with all items not satisfying PREDICATE removed."
  (let (out)
    (while list
      (when (funcall predicate (car list))
        (push (car list) out))
      (setq list (cdr list)))
    (nreverse out)))

(Why oh why Elisp hasn't just made remove-if and remove-if-not
standard I don't know; they're redefined all over the Elisp code base.
 Any everybody's afraid to use cl-seq's remove-if-not because it's so
ridiculously complicated.)

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1.1: Type: text/plain, Size: 3250 bytes --]

On Wed, 25 May 2011 15:11:16 -0400, Austin Clements <amdragon@mit.edu> wrote:
> At least in Emacs 23.3.1, it has to be (const :tag "tag:TAG" nil).  I
> didn't think the order mattered, but the tag didn't display otherwise.
>  It would also be good to give descriptive tags to the other choices.
> 
> It would be more consistent if the function form of this also returned
> a filter expression, rather than a whole expression.  This also
> simplifies the documentation.

Thanks for the suggestions. (At first, I wanted to allow the user
to be able to also include messages that don't have that tag, but this
really no longer belongs in the all tags-section).

> So, perhaps something like
>
> (defcustom notmuch-hello-tag-list-counts nil
>   "Method for generating counts displayed in the all tags list.
> 
> This variable controls the query used to generate counts for each
> tag in the \"all tags\" list.  If nil, the tag list will count
> all messages with each tag.  This can be a query string that will
> filter the messages counted for each tag.  Finally, this can be a
> function that will be called for each tag and should return a
> filter query for that tag, or nil to hide the tag."
>   :type '(choice (const :tag "Count all messages" nil)
>                  (const :tag "Count unread messages" "tag:unread")
>                  (const :tag "Custom filter" string)
>                  (const :tag "Custom filter function" function))
>   :group 'notmuch)

That is slightly less accurate though, since the query is not only used
to generate the counts, but also the searches that are performed when
one presses return on a tag in the list.

> > +(defun notmuch-hello-generate-tag-alist ()
> > +  "Return an alist from tags to queries to display in the all-tags section."
> > +  (notmuch-filter
> > +   'cdr
> > +   (mapcar '(lambda (tag)
> 
> You don't need the quote before a lambda form.  (You can also change
> 'cdr to #'cdr to further hint the compiler, though it appears to not
> matter in this case.)

Oh, yeah, normally I'm aware of that, don't know what made me quote it
at the time I wrote it.

> > +(defun notmuch-filter (pred lst)
> > +  "Return a list containing all elements from LST that satisfy PRED."
> 
> notmuch-remove-if-not would be more canonical (yeah, it's unwieldy,
> blame Common Lisp).  Also, since Elisp doesn't do tail-recursion, the
> standard way to define a remove-if-not function is
> 
> (defun notmuch-remove-if-not (predicate list)
>   "Return a copy of LIST with all items not satisfying PREDICATE removed."
>   (let (out)
>     (while list
>       (when (funcall predicate (car list))
>         (push (car list) out))
>       (setq list (cdr list)))
>     (nreverse out)))
> 
> (Why oh why Elisp hasn't just made remove-if and remove-if-not
> standard I don't know; they're redefined all over the Elisp code base.
>  Any everybody's afraid to use cl-seq's remove-if-not because it's so
> ridiculously complicated.)

Ah, thanks. I guess too much exposure to Haskell (and how smart GHC is)
makes me write things in a functional style without thinking about
performance.

I incorporated most of you recommendations in the attached patch.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-emacs-Make-queries-used-in-the-all-tags-section-conf.patch --]
[-- Type: text/x-diff, Size: 4426 bytes --]

From 2056a455cdec6b980426552b8dcd7055f6cd0805 Mon Sep 17 00:00:00 2001
From: Daniel Schoepe <daniel.schoepe@googlemail.com>
Date: Fri, 20 May 2011 00:53:50 +0200
Subject: [PATCH] emacs: Make queries used in the all-tags section
 configurable

This patch adds a customization variable that controls what queries
are used to construct the all-tags section in notmuch-hello. It allows
the user to specify a function to construct the query given a tag or
a string that is used as a filter for each tag.
It also adds a variable to hide various tags from the all-tags section.

Signed-off-by: Daniel Schoepe <daniel.schoepe@googlemail.com>
---
 emacs/notmuch-hello.el |   43 ++++++++++++++++++++++++++++++++++++++++---
 emacs/notmuch-lib.el   |    9 +++++++++
 2 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index e58dd24..bd95849 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -55,6 +55,26 @@
   :type 'boolean
   :group 'notmuch)
 
+(defcustom notmuch-hello-tag-list-make-query nil
+  "Function or string to generate queries for the all tags list.
+
+This variable controls which query results are shown for each tag
+in the \"all tags\" list. If nil, it will use all messages with
+that tag. If this is set to a string, it is used as a filter for
+messages having that tag (equivalent to \"tag:TAG and (THIS-VARIABLE)\").
+Finally this can be a function that will be called for each tag and
+should return a filter for that tag, or nil to hide the tag."
+  :type '(choice (const :tag "All messages" nil)
+		 (const :tag "Unread messages" "tag:unread")
+		 (const :tag "Custom filter" string)
+		 (const :tag "Custom filter function" function))
+  :group 'notmuch)
+
+(defcustom notmuch-hello-hide-tags nil
+  "List of tags to be hidden in the \"all tags\"-section."
+  :type '(repeat string)
+  :group 'notmuch)
+
 (defface notmuch-hello-logo-background
   '((((class color)
       (background dark))
@@ -318,6 +338,25 @@ Complete list of currently available key bindings:
  ;;(setq buffer-read-only t)
 )
 
+(defun notmuch-hello-generate-tag-alist ()
+  "Return an alist from tags to queries to display in the all-tags section."
+  (notmuch-remove-if-not
+   #'cdr
+   (mapcar (lambda (tag)
+	     (cons tag
+		   (cond
+		    ((functionp notmuch-hello-tag-list-make-query)
+		     (concat "tag:" tag " and ("
+			     (funcall notmuch-hello-tag-list-make-query tag) ")"))
+		    ((stringp notmuch-hello-tag-list-make-query)
+		     (concat "tag:" tag " and ("
+			     notmuch-hello-tag-list-make-query ")"))
+		    (t (concat "tag:" tag)))))
+	   (notmuch-remove-if-not
+	    (lambda (tag)
+	      (not (member tag notmuch-hello-hide-tags)))
+	    (process-lines notmuch-command "search-tags")))))
+
 ;;;###autoload
 (defun notmuch-hello (&optional no-display)
   "Run notmuch and display saved searches, known tags, etc."
@@ -396,9 +435,7 @@ Complete list of currently available key bindings:
 		      if (> (string-to-number (notmuch-saved-search-count (cdr elem))) 0)
 		      collect elem)))
 	     (saved-widest (notmuch-hello-longest-label saved-alist))
-	     (alltags-alist (if notmuch-show-all-tags-list
-				(mapcar '(lambda (tag) (cons tag (concat "tag:" tag)))
-					(process-lines notmuch-command "search-tags"))))
+	     (alltags-alist (if notmuch-show-all-tags-list (notmuch-hello-generate-tag-alist)))
 	     (alltags-widest (notmuch-hello-longest-label alltags-alist))
 	     (widest (max saved-widest alltags-widest)))
 
diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index cc80fb2..d5ca0f4 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -120,6 +120,15 @@ within the current window."
       (or (memq prop buffer-invisibility-spec)
 	  (assq prop buffer-invisibility-spec)))))
 
+(defun notmuch-remove-if-not (predicate list)
+  "Return a copy of LIST with all items not satisfying PREDICATE removed."
+  (let (out)
+    (while list
+      (when (funcall predicate (car list))
+        (push (car list) out))
+      (setq list (cdr list)))
+    (nreverse out)))
+
 ; This lets us avoid compiling these replacement functions when emacs
 ; is sufficiently new enough to supply them alone. We do the macro
 ; treatment rather than just wrapping our defun calls in a when form
-- 
1.7.5.1


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Austin Clements]

On Wed, May 25, 2011 at 5:21 PM, Daniel Schoepe
<daniel.schoepe@googlemail.com> wrote:
> On Wed, 25 May 2011 15:11:16 -0400, Austin Clements <amdragon@mit.edu> wrote:
>> So, perhaps something like
>>
>> (defcustom notmuch-hello-tag-list-counts nil
>>   "Method for generating counts displayed in the all tags list.
>>
>> This variable controls the query used to generate counts for each
>> tag in the \"all tags\" list.  If nil, the tag list will count
>> all messages with each tag.  This can be a query string that will
>> filter the messages counted for each tag.  Finally, this can be a
>> function that will be called for each tag and should return a
>> filter query for that tag, or nil to hide the tag."
>>   :type '(choice (const :tag "Count all messages" nil)
>>                  (const :tag "Count unread messages" "tag:unread")
>>                  (const :tag "Custom filter" string)
>>                  (const :tag "Custom filter function" function))
>>   :group 'notmuch)
>
> That is slightly less accurate though, since the query is not only used
> to generate the counts, but also the searches that are performed when
> one presses return on a tag in the list.

'Doh.  That's what I get for not reading the surrounding code.  I
misunderstood what your patch was going for and assumed it was what
*I* wanted notmuch to do, which is to show me useful counts (e.g.,
unread count), but not to change the underlying query.  ]:--8)

So, to me, it seems like this turns the all tags section into another
saved searches section, just with a slight twist, and makes me wonder
if there's a better way to reconcile these.

> Ah, thanks. I guess too much exposure to Haskell (and how smart GHC is)
> makes me write things in a functional style without thinking about
> performance.

Hah, been there.  I have to constantly remind myself that Elisp is
*not* a self-respecting functional programming language, despite how
much it may look like Scheme.

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

On Wed, 25 May 2011 18:42:30 -0400, Austin Clements <amdragon@mit.edu> wrote:
> 'Doh.  That's what I get for not reading the surrounding code.  I
> misunderstood what your patch was going for and assumed it was what
> *I* wanted notmuch to do, which is to show me useful counts (e.g.,
> unread count), but not to change the underlying query.  ]:--8)

I thought about that too, but figured it should be rather rare that
someone wants only a portion of some messages _counted_, but all
displayed when clicking on the search next to the count.

I'm somewhat indifferent, since I rarely use those links
directly, so any more opinions on this are very much appreciated.

> So, to me, it seems like this turns the all tags section into another
> saved searches section, just with a slight twist, and makes me wonder
> if there's a better way to reconcile these.

Well, it already was sort of a non-configurable saved searches section
before, so I didn't really turn it into one. :)

Since the main difference between those sections is the clear visual
distinction, it might be an option to provide the user with functions to
easily declare new sections for the hello screen, where this sort of
thing is configurable. (One possible use that comes to mind would be to
group tags into different categories.)

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

On Thu, 26 May 2011 01:21:17 +0200, Daniel Schoepe <daniel.schoepe@googlemail.com> wrote:
> Since the main difference between those sections is the clear visual
> distinction, it might be an option to provide the user with functions to
> easily declare new sections for the hello screen, where this sort of
> thing is configurable. 

Yes!

This sounds very much like what I want myself. (I'm planning to write a
new "How I process mail" post summarizing some things I've learned about
notmuch and my personal habits over the past two months.)

I definitely have two very distinct groups of saved searches based on
how I use them. And it would be nice to have them visually separated in
the interface.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On Wed, 25 May 2011 23:21:54 +0200, Daniel Schoepe <daniel.schoepe@googlemail.com> wrote:
> I incorporated most of you recommendations in the attached patch.

Looks good, thanks!

I've pushed this out now.

I only have two suggestions:

1. Could you please add some basic testing for this?

   The existing notmuch-hello tests should make a good starting point.

2. The hello screen is now lying a bit by saying "all tags".

   Perhaps if the user has the notmuch-hello-hide-tags variable set to
   non-nil the text might change somewhat? I'm not sure to what exactly.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

On 05/26/2011 06:04 PM, Carl Worth wrote:
> 2. The hello screen is now lying a bit by saying "all tags".
> 
>    Perhaps if the user has the notmuch-hello-hide-tags variable set to
>    non-nil the text might change somewhat? I'm not sure to what exactly.

maybe "all non-hidden tags" ?

	--dkg


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 1030 bytes --]

Re: [PATCH] emacs: Make the queries used in the all-tags section

[Austin Clements]

[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]

On May 25, 2011 7:21 PM, "Daniel Schoepe" <daniel.schoepe@googlemail.com>
wrote:
>
> On Wed, 25 May 2011 18:42:30 -0400, Austin Clements <amdragon@mit.edu>
wrote:
> > 'Doh.  That's what I get for not reading the surrounding code.  I
> > misunderstood what your patch was going for and assumed it was what
> > *I* wanted notmuch to do, which is to show me useful counts (e.g.,
> > unread count), but not to change the underlying query.  ]:--8)
>
> I thought about that too, but figured it should be rather rare that
> someone wants only a portion of some messages _counted_, but all
> displayed when clicking on the search next to the count.

My use case of counting only unread messages (but still showing all of them,
like just about every other mail client out there) is an example of this.
But you're right that I can't think of any other examples (which is why I
suggested just baking this specific thing in earlier).

> I'm somewhat indifferent, since I rarely use those links
> directly, so any more opinions on this are very much appreciated.

Actually, here's a new thought that may help address your original problem
(assuming I understand it correctly now).  One of my design criteria for the
custom query parser (need to get back to those patches...) was to support
"implicit tag filters" such as hiding everything with tag:spam or tag:killed
(or tag:muted, etc) unless these tags are specifically requested by a
query.  That would apply equally to generated queries like the all tags
list, so you could filter out tag:killed messages centrally using a
mechanism like this, rather than having to work it in to every query
somehow.

> > So, to me, it seems like this turns the all tags section into another
> > saved searches section, just with a slight twist, and makes me wonder
> > if there's a better way to reconcile these.
>
> Well, it already was sort of a non-configurable saved searches section
> before, so I didn't really turn it into one. :)

True.

> Since the main difference between those sections is the clear visual
> distinction, it might be an option to provide the user with functions to
> easily declare new sections for the hello screen, where this sort of
> thing is configurable. (One possible use that comes to mind would be to
> group tags into different categories.)

That would be awesome.

[-- Attachment #2: Type: text/html, Size: 2738 bytes --]