[PATCH 0/7] emacs: hello: custom display of saved searches

[PATCH 0/7] emacs: hello: custom display of saved searches

[Mark Walters]

This series allows the user to customize the display of individual
saved searches, including showing thread counts, changed counts,
omitting counts (for speed), a read/unread count etc.

The series is not as large as it looks. Patch 1 is just code movement,
and patches 4-7 implement some of the above possibilities. Patches 4-7
could be applied but they are more intended to demonstrate and let
people test the infrastructure (ie showing what can be done).

Comments on the series: 

the display function is given a keyword list of arguments (so use
&rest args, followed by (plist-get args :keyword)). This is to make
the interface easily extendible without breaking backwards
compatibility.

custom display functions do not link into the batch count
functionality (but are omitted from it). Thus if the custom function
doesn't always do a query hello may be quicker, but in other cases,
particularly over a remote link, it may be slower.

some custom display functions will be slow, but they may be fast
enough for small searches (eg inbox).

Finally, a lisp query: I use a plist for saving the results of the
previous run (so display functions can highlight changes). This is
indexed by the saved searches themselves. I think this is OK (because
plist uses eq not equal) but would like a confirmation.

Finally, some of this infrastructure could be used to do async counts
at a later stage. I have some preliminary patches, but that is all
rather more fragile.

Best wishes

Mark



Mark Walters (7):
  emacs: hello: separate out the saved-search display function
  emacs: hello: allow saved search display functions
  emacs: hello: store previous saved-search results
  emacs: hello: add a threads-and-messages function
  emacs: hello: require cl
  emacs: hello: add highlight newly arrived messages option
  emacs: hello: add option to omit count for a search

 emacs/notmuch-hello.el |  139 ++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 111 insertions(+), 28 deletions(-)

-- 
1.7.10.4

[PATCH 1/7] emacs: hello: separate out the saved-search display function

[Mark Walters]

Separate the code that prints a saved-search and count into its own
function.
---
 emacs/notmuch-hello.el |   30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 3de5238..0a7004c 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -475,6 +475,22 @@ (defun notmuch-hello-filtered-query (query filter)
     (concat "(" query ") and (" filter ")"))
    (t query)))
 
+(defun notmuch-hello-batch-message-count (elem-plist options)
+  "Update the message count for a saved search.
+
+This takes the next line in the current buffer (the output from a
+call to notmuch count --batch) as the message count and sets
+the :count property in ELEM-PLIST to it."
+  (let* ((count (prog1 (read (current-buffer))
+		  (forward-line 1)))
+	 (search-query (plist-get elem-plist :query))
+	 (filtered-query (notmuch-hello-filtered-query
+			  search-query (plist-get options :filter))))
+    (when (and filtered-query (or (plist-get options :show-empty-searches)
+				  (> count 0)))
+      (setq elem-plist (plist-put elem-plist :query filtered-query))
+      (plist-put elem-plist :count (notmuch-hello-nice-number count)))))
+
 (defun notmuch-hello-query-counts (query-list &rest options)
   "Compute list of counts of matched messages from QUERY-LIST.
 
@@ -514,15 +530,8 @@ (defun notmuch-hello-query-counts (query-list &rest options)
      #'identity
      (mapcar
       (lambda (elem)
-	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
-	       (search-query (plist-get elem-plist :query))
-	       (filtered-query (notmuch-hello-filtered-query
-				search-query (plist-get options :filter)))
-	       (message-count (prog1 (read (current-buffer))
-				(forward-line 1))))
-	  (when (and filtered-query (or (plist-get options :show-empty-searches) (> message-count 0)))
-	    (setq elem-plist (plist-put elem-plist :query filtered-query))
-	    (plist-put elem-plist :count message-count))))
+	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)))
+	  (notmuch-hello-batch-message-count elem-plist options)))
       query-list))))
 
 (defun notmuch-hello-insert-buttons (searches)
@@ -558,8 +567,7 @@ (defun notmuch-hello-insert-buttons (searches)
 				     (oldest-first t)
 				     (otherwise notmuch-search-oldest-first)))
 		     (msg-count (plist-get elem :count)))
-		(widget-insert (format "%8s "
-				       (notmuch-hello-nice-number msg-count)))
+		(widget-insert (format "%8s " msg-count))
 		(widget-create 'push-button
 			       :notify #'notmuch-hello-widget-search
 			       :notmuch-search-terms query
-- 
1.7.10.4

[PATCH 2/7] emacs: hello: allow saved search display functions

[Mark Walters]

Extend the saved search plist to include a :display-function property
that can customise the display of the saved search. It can change the
count string displayed and the name string. Thus the user can
customise so that a particular search:

does not show a count (and is thus hello is faster), shows a thread
count, show a combined message/thread count, changes colour of the
search button based on new messages arriving etc.

The display function should use (&rest args) to take a keyword list of
arguments. The advantage of this is that it is easy to add extra
arguments in a backwards compatible way (existing user scripts will
still work).

If a user uses this it will not take advantage of the batch counting
currently done so will make things slower over remote links (except in
cases where no query is done!).

It also deletes the :count-query option from the customise for saved
searches as this method is much more general. The code still supports
the :count-query option though (just the defcustom does not).
---
 emacs/notmuch-hello.el |   47 +++++++++++++++++++++++++++++++----------------
 1 file changed, 31 insertions(+), 16 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 0a7004c..7075860 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -85,12 +85,13 @@ (define-widget 'notmuch-saved-search-plist 'list
 		(group :format "%v" :inline t (const :format "  Query: " :query) (string :format "%v")))
 	  (checklist :inline t
 		     :format "%v"
-		     (group :format "%v" :inline t (const :format "Count-Query: " :count-query) (string :format "%v"))
 		     (group :format "%v" :inline t (const :format "" :sort-order)
 			    (choice :tag " Sort Order"
 				    (const :tag "Default" nil)
 				    (const :tag "Oldest-first" oldest-first)
-				    (const :tag "Newest-first" newest-first))))))
+				    (const :tag "Newest-first" newest-first)))
+		     (group :format "%v" :inline t (const :format "Display-function: " :display-function) (function :format "%v")))))
+
 
 (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
 				    (:name "unread" :query "tag:unread"))
@@ -101,12 +102,19 @@ (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
 
   :name            Name of the search (required).
   :query           Search to run (required).
-  :count-query     Optional extra query to generate the count
-                   shown. If not present then the :query property
-                   is used.
   :sort-order      Specify the sort order to be used for the search.
                    Possible values are 'oldest-first 'newest-first or
                    nil. Nil means use the default sort order.
+  :display-function Optional function to generate the count and
+                   name to be displayed. The function takes a
+                   keyword list of arguments (it should use
+                   &rest). Keywords include :current for the
+                   current saved search plist and :options. If
+                   this function is not set then the default
+                   display of message count and name is used. The
+                   function should return an updated saved search
+                   plist including :name and :count as the name
+                   and count-string to be displayed.
 
 Other accepted forms are a cons cell of the form (NAME . QUERY)
 or a list of the form (NAME QUERY COUNT-QUERY)."
@@ -507,15 +515,16 @@ (defun notmuch-hello-query-counts (query-list &rest options)
 `notmuch-hello-insert-searches'."
   (with-temp-buffer
     (dolist (elem query-list nil)
-      (let ((count-query (or (notmuch-saved-search-get elem :count-query)
-			     (notmuch-saved-search-get elem :query))))
-	(insert
-	 (replace-regexp-in-string
-	  "\n" " "
-	  (notmuch-hello-filtered-query count-query
-					(or (plist-get options :filter-count)
-					    (plist-get options :filter))))
-	  "\n")))
+      (unless (notmuch-saved-search-get elem :display-function)
+	(let ((count-query (or (notmuch-saved-search-get elem :count-query)
+			       (notmuch-saved-search-get elem :query))))
+	  (insert
+	   (replace-regexp-in-string
+	    "\n" " "
+	    (notmuch-hello-filtered-query count-query
+					  (or (plist-get options :filter-count)
+					      (plist-get options :filter))))
+	   "\n"))))
 
     (unless (= (call-process-region (point-min) (point-max) notmuch-command
 				    t t nil "count" "--batch") 0)
@@ -530,8 +539,14 @@ (defun notmuch-hello-query-counts (query-list &rest options)
      #'identity
      (mapcar
       (lambda (elem)
-	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)))
-	  (notmuch-hello-batch-message-count elem-plist options)))
+	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
+	       (display-function (plist-get elem-plist :display-function))
+	       (result (if display-function
+			    (funcall display-function
+				     :current elem-plist
+				     :option options)
+			  (notmuch-hello-batch-message-count elem-plist options))))
+	  result))
       query-list))))
 
 (defun notmuch-hello-insert-buttons (searches)
-- 
1.7.10.4

[PATCH 3/7] emacs: hello: store previous saved-search results

[Mark Walters]

This adds a variable which keeps track of the previous saved search
results. It is stored as a plist with keys being the saved
searches. The result the saved search display function gave last time
is passed to the display-function as the the keyword :old argument.

This is useful for display functions that want to keep some state: eg
to show that the count has changed, new messages have arrived etc. The
function returns a saved search style plist so it can store any
information it likes inside this plist.
---
 emacs/notmuch-hello.el |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 7075860..e11006b 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -127,6 +127,10 @@ (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
   :tag "List of Saved Searches"
   :group 'notmuch-hello)
 
+(defvar notmuch-hello-saved-search-results nil
+  "Current saved search counts as a plist")
+(make-local-variable 'notmuch-hello-saved-search-results)
+
 (defcustom notmuch-hello-recent-searches-max 10
   "The number of recent searches to display."
   :type 'integer
@@ -541,11 +545,15 @@ (defun notmuch-hello-query-counts (query-list &rest options)
       (lambda (elem)
 	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
 	       (display-function (plist-get elem-plist :display-function))
+	       (old-result (plist-get notmuch-hello-saved-search-results elem))
 	       (result (if display-function
 			    (funcall display-function
 				     :current elem-plist
+				     :old old-result
 				     :option options)
 			  (notmuch-hello-batch-message-count elem-plist options))))
+	  (setq notmuch-hello-saved-search-results
+		(plist-put notmuch-hello-saved-search-results elem result))
 	  result))
       query-list))))
 
-- 
1.7.10.4

[PATCH 4/7] emacs: hello: add a threads-and-messages function

[Mark Walters]

This is an example of a possible display function. It displays the
count for a search as messages/threads. This will be slow on a large
search but could be useful for a small search.

It also extends and renames the existing (unused)
notmuch-saved-search-count function to count threads if wanted. The
rename is done because the function is not saved-search specific.
---
 emacs/notmuch-hello.el |   30 +++++++++++++++++++++++++-----
 1 file changed, 25 insertions(+), 5 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index e11006b..6b2dedc 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -90,8 +90,13 @@ (define-widget 'notmuch-saved-search-plist 'list
 				    (const :tag "Default" nil)
 				    (const :tag "Oldest-first" oldest-first)
 				    (const :tag "Newest-first" newest-first)))
-		     (group :format "%v" :inline t (const :format "Display-function: " :display-function) (function :format "%v")))))
-
+		     (group :format "%v" :inline t
+			    (const :format "" :display-function)
+			    (choice :tag  " Display function"
+				    (const :tag "Default (messages)" nil)
+				    (function-item :tag "messages/threads"
+						   notmuch-hello-display-count-threads-and-messages)
+				    function)))))
 
 (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
 				    (:name "unread" :query "tag:unread"))
@@ -435,9 +440,6 @@ (defun notmuch-hello-widget-search (widget &rest ignore)
 		  (widget-get widget
 			      :notmuch-search-oldest-first)))
 
-(defun notmuch-saved-search-count (search)
-  (car (process-lines notmuch-command "count" search)))
-
 (defun notmuch-hello-tags-per-line (widest)
   "Determine how many tags to show per line and how wide they
 should be. Returns a cons cell `(tags-per-line width)'."
@@ -487,6 +489,24 @@ (defun notmuch-hello-filtered-query (query filter)
     (concat "(" query ") and (" filter ")"))
    (t query)))
 
+(defun notmuch-hello-count (query &optional threads)
+  (let ((arg (if threads "--output=threads" "--output=messages")))
+    (string-to-number
+     (car (process-lines notmuch-command "count" arg query)))))
+
+(defun notmuch-hello-display-count-threads-and-messages (&rest args)
+  "Display the saved search count as messages/threads.
+
+This will be slow for large queries."
+  (let* ((current (plist-get args :current))
+	 (query (plist-get current :query))
+	 (messages (notmuch-hello-count query nil))
+	 (threads (notmuch-hello-count query t))
+	 (display (concat (notmuch-hello-nice-number messages)
+			  "/"
+			  (notmuch-hello-nice-number threads))))
+    (plist-put current :count display)))
+
 (defun notmuch-hello-batch-message-count (elem-plist options)
   "Update the message count for a saved search.
 
-- 
1.7.10.4

[PATCH 5/7] emacs: hello: require cl

[Mark Walters]

We will use cl at runtime in the next patch so require cl (rather than
just having it eval-when-compile). We require it in notmuch-lib anyway
so this is no loss in requiring it here too.
---
 emacs/notmuch-hello.el |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 6b2dedc..28e16be 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -19,7 +19,7 @@
 ;;
 ;; Authors: David Edmondson <dme@dme.org>
 
-(eval-when-compile (require 'cl))
+(require 'cl)
 (require 'widget)
 (require 'wid-edit) ; For `widget-forward'.
 
@@ -1017,3 +1017,7 @@ (defun notmuch-folder ()
 ;;
 
 (provide 'notmuch-hello)
+
+;; Local Variables:
+;; byte-compile-warnings: (not cl-functions)
+;; End:
-- 
1.7.10.4

[PATCH 6/7] emacs: hello: add highlight newly arrived messages option

[Mark Walters]

This adds a function that highlights searches which have newly arrived
messages (ones which arrived since the previous refresh of
notmuch-hello). It does that by getting a full list of matching
message ids and checking whether any new messages have appeared. Thus
it will be slow on large saved searches.
---
 emacs/notmuch-hello.el |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 28e16be..265b879 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -96,6 +96,8 @@ (define-widget 'notmuch-saved-search-plist 'list
 				    (const :tag "Default (messages)" nil)
 				    (function-item :tag "messages/threads"
 						   notmuch-hello-display-count-threads-and-messages)
+				    (function-item :tag "highlight newly arrived messages"
+						   notmuch-hello-display-new-messages)
 				    function)))))
 
 (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
@@ -507,6 +509,25 @@ (defun notmuch-hello-display-count-threads-and-messages (&rest args)
 			  (notmuch-hello-nice-number threads))))
     (plist-put current :count display)))
 
+(defun notmuch-hello-display-new-messages (&rest args)
+  "Highlight saved searches which have new messages.
+
+New messages are ones which have appeared since the last time
+notmuch hello was refreshed. This will be slow on large queries."
+  (let* ((current (plist-get args :current))
+	 (old (plist-get args :old))
+	 (query (plist-get current :query))
+	 (name (plist-get current :name))
+	 (new-list (notmuch-call-notmuch-sexp "search"
+					      "--output=messages"
+					      "--format=sexp"
+					      query))
+	 (old-list (plist-get old :message-list)))
+    (unless (subsetp new-list old-list :test 'equal)
+      (plist-put current :name (propertize name 'face '(:foreground "red"))))
+    (plist-put current :count (notmuch-hello-nice-number (length new-list)))
+    (plist-put current :message-list new-list)))
+
 (defun notmuch-hello-batch-message-count (elem-plist options)
   "Update the message count for a saved search.
 
-- 
1.7.10.4

[PATCH 7/7] emacs: hello: add option to omit count for a search

[Mark Walters]

This omits the count for the specific search. This makes notmuch-hello
faster, particularly for large complex saved searches.
---
 emacs/notmuch-hello.el |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 265b879..b3b539d 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -94,6 +94,8 @@ (define-widget 'notmuch-saved-search-plist 'list
 			    (const :format "" :display-function)
 			    (choice :tag  " Display function"
 				    (const :tag "Default (messages)" nil)
+				    (function-item :tag "omit count"
+						   notmuch-hello-display-omit-count)
 				    (function-item :tag "messages/threads"
 						   notmuch-hello-display-count-threads-and-messages)
 				    (function-item :tag "highlight newly arrived messages"
@@ -528,6 +530,11 @@ (defun notmuch-hello-display-new-messages (&rest args)
     (plist-put current :count (notmuch-hello-nice-number (length new-list)))
     (plist-put current :message-list new-list)))
 
+(defun notmuch-hello-display-omit-count (&rest args)
+  "Omit the count."
+  (let* ((current (plist-get args :current)))
+    (plist-put current :count "---")))
+
 (defun notmuch-hello-batch-message-count (elem-plist options)
   "Update the message count for a saved search.
 
-- 
1.7.10.4

Re: [PATCH 6/7] emacs: hello: add highlight newly arrived messages option

[David Edmondson]

[-- Attachment #1: Type: text/plain, Size: 2601 bytes --]

On Sun, May 11 2014, Mark Walters wrote:
> This adds a function that highlights searches which have newly arrived
> messages (ones which arrived since the previous refresh of
> notmuch-hello). It does that by getting a full list of matching
> message ids and checking whether any new messages have appeared. Thus
> it will be slow on large saved searches.
> ---
>  emacs/notmuch-hello.el |   21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>
> diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
> index 28e16be..265b879 100644
> --- a/emacs/notmuch-hello.el
> +++ b/emacs/notmuch-hello.el
> @@ -96,6 +96,8 @@ (define-widget 'notmuch-saved-search-plist 'list
>  				    (const :tag "Default (messages)" nil)
>  				    (function-item :tag "messages/threads"
>  						   notmuch-hello-display-count-threads-and-messages)
> +				    (function-item :tag "highlight newly arrived messages"
> +						   notmuch-hello-display-new-messages)
>  				    function)))))
>  
>  (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
> @@ -507,6 +509,25 @@ (defun notmuch-hello-display-count-threads-and-messages (&rest args)
>  			  (notmuch-hello-nice-number threads))))
>      (plist-put current :count display)))
>  
> +(defun notmuch-hello-display-new-messages (&rest args)
> +  "Highlight saved searches which have new messages.
> +
> +New messages are ones which have appeared since the last time
> +notmuch hello was refreshed. This will be slow on large queries."
> +  (let* ((current (plist-get args :current))
> +	 (old (plist-get args :old))
> +	 (query (plist-get current :query))
> +	 (name (plist-get current :name))
> +	 (new-list (notmuch-call-notmuch-sexp "search"
> +					      "--output=messages"
> +					      "--format=sexp"
> +					      query))
> +	 (old-list (plist-get old :message-list)))
> +    (unless (subsetp new-list old-list :test 'equal)
> +      (plist-put current :name (propertize name 'face '(:foreground "red"))))
> +    (plist-put current :count (notmuch-hello-nice-number (length new-list)))
> +    (plist-put current :message-list new-list)))

I wonder if it would be better to store a hash value derived from the
message list rather than the list itself. The list could be large.

> +
>  (defun notmuch-hello-batch-message-count (elem-plist options)
>    "Update the message count for a saved search.
>  
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 310 bytes --]

Re: [PATCH 2/7] emacs: hello: allow saved search display functions

[David Edmondson]

[-- Attachment #1: Type: text/plain, Size: 6069 bytes --]

On Sun, May 11 2014, Mark Walters wrote:
> Extend the saved search plist to include a :display-function property
> that can customise the display of the saved search. It can change the
> count string displayed and the name string. Thus the user can
> customise so that a particular search:
>
> does not show a count (and is thus hello is faster), shows a thread
> count, show a combined message/thread count, changes colour of the
> search button based on new messages arriving etc.
>
> The display function should use (&rest args) to take a keyword list of
> arguments. The advantage of this is that it is easy to add extra
> arguments in a backwards compatible way (existing user scripts will
> still work).
>
> If a user uses this it will not take advantage of the batch counting
> currently done so will make things slower over remote links (except in
> cases where no query is done!).
>
> It also deletes the :count-query option from the customise for saved
> searches as this method is much more general. The code still supports
> the :count-query option though (just the defcustom does not).
> ---
>  emacs/notmuch-hello.el |   47 +++++++++++++++++++++++++++++++----------------
>  1 file changed, 31 insertions(+), 16 deletions(-)
>
> diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
> index 0a7004c..7075860 100644
> --- a/emacs/notmuch-hello.el
> +++ b/emacs/notmuch-hello.el
> @@ -85,12 +85,13 @@ (define-widget 'notmuch-saved-search-plist 'list
>  		(group :format "%v" :inline t (const :format "  Query: " :query) (string :format "%v")))
>  	  (checklist :inline t
>  		     :format "%v"
> -		     (group :format "%v" :inline t (const :format "Count-Query: " :count-query) (string :format "%v"))
>  		     (group :format "%v" :inline t (const :format "" :sort-order)
>  			    (choice :tag " Sort Order"
>  				    (const :tag "Default" nil)
>  				    (const :tag "Oldest-first" oldest-first)
> -				    (const :tag "Newest-first" newest-first))))))
> +				    (const :tag "Newest-first" newest-first)))
> +		     (group :format "%v" :inline t (const :format "Display-function: " :display-function) (function :format "%v")))))
> +
>  
>  (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
>  				    (:name "unread" :query "tag:unread"))
> @@ -101,12 +102,19 @@ (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
>  
>    :name            Name of the search (required).
>    :query           Search to run (required).
> -  :count-query     Optional extra query to generate the count
> -                   shown. If not present then the :query property
> -                   is used.
>    :sort-order      Specify the sort order to be used for the search.
>                     Possible values are 'oldest-first 'newest-first or
>                     nil. Nil means use the default sort order.
> +  :display-function Optional function to generate the count and
> +                   name to be displayed. The function takes a
> +                   keyword list of arguments (it should use
> +                   &rest). Keywords include :current for the
> +                   current saved search plist and :options. If
> +                   this function is not set then the default
> +                   display of message count and name is used. The
> +                   function should return an updated saved search
> +                   plist including :name and :count as the name
> +                   and count-string to be displayed.
>  
>  Other accepted forms are a cons cell of the form (NAME . QUERY)
>  or a list of the form (NAME QUERY COUNT-QUERY)."
> @@ -507,15 +515,16 @@ (defun notmuch-hello-query-counts (query-list &rest options)
>  `notmuch-hello-insert-searches'."
>    (with-temp-buffer
>      (dolist (elem query-list nil)
> -      (let ((count-query (or (notmuch-saved-search-get elem :count-query)
> -			     (notmuch-saved-search-get elem :query))))
> -	(insert
> -	 (replace-regexp-in-string
> -	  "\n" " "
> -	  (notmuch-hello-filtered-query count-query
> -					(or (plist-get options :filter-count)
> -					    (plist-get options :filter))))
> -	  "\n")))
> +      (unless (notmuch-saved-search-get elem :display-function)
> +	(let ((count-query (or (notmuch-saved-search-get elem :count-query)
> +			       (notmuch-saved-search-get elem :query))))
> +	  (insert
> +	   (replace-regexp-in-string
> +	    "\n" " "
> +	    (notmuch-hello-filtered-query count-query
> +					  (or (plist-get options :filter-count)
> +					      (plist-get options :filter))))
> +	   "\n"))))
>  
>      (unless (= (call-process-region (point-min) (point-max) notmuch-command
>  				    t t nil "count" "--batch") 0)
> @@ -530,8 +539,14 @@ (defun notmuch-hello-query-counts (query-list &rest options)
>       #'identity
>       (mapcar
>        (lambda (elem)
> -	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)))
> -	  (notmuch-hello-batch-message-count elem-plist options)))
> +	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
> +	       (display-function (plist-get elem-plist :display-function))
> +	       (result (if display-function
> +			    (funcall display-function
> +				     :current elem-plist
> +				     :option options)
> +			  (notmuch-hello-batch-message-count elem-plist
>  			  options))))

If `notmuch-hello-batch-message-count' were re-written to operate as a
display-function, this could be replaced with something like:

	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
	       (result (funcall (or (plist-get elem-plist :display-function)
				    #'notmuch-hello-batch-message-count)
				:current elem-plist
				:option options))

> +	  result))
>        query-list))))
>  
>  (defun notmuch-hello-insert-buttons (searches)
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 310 bytes --]

Re: [PATCH 2/7] emacs: hello: allow saved search display functions

[Mark Walters]

On Mon, 12 May 2014, David Edmondson <dme@dme.org> wrote:
> On Sun, May 11 2014, Mark Walters wrote:
>> Extend the saved search plist to include a :display-function property
>> that can customise the display of the saved search. It can change the
>> count string displayed and the name string. Thus the user can
>> customise so that a particular search:
>>
>> does not show a count (and is thus hello is faster), shows a thread
>> count, show a combined message/thread count, changes colour of the
>> search button based on new messages arriving etc.
>>
>> The display function should use (&rest args) to take a keyword list of
>> arguments. The advantage of this is that it is easy to add extra
>> arguments in a backwards compatible way (existing user scripts will
>> still work).
>>
>> If a user uses this it will not take advantage of the batch counting
>> currently done so will make things slower over remote links (except in
>> cases where no query is done!).
>>
>> It also deletes the :count-query option from the customise for saved
>> searches as this method is much more general. The code still supports
>> the :count-query option though (just the defcustom does not).
>> ---
>>  emacs/notmuch-hello.el |   47 +++++++++++++++++++++++++++++++----------------
>>  1 file changed, 31 insertions(+), 16 deletions(-)
>>
>> diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
>> index 0a7004c..7075860 100644
>> --- a/emacs/notmuch-hello.el
>> +++ b/emacs/notmuch-hello.el
>> @@ -85,12 +85,13 @@ (define-widget 'notmuch-saved-search-plist 'list
>>  		(group :format "%v" :inline t (const :format "  Query: " :query) (string :format "%v")))
>>  	  (checklist :inline t
>>  		     :format "%v"
>> -		     (group :format "%v" :inline t (const :format "Count-Query: " :count-query) (string :format "%v"))
>>  		     (group :format "%v" :inline t (const :format "" :sort-order)
>>  			    (choice :tag " Sort Order"
>>  				    (const :tag "Default" nil)
>>  				    (const :tag "Oldest-first" oldest-first)
>> -				    (const :tag "Newest-first" newest-first))))))
>> +				    (const :tag "Newest-first" newest-first)))
>> +		     (group :format "%v" :inline t (const :format "Display-function: " :display-function) (function :format "%v")))))
>> +
>>  
>>  (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
>>  				    (:name "unread" :query "tag:unread"))
>> @@ -101,12 +102,19 @@ (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
>>  
>>    :name            Name of the search (required).
>>    :query           Search to run (required).
>> -  :count-query     Optional extra query to generate the count
>> -                   shown. If not present then the :query property
>> -                   is used.
>>    :sort-order      Specify the sort order to be used for the search.
>>                     Possible values are 'oldest-first 'newest-first or
>>                     nil. Nil means use the default sort order.
>> +  :display-function Optional function to generate the count and
>> +                   name to be displayed. The function takes a
>> +                   keyword list of arguments (it should use
>> +                   &rest). Keywords include :current for the
>> +                   current saved search plist and :options. If
>> +                   this function is not set then the default
>> +                   display of message count and name is used. The
>> +                   function should return an updated saved search
>> +                   plist including :name and :count as the name
>> +                   and count-string to be displayed.
>>  
>>  Other accepted forms are a cons cell of the form (NAME . QUERY)
>>  or a list of the form (NAME QUERY COUNT-QUERY)."
>> @@ -507,15 +515,16 @@ (defun notmuch-hello-query-counts (query-list &rest options)
>>  `notmuch-hello-insert-searches'."
>>    (with-temp-buffer
>>      (dolist (elem query-list nil)
>> -      (let ((count-query (or (notmuch-saved-search-get elem :count-query)
>> -			     (notmuch-saved-search-get elem :query))))
>> -	(insert
>> -	 (replace-regexp-in-string
>> -	  "\n" " "
>> -	  (notmuch-hello-filtered-query count-query
>> -					(or (plist-get options :filter-count)
>> -					    (plist-get options :filter))))
>> -	  "\n")))
>> +      (unless (notmuch-saved-search-get elem :display-function)
>> +	(let ((count-query (or (notmuch-saved-search-get elem :count-query)
>> +			       (notmuch-saved-search-get elem :query))))
>> +	  (insert
>> +	   (replace-regexp-in-string
>> +	    "\n" " "
>> +	    (notmuch-hello-filtered-query count-query
>> +					  (or (plist-get options :filter-count)
>> +					      (plist-get options :filter))))
>> +	   "\n"))))
>>  
>>      (unless (= (call-process-region (point-min) (point-max) notmuch-command
>>  				    t t nil "count" "--batch") 0)
>> @@ -530,8 +539,14 @@ (defun notmuch-hello-query-counts (query-list &rest options)
>>       #'identity
>>       (mapcar
>>        (lambda (elem)
>> -	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)))
>> -	  (notmuch-hello-batch-message-count elem-plist options)))
>> +	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
>> +	       (display-function (plist-get elem-plist :display-function))
>> +	       (result (if display-function
>> +			    (funcall display-function
>> +				     :current elem-plist
>> +				     :option options)
>> +			  (notmuch-hello-batch-message-count elem-plist
>>  			  options))))
>
> If `notmuch-hello-batch-message-count' were re-written to operate as a
> display-function, this could be replaced with something like:
>
> 	(let* ((elem-plist (notmuch-hello-saved-search-to-plist elem))
> 	       (result (funcall (or (plist-get elem-plist :display-function)
> 				    #'notmuch-hello-batch-message-count)
> 				:current elem-plist
> 				:option options))
>

The reason I didn't do this is that things would break if a user
manually configured display function to be
#'notmuch-hello-batch-message-count as the query would not have been
batched. Thus my version made the distinction fundamental. But perhaps
the batch queuing code could queue the query if either there is no
display function or the display-function is
notmuch-hello-batch-message-count, but that seems a bit gross. 

I also played with calling each display function in the batch phase too
and then they could queue any message counts they wanted. But I thought
that might be too complex. OTOH being able to do display
match/match-and-unread might be nice and could be batched with the
correct infrastructure.

Any thoughts?

Best wishes

Mark





>> +	  result))
>>        query-list))))
>>  
>>  (defun notmuch-hello-insert-buttons (searches)
>> -- 
>> 1.7.10.4
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 6/7] emacs: hello: add highlight newly arrived messages option

[Mark Walters]

On Mon, 12 May 2014, David Edmondson <dme@dme.org> wrote:
> On Sun, May 11 2014, Mark Walters wrote:
>> This adds a function that highlights searches which have newly arrived
>> messages (ones which arrived since the previous refresh of
>> notmuch-hello). It does that by getting a full list of matching
>> message ids and checking whether any new messages have appeared. Thus
>> it will be slow on large saved searches.
>> ---
>>  emacs/notmuch-hello.el |   21 +++++++++++++++++++++
>>  1 file changed, 21 insertions(+)
>>
>> diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
>> index 28e16be..265b879 100644
>> --- a/emacs/notmuch-hello.el
>> +++ b/emacs/notmuch-hello.el
>> @@ -96,6 +96,8 @@ (define-widget 'notmuch-saved-search-plist 'list
>>  				    (const :tag "Default (messages)" nil)
>>  				    (function-item :tag "messages/threads"
>>  						   notmuch-hello-display-count-threads-and-messages)
>> +				    (function-item :tag "highlight newly arrived messages"
>> +						   notmuch-hello-display-new-messages)
>>  				    function)))))
>>  
>>  (defcustom notmuch-saved-searches '((:name "inbox" :query "tag:inbox")
>> @@ -507,6 +509,25 @@ (defun notmuch-hello-display-count-threads-and-messages (&rest args)
>>  			  (notmuch-hello-nice-number threads))))
>>      (plist-put current :count display)))
>>  
>> +(defun notmuch-hello-display-new-messages (&rest args)
>> +  "Highlight saved searches which have new messages.
>> +
>> +New messages are ones which have appeared since the last time
>> +notmuch hello was refreshed. This will be slow on large queries."
>> +  (let* ((current (plist-get args :current))
>> +	 (old (plist-get args :old))
>> +	 (query (plist-get current :query))
>> +	 (name (plist-get current :name))
>> +	 (new-list (notmuch-call-notmuch-sexp "search"
>> +					      "--output=messages"
>> +					      "--format=sexp"
>> +					      query))
>> +	 (old-list (plist-get old :message-list)))
>> +    (unless (subsetp new-list old-list :test 'equal)
>> +      (plist-put current :name (propertize name 'face '(:foreground "red"))))
>> +    (plist-put current :count (notmuch-hello-nice-number (length new-list)))
>> +    (plist-put current :message-list new-list)))
>
> I wonder if it would be better to store a hash value derived from the
> message list rather than the list itself. The list could be large.

My view was that the list was OK as a user would be foolish to set this
for any large queries anyway (as the query itself would be too
slow). Also, a hash itself wouldn't get the effect I want as I do want
to be to test "subset" for the two lists (as I don't care about messages
no longer matching, only new messages that do match: think inbox type
things) 

I could store the hash of the message-id. That might save a substantial
chunk on storage (eg 40 byte message if to 4byte hash or something)

More generally, it is not clear to me whether we want to make it easy
for a user to cause notmuch-hello to be very slow. Perhaps we need a
timing for each display-function. If any one takes over 0.5s then we can
report it to the user!

Best wishes

Mark






>
>> +
>>  (defun notmuch-hello-batch-message-count (elem-plist options)
>>    "Update the message count for a saved search.
>>  
>> -- 
>> 1.7.10.4
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 6/7] emacs: hello: add highlight newly arrived messages option

[Tomi Ollila]

On Sun, May 11 2014, Mark Walters <markwalters1009@gmail.com> wrote:

> This adds a function that highlights searches which have newly arrived
> messages (ones which arrived since the previous refresh of
> notmuch-hello). It does that by getting a full list of matching
> message ids and checking whether any new messages have appeared. Thus
> it will be slow on large saved searches.

What we need is docid: support -- then this could be faaast !!111!!11??? ;D

-> ZZZ

Tomi

Re: [PATCH 1/7] emacs: hello: separate out the saved-search display function

[David Bremner]

Mark Walters <markwalters1009@gmail.com> writes:

> Separate the code that prints a saved-search and count into its own
> function.

Hi Mark;

After the first patch, this series doesn't apply any more because of
conflicts.

d

Re: [PATCH 5/7] emacs: hello: require cl

[David Bremner]

Mark Walters <markwalters1009@gmail.com> writes:
>  
> -(eval-when-compile (require 'cl))
> +(require 'cl)
>  (require 'widget)
>  (require 'wid-edit) ; For `widget-forward'.
>  
> @@ -1017,3 +1017,7 @@ (defun notmuch-folder ()
>  ;;
>  
>  (provide 'notmuch-hello)
> +
> +;; Local Variables:
> +;; byte-compile-warnings: (not cl-functions)
> +;; End:

I was wondering if we should make any attempt to use cl-lib instead of
cl.el, since the former is more supported in newer emacs (e.g. allowed
for use in core code, I think). On the other hand, maybe it just causes
problems with emacs 23 (which we haven't completely dropped yet)