[PATCH 0/4] Add refresh all buffers functionality

[PATCH 0/4] Add refresh all buffers functionality

[Ioan-Adrian Ratiu]

This patch series adds a function to refresh all buffers, including an
option to silently refresh them in the background i.e. to not show the
newly refreshed buffer in any window.

This is very useful for asynchronously updating all buffers when new
mail arrives, using logic similar to the following (it's what I use):

(setq process-connection-type nil)

(defun done-index-sentinel (process event)
  (notmuch-refresh-all-buffers t)
  (message "Mail sync complete"))

(defun done-sync-sentinel (process event)
  (message "Indexing mail using notmuch")
  (set-process-sentinel (start-process "notmuch" nil "notmuch" "new")
			'done-index-sentinel))

(defun run-mail-sync ()
  (message "Syncing mail in background")
  (set-process-sentinel (start-process "mbsync" nil "mbsync" "gmail")
			  'done-sync-sentinel))

(run-with-idle-timer 600 nil 'run-mail-sync)

Ioan-Adrian Ratiu (4):
  emacs: reuse buffer when refreshing searches
  emacs: adjust all types of notmuch show buffers
  emacs: add refresh buffer optional no-display arg
  emacs: add refresh all buffers function

 emacs/notmuch-lib.el  | 25 ++++++++++++++++++++++---
 emacs/notmuch-show.el | 11 ++++++++++-
 emacs/notmuch.el      | 24 ++++++++++++++++--------
 3 files changed, 48 insertions(+), 12 deletions(-)

-- 
2.9.3

[PATCH 1/4] emacs: reuse buffer when refreshing searches

[Ioan-Adrian Ratiu]

There's no reason to completely kill a buffer while refreshing its
search results because its buffer name is constant between refreshes
(based on the search query) and only its contents may change.

Reusing the same buffer also makes it possible to do things like
refreshing a buffer which is not focused or even not shown in any
window - this will be used in the next patches to add auto-refresh
capabilities to all existing notmuch buffers + a function to call
after syncing mail to refresh everything.

Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
---
 emacs/notmuch.el | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 43d56f7..c33c55c 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -990,7 +990,7 @@ the configured default sort order."
 (defun notmuch-search-refresh-view ()
   "Refresh the current view.
 
-Kills the current buffer and runs a new search with the same
+Erases the current buffer and runs a new search with the same
 query string as the current search. If the current thread is in
 the new search results, then point will be placed on the same
 thread. Otherwise, point will be moved to attempt to be in the
@@ -998,8 +998,9 @@ same relative position within the new buffer."
   (let ((target-line (line-number-at-pos))
 	(oldest-first notmuch-search-oldest-first)
 	(target-thread (notmuch-search-find-thread-id 'bare))
-	(query notmuch-search-query-string))
-    (notmuch-bury-or-kill-this-buffer)
+	(query notmuch-search-query-string)
+	(inhibit-read-only t))
+    (erase-buffer)
     (notmuch-search query oldest-first target-thread target-line)
     (goto-char (point-min))))
 
-- 
2.9.3

[PATCH 2/4] emacs: adjust all types of notmuch show buffers

[Ioan-Adrian Ratiu]

The current notmuch-show-message-adjust logic only adjusts the buffer
focused in the current window. Extend it to adjust any kind of buffer,
even buffers in a window without focus or in a different frame or even
not shown at all.

This new logic is very useful to build upon for the auto-refresh all
buffers feature because you can use similar code to refresh any buffer:

(with-current-buffer "*random show buffer*"
  (notmuch-refresh-this-buffer))

Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
---
 emacs/notmuch-show.el | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6d3149b..74818cc 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1505,7 +1505,16 @@ All currently available key bindings:
   (goto-char (notmuch-show-message-bottom)))
 
 (defun notmuch-show-message-adjust ()
-  (recenter 0))
+  (let ((buffer-window (get-buffer-window (current-buffer) t))
+	(msg-position (point)))
+    (if buffer-window
+	(with-selected-window buffer-window
+	  (goto-char msg-position)
+	  (recenter 0))
+      (save-window-excursion
+	(select-window (display-buffer (current-buffer)))
+	(goto-char msg-position)
+	(recenter 0)))))
 
 ;; Movement related functions.
 
-- 
2.9.3

[PATCH 3/4] emacs: add refresh buffer optional no-display arg

[Ioan-Adrian Ratiu]

Add an optional no-display arg to the generic buffer refresh function,
notmuch-refresh-this-buffer, which works the same way as notmuch-hello
mode's notmuch-hello-update no-display arg.

The idea is for the generic notmuch-refresh-this-buffer to pass down
this arg to the "mode-specific" refresh functions to be able to update
buffers without bringing them to the foreground (if they are already
foregrounded, i.e. displayed in a window, this has no effect).

When updating a search buffer, notmuch currently always brings results
in a window to the foreground. Perhaps counter-intuitively, we do not
want this behaviour necessarily, because we want to auto-refresh any
kind of search buffers, even those backgrounded (not displayed in any
window/frame) from previous searches. This is why we add a no-display
arg to notmuch-search.

We do this to show which mails have appeard or dissapeared since the
last search refresh and have this information updated in real time
even when switching buffers. The ultimate goal of this is to have all
notmuch buffers auto-refresh when the email client syncs (this function
is added in the next patch).

Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
---
 emacs/notmuch-lib.el | 10 +++++++---
 emacs/notmuch.el     | 17 ++++++++++++-----
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 2f015b0..6618365 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -409,14 +409,18 @@ of its command symbol."
   "Function to call to refresh the current buffer.")
 (make-variable-buffer-local 'notmuch-buffer-refresh-function)
 
-(defun notmuch-refresh-this-buffer ()
-  "Refresh the current buffer."
+(defun notmuch-refresh-this-buffer (&optional no-display)
+  "Refresh the current buffer.
+
+If no-display is non-nil do not try to bring the buffer to the
+foreground. If the buffer is already foregrounded i.e. displayed
+in a window on screen, no-display has no effect."
   (interactive)
   (when notmuch-buffer-refresh-function
     (if (commandp notmuch-buffer-refresh-function)
 	;; Pass prefix argument, etc.
 	(call-interactively notmuch-buffer-refresh-function)
-      (funcall notmuch-buffer-refresh-function))))
+      (funcall notmuch-buffer-refresh-function no-display))))
 
 (defun notmuch-poll-and-refresh-this-buffer ()
   "Invoke `notmuch-poll' to import mail, then refresh the current buffer."
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index c33c55c..ed93e66 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -930,7 +930,7 @@ PROMPT is the string to prompt with."
 
 ;;;###autoload
 (put 'notmuch-search 'notmuch-doc "Search for messages.")
-(defun notmuch-search (&optional query oldest-first target-thread target-line)
+(defun notmuch-search (&optional query oldest-first target-thread target-line no-display)
   "Display threads matching QUERY in a notmuch-search buffer.
 
 If QUERY is nil, it is read interactively from the minibuffer.
@@ -941,6 +941,9 @@ Other optional parameters are used as follows:
                  current if it appears in the search results.
   TARGET-LINE: The line number to move to if the target thread does not
                appear in the search results.
+  NO-DISPLAY: Do not try to foreground the search results buffer. If it is
+              already foregrounded i.e. displayed in a window, this has no
+              effect, meaning the buffer will remain visible.
 
 When called interactively, this will prompt for a query and use
 the configured default sort order."
@@ -954,7 +957,9 @@ the configured default sort order."
 
   (let* ((query (or query (notmuch-read-query "Notmuch search: ")))
 	 (buffer (get-buffer-create (notmuch-search-buffer-title query))))
-    (switch-to-buffer buffer)
+    (if no-display
+	(set-buffer buffer)
+      (switch-to-buffer buffer))
     (notmuch-search-mode)
     ;; Don't track undo information for this buffer
     (set 'buffer-undo-list t)
@@ -987,21 +992,23 @@ the configured default sort order."
 	  (set-process-query-on-exit-flag proc nil))))
     (run-hooks 'notmuch-search-hook)))
 
-(defun notmuch-search-refresh-view ()
+(defun notmuch-search-refresh-view (&optional no-display)
   "Refresh the current view.
 
 Erases the current buffer and runs a new search with the same
 query string as the current search. If the current thread is in
 the new search results, then point will be placed on the same
 thread. Otherwise, point will be moved to attempt to be in the
-same relative position within the new buffer."
+same relative position within the new buffer. If no-display is
+non-nil, the search results buffer will not be foregrounded, if
+it already is displayed in a window, then no-display has no effect."
   (let ((target-line (line-number-at-pos))
 	(oldest-first notmuch-search-oldest-first)
 	(target-thread (notmuch-search-find-thread-id 'bare))
 	(query notmuch-search-query-string)
 	(inhibit-read-only t))
     (erase-buffer)
-    (notmuch-search query oldest-first target-thread target-line)
+    (notmuch-search query oldest-first target-thread target-line no-display)
     (goto-char (point-min))))
 
 (defun notmuch-search-toggle-order ()
-- 
2.9.3

[PATCH 4/4] emacs: add refresh all buffers function

[Ioan-Adrian Ratiu]

This new notmuch-refresh-all-buffers function calls each buffer's major
mode specific refresh function using the generic function
notmuch-refresh-this-buffer.

It is very useful because by passing a non-nil arg to the buffer specific
refresh functions it refreshes all notmuch buffers in the background and
this again is very useful when doing periodic timer-based mail syncing.

Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
---
 emacs/notmuch-lib.el | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 6618365..4cc2041 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -428,6 +428,21 @@ in a window on screen, no-display has no effect."
   (notmuch-poll)
   (notmuch-refresh-this-buffer))
 
+(defun notmuch-refresh-all-buffers (&optional no-display)
+  "Invoke `notmuch-refresh-this-buffer' on all notmuch major-mode buffers.
+
+If no-display is non-nil all buffers are silently refreshed, i.e. they are
+not foregrounded even if not displayed in any window. If no-display is nil
+then each buffer's mode-specific refresh function uses its default behaviour."
+  (let ((buffers (buffer-list)))
+    (while buffers
+      (setq buffer (car buffers)
+	    buffers (cdr buffers)
+	    buffer-mode (buffer-local-value 'major-mode buffer))
+      (when (string-prefix-p "notmuch" (format "%s" buffer-mode))
+	(with-current-buffer buffer
+	  (notmuch-refresh-this-buffer no-display))))))
+
 (defun notmuch-prettify-subject (subject)
   ;; This function is used by `notmuch-search-process-filter' which
   ;; requires that we not disrupt its' matching state.
-- 
2.9.3

Re: [PATCH 1/4] emacs: reuse buffer when refreshing searches

[Mark Walters]

Hi

On Sat, 10 Sep 2016, Ioan-Adrian Ratiu <adi@adirat.com> wrote:
> There's no reason to completely kill a buffer while refreshing its
> search results because its buffer name is constant between refreshes
> (based on the search query) and only its contents may change.
>
> Reusing the same buffer also makes it possible to do things like
> refreshing a buffer which is not focused or even not shown in any
> window - this will be used in the next patches to add auto-refresh
> capabilities to all existing notmuch buffers + a function to call
> after syncing mail to refresh everything.
>
> Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
> ---
>  emacs/notmuch.el | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/emacs/notmuch.el b/emacs/notmuch.el
> index 43d56f7..c33c55c 100644
> --- a/emacs/notmuch.el
> +++ b/emacs/notmuch.el
> @@ -990,7 +990,7 @@ the configured default sort order."
>  (defun notmuch-search-refresh-view ()
>    "Refresh the current view.
>  
> -Kills the current buffer and runs a new search with the same
> +Erases the current buffer and runs a new search with the same
>  query string as the current search. If the current thread is in
>  the new search results, then point will be placed on the same
>  thread. Otherwise, point will be moved to attempt to be in the
> @@ -998,8 +998,9 @@ same relative position within the new buffer."
>    (let ((target-line (line-number-at-pos))
>  	(oldest-first notmuch-search-oldest-first)
>  	(target-thread (notmuch-search-find-thread-id 'bare))
> -	(query notmuch-search-query-string))
> -    (notmuch-bury-or-kill-this-buffer)
> +	(query notmuch-search-query-string)
> +	(inhibit-read-only t))
> +    (erase-buffer)
>      (notmuch-search query oldest-first target-thread target-line)

This looks fine.  It might be worth putting a comment in that
notmuch-search kills all local variables as otherwise a reader may worry
about that. The chain to get to kill-all-local-variables is relatively
long:

notmuch-search
notmuch-search-mode
define-derived-mode
fundamental-mode


Best wishes

Mark

Re: [PATCH 2/4] emacs: adjust all types of notmuch show buffers

[Mark Walters]

On Sat, 10 Sep 2016, Ioan-Adrian Ratiu <adi@adirat.com> wrote:
> The current notmuch-show-message-adjust logic only adjusts the buffer
> focused in the current window. Extend it to adjust any kind of buffer,
> even buffers in a window without focus or in a different frame or even
> not shown at all.
>
> This new logic is very useful to build upon for the auto-refresh all
> buffers feature because you can use similar code to refresh any buffer:
>
> (with-current-buffer "*random show buffer*"
>   (notmuch-refresh-this-buffer))
>
> Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
> ---
>  emacs/notmuch-show.el | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
> index 6d3149b..74818cc 100644
> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -1505,7 +1505,16 @@ All currently available key bindings:
>    (goto-char (notmuch-show-message-bottom)))
>  
>  (defun notmuch-show-message-adjust ()
> -  (recenter 0))
> +  (let ((buffer-window (get-buffer-window (current-buffer) t))
> +	(msg-position (point)))
> +    (if buffer-window
> +	(with-selected-window buffer-window
> +	  (goto-char msg-position)
> +	  (recenter 0))
> +      (save-window-excursion
> +	(select-window (display-buffer (current-buffer)))
> +	(goto-char msg-position)
> +	(recenter 0)))))

Hi

I haven't tested things yet, but what happens if the buffer is open in
multiple windows?

I think it would be worth adding something to the commit message along
the lines of

    notmuch-show-refresh-view calls notmuch-show-message-adjust in its
    call chain. Since we want to call notmuch-show-refresh-view on
    buffers than are not displayed we need to modify
    notmuch-show-message-adjust to work in this case.

Best wishes

Mark


>  ;; Movement related functions.
>  
> -- 
> 2.9.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] emacs: add refresh all buffers function

[Mark Walters]

On Sat, 10 Sep 2016, Ioan-Adrian Ratiu <adi@adirat.com> wrote:
> This new notmuch-refresh-all-buffers function calls each buffer's major
> mode specific refresh function using the generic function
> notmuch-refresh-this-buffer.
>
> It is very useful because by passing a non-nil arg to the buffer specific
> refresh functions it refreshes all notmuch buffers in the background and
> this again is very useful when doing periodic timer-based mail syncing.
>
> Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
> ---
>  emacs/notmuch-lib.el | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
> index 6618365..4cc2041 100644
> --- a/emacs/notmuch-lib.el
> +++ b/emacs/notmuch-lib.el
> @@ -428,6 +428,21 @@ in a window on screen, no-display has no effect."
>    (notmuch-poll)
>    (notmuch-refresh-this-buffer))
>  
> +(defun notmuch-refresh-all-buffers (&optional no-display)
> +  "Invoke `notmuch-refresh-this-buffer' on all notmuch major-mode buffers.
> +
> +If no-display is non-nil all buffers are silently refreshed, i.e. they are
> +not foregrounded even if not displayed in any window. If no-display is nil
> +then each buffer's mode-specific refresh function uses its default behaviour."
> +  (let ((buffers (buffer-list)))
> +    (while buffers
> +      (setq buffer (car buffers)
> +	    buffers (cdr buffers)
> +	    buffer-mode (buffer-local-value 'major-mode buffer))

I think this is a case where dolist might be more idiomatic (and maybe
you want buffer to be local to this function?)

> +      (when (string-prefix-p "notmuch" (format "%s" buffer-mode))
> +	(with-current-buffer buffer
> +	  (notmuch-refresh-this-buffer no-display))))))

Is there a problem with this being slow if there are lots of show
buffers? notmuch show is synchronous? I am not saying it is a problem,
just wondered if you had considered it.

Best wishes

Mark

> +
>  (defun notmuch-prettify-subject (subject)
>    ;; This function is used by `notmuch-search-process-filter' which
>    ;; requires that we not disrupt its' matching state.
> -- 
> 2.9.3
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/4] emacs: adjust all types of notmuch show buffers

[Ioan-Adrian Ratiu]

Hi and thank you for the feedback!

On Fri, 16 Sep 2016, Mark Walters <markwalters1009@gmail.com> wrote:
> On Sat, 10 Sep 2016, Ioan-Adrian Ratiu <adi@adirat.com> wrote:
>> The current notmuch-show-message-adjust logic only adjusts the buffer
>> focused in the current window. Extend it to adjust any kind of buffer,
>> even buffers in a window without focus or in a different frame or even
>> not shown at all.
>>
>> This new logic is very useful to build upon for the auto-refresh all
>> buffers feature because you can use similar code to refresh any buffer:
>>
>> (with-current-buffer "*random show buffer*"
>>   (notmuch-refresh-this-buffer))
>>
>> Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
>> ---
>>  emacs/notmuch-show.el | 11 ++++++++++-
>>  1 file changed, 10 insertions(+), 1 deletion(-)
>>
>> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
>> index 6d3149b..74818cc 100644
>> --- a/emacs/notmuch-show.el
>> +++ b/emacs/notmuch-show.el
>> @@ -1505,7 +1505,16 @@ All currently available key bindings:
>>    (goto-char (notmuch-show-message-bottom)))
>>  
>>  (defun notmuch-show-message-adjust ()
>> -  (recenter 0))
>> +  (let ((buffer-window (get-buffer-window (current-buffer) t))
>> +	(msg-position (point)))
>> +    (if buffer-window
>> +	(with-selected-window buffer-window
>> +	  (goto-char msg-position)
>> +	  (recenter 0))
>> +      (save-window-excursion
>> +	(select-window (display-buffer (current-buffer)))
>> +	(goto-char msg-position)
>> +	(recenter 0)))))
>
> Hi
>
> I haven't tested things yet, but what happens if the buffer is open in
> multiple windows?

Good catch.

If one of the windows has focus, its point gets reset to the message
containing point before the refresh call (the standard notmuch-show
current refresh behaviour). The windows which don't have focus get reset
to the first message.

We want to make all windows showing a buffer get adjusted, right?
I can add this to v2.

>
> I think it would be worth adding something to the commit message along
> the lines of
>
>     notmuch-show-refresh-view calls notmuch-show-message-adjust in its
>     call chain. Since we want to call notmuch-show-refresh-view on
>     buffers than are not displayed we need to modify
>     notmuch-show-message-adjust to work in this case.

Yes, this is much clearer, I will update in v2.

>
> Best wishes
>
> Mark
>
>
>>  ;; Movement related functions.
>>  
>> -- 
>> 2.9.3
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] emacs: add refresh all buffers function

[Ioan-Adrian Ratiu]

On Fri, 16 Sep 2016, Mark Walters <markwalters1009@gmail.com> wrote:
> On Sat, 10 Sep 2016, Ioan-Adrian Ratiu <adi@adirat.com> wrote:
>> This new notmuch-refresh-all-buffers function calls each buffer's major
>> mode specific refresh function using the generic function
>> notmuch-refresh-this-buffer.
>>
>> It is very useful because by passing a non-nil arg to the buffer specific
>> refresh functions it refreshes all notmuch buffers in the background and
>> this again is very useful when doing periodic timer-based mail syncing.
>>
>> Signed-off-by: Ioan-Adrian Ratiu <adi@adirat.com>
>> ---
>>  emacs/notmuch-lib.el | 15 +++++++++++++++
>>  1 file changed, 15 insertions(+)
>>
>> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
>> index 6618365..4cc2041 100644
>> --- a/emacs/notmuch-lib.el
>> +++ b/emacs/notmuch-lib.el
>> @@ -428,6 +428,21 @@ in a window on screen, no-display has no effect."
>>    (notmuch-poll)
>>    (notmuch-refresh-this-buffer))
>>  
>> +(defun notmuch-refresh-all-buffers (&optional no-display)
>> +  "Invoke `notmuch-refresh-this-buffer' on all notmuch major-mode buffers.
>> +
>> +If no-display is non-nil all buffers are silently refreshed, i.e. they are
>> +not foregrounded even if not displayed in any window. If no-display is nil
>> +then each buffer's mode-specific refresh function uses its default behaviour."
>> +  (let ((buffers (buffer-list)))
>> +    (while buffers
>> +      (setq buffer (car buffers)
>> +	    buffers (cdr buffers)
>> +	    buffer-mode (buffer-local-value 'major-mode buffer))
>
> I think this is a case where dolist might be more idiomatic (and maybe
> you want buffer to be local to this function?)

Yes, good idea, I'll rewrite in v2.

>
>> +      (when (string-prefix-p "notmuch" (format "%s" buffer-mode))
>> +	(with-current-buffer buffer
>> +	  (notmuch-refresh-this-buffer no-display))))))
>
> Is there a problem with this being slow if there are lots of show
> buffers? notmuch show is synchronous? I am not saying it is a problem,
> just wondered if you had considered it.

I hadn't encountered any problems in practice. I tested with a maximum
of 20 or so show buffers at a time and didn't notice any slowness, but I
wasn't paying close attention because updates were happening in the
background (only a few buffers were actually shown in windows).

If this is a problem, then we can try to make it faster, sure.

>
> Best wishes
>
> Mark
>
>> +
>>  (defun notmuch-prettify-subject (subject)
>>    ;; This function is used by `notmuch-search-process-filter' which
>>    ;; requires that we not disrupt its' matching state.
>> -- 
>> 2.9.3
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch