feature request: fetch missing keys in the background

feature request: fetch missing keys in the background

[Antoine Beaupré]

Hi!

Because I have a rather large keyring, fetching new keys or refreshing
old ones takes a long time. When I click on the red red button saying
"Unknown key ID [...] or unsupported algorithm", Emacs freezes for a
good 90 seconds. Considering I use that editor^Wvirtual machine for
nearly everything, it's quite annoying and sends me off browsing the web
in Firefox forever. I end up having a horde of shaven yaks and then
finally remember that I had that key update, by which time I had
forgotten what I was doing reading email in the first place, let alone
what I was doing *before* opening my inbox...

So. It would be great if Notmuch would run those key updates
asynchronously. I am not sure how that would work: as Bremner said on
IRC, it might make it difficult to update the button automatically. But
I don't mind that: I can refresh the page myself. He suggested running
things in the background when clicking with a prefix (C-u?) but I would
argue that freezing Emacs is just a no-no in general.

I couldn't find directly what function was called behind that button:
"C-h k" just says it's, obviously, `push-button'. Bremner says it might
be `notmuch-crypto-sigstatus-good-callback' which looks reasonable. I'm
only a junior elisp programmer, but it seems to me the
`notmuch-show-refresh-view' call there could be an asynchronous callback
to an async `make-process' call, as opposed to `call-process', which is
synchronous.

But I'd like others to chime in here: is this something that would be
accepted? Would the above work?

A.

-- 
If quantum mechanics hasn't profoundly shocked you, you haven't
understood it yet.
                       - Niels Bohr

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Monday, 2018-09-03 at 11:49:09 -04, Antoine Beaupré wrote:

> Hi!
>
> Because I have a rather large keyring, fetching new keys or refreshing
> old ones takes a long time. When I click on the red red button saying
> "Unknown key ID [...] or unsupported algorithm", Emacs freezes for a
> good 90 seconds. Considering I use that editor^Wvirtual machine for
> nearly everything, it's quite annoying and sends me off browsing the web
> in Firefox forever. I end up having a horde of shaven yaks and then
> finally remember that I had that key update, by which time I had
> forgotten what I was doing reading email in the first place, let alone
> what I was doing *before* opening my inbox...
>
> So. It would be great if Notmuch would run those key updates
> asynchronously. I am not sure how that would work: as Bremner said on
> IRC, it might make it difficult to update the button automatically. But
> I don't mind that: I can refresh the page myself. He suggested running
> things in the background when clicking with a prefix (C-u?) but I would
> argue that freezing Emacs is just a no-no in general.
>
> I couldn't find directly what function was called behind that button:
> "C-h k" just says it's, obviously, `push-button'. Bremner says it might
> be `notmuch-crypto-sigstatus-good-callback' which looks reasonable. I'm
> only a junior elisp programmer, but it seems to me the
> `notmuch-show-refresh-view' call there could be an asynchronous callback
> to an async `make-process' call, as opposed to `call-process', which is
> synchronous.
>
> But I'd like others to chime in here: is this something that would be
> accepted? Would the above work?

I'm puzzled about the currently expected behaviour based on your
question.

I have “notmuch-crypto-process-mime” set to t.

I *don't* have “keyserver-options auto-key-retrieve” set in my gpg.conf,
so I don't expect to get any automatic retrieval of keys (it was
annoyingly slow sometimes).

When I read a message with a signature using a key that I don't have, I
get the expected “Unknown key ID ...” button. If I click on that button
a new window appears showing me the details of that key (with a slight
delay).

The referenced key is *not* imported into my keyring, so updating the
view of the message does nothing to change the display of the button -
it still says “Unknown key ID ...”.

Enabling “auto-key-retrieve” changes this behaviour, as you would expect
(I see a delay opening the message while the key is retrieved, and then
the button shows it as good).

(Caveat: I'm using “remote-notmuch”, so emacs and notmuch/gpg are on
different machines, but I don't know that this changes any of the
above.)

dme.
-- 
She's as sweet as Tupelo honey, she's an angel of the first degree.

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Monday, 2018-09-03 at 17:46:00 +01, David Edmondson wrote:

> (Caveat: I'm using “remote-notmuch”, so emacs and notmuch/gpg are on
> different machines, but I don't know that this changes any of the
> above.)

Ah, of course it does - “gpg --recv-keys ...” runs on the same machine
as emacs, not on the server in this case. Ick.

dme.
-- 
But uh oh, I love her because, she moves in her own way.

Re: feature request: fetch missing keys in the background

[Antoine Beaupré]

On 2018-09-03 17:46:00, David Edmondson wrote:

[...]

> I'm puzzled about the currently expected behaviour based on your
> question.

[...]

> Enabling “auto-key-retrieve” changes this behaviour, as you would expect
> (I see a delay opening the message while the key is retrieved, and then
> the button shows it as good).

The current behavior is somewhat expected: I'm happy notmuch fetches the
keys on click. The problem is it takes forever (that's GPG's fault) and
that Emacs freezes while it happens (that's Notmuch's fault).

> (Caveat: I'm using “remote-notmuch”, so emacs and notmuch/gpg are on
> different machines, but I don't know that this changes any of the
> above.)

I'm not sure either.

A.
-- 
If I can't dance, I don't want to be part of your revolution.
                        - Emma Goldman

Re: feature request: fetch missing keys in the background

[David Edmondson]

[-- Attachment #1: Type: text/plain, Size: 124 bytes --]

How about this patch?

You'll need to set “notmuch-crypto-get-keys-asynchronously” to “t” to
see any benefit.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: async-keys.patch --]
[-- Type: text/x-diff, Size: 1982 bytes --]

diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el
index fc2b5301..97396ba0 100644
--- a/emacs/notmuch-crypto.el
+++ b/emacs/notmuch-crypto.el
@@ -43,6 +43,11 @@ mode."
   :package-version '(notmuch . "0.25")
   :group 'notmuch-crypto)
 
+(defcustom notmuch-crypto-get-keys-asynchronously nil
+  "Retrieve gpg keys asynchronously."
+  :type 'boolean
+  :group 'notmuch-crypto)
+
 (defface notmuch-crypto-part-header
   '((((class color)
       (background dark))
@@ -145,6 +150,36 @@ mode."
 	(call-process epg-gpg-program nil t t "--list-keys" fingerprint))
       (recenter -1))))
 
+(defun notmuch-crypto--async-key-sentinel (process event)
+  (let ((status (process-status process))
+	(exit-status (process-exit-status process)))
+    (when (memq status '(exit signal))
+      (message "Asynchronous GPG key retrieval %s."
+	       (if (= exit-status 0)
+		   "completed"
+		 "failed")))))
+
+(defun notmuch-crypto-sigstatus-error-callback (button)
+  (let* ((sigstatus (button-get button :notmuch-sigstatus))
+	 (keyid (concat "0x" (plist-get sigstatus :keyid)))
+	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")))
+    (if notmuch-crypto-get-keys-asynchronously
+	(progn
+	  (message "Getting the GPG key %s asynchronously..." keyid)
+	  (make-process :name "notmuch GPG key retrieval"
+		       :buffer buffer
+		       :command (list epg-gpg-program "--recv-keys" keyid)
+		       :sentinel #'notmuch-crypto--async-key-sentinel))
+      (let ((window (display-buffer buffer t nil)))
+	(with-selected-window window
+	  (with-current-buffer buffer
+	    (goto-char (point-max))
+	    (call-process epg-gpg-program nil t t "--recv-keys" keyid)
+	    (insert "\n")
+	    (call-process epg-gpg-program nil t t "--list-keys" keyid))
+	  (recenter -1))
+	(notmuch-show-refresh-view)))))
+
 (defun notmuch-crypto-sigstatus-error-callback (button)
   (let* ((sigstatus (button-get button :notmuch-sigstatus))
 	 (keyid (concat "0x" (plist-get sigstatus :keyid)))

[-- Attachment #3: Type: text/plain, Size: 59 bytes --]


dme.
-- 
For a long time I felt, without style and grace.

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Monday, 2018-09-03 at 18:21:32 +01, David Edmondson wrote:

> How about this patch?
>
> You'll need to set “notmuch-crypto-get-keys-asynchronously” to “t” to
> see any benefit.

Oops, and delete the old version of
“notmuch-crypto-sigstatus-error-callback” from just below the new one.

> diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el
> index fc2b5301..97396ba0 100644
> --- a/emacs/notmuch-crypto.el
> +++ b/emacs/notmuch-crypto.el
> @@ -43,6 +43,11 @@ mode."
>    :package-version '(notmuch . "0.25")
>    :group 'notmuch-crypto)
>  
> +(defcustom notmuch-crypto-get-keys-asynchronously nil
> +  "Retrieve gpg keys asynchronously."
> +  :type 'boolean
> +  :group 'notmuch-crypto)
> +
>  (defface notmuch-crypto-part-header
>    '((((class color)
>        (background dark))
> @@ -145,6 +150,36 @@ mode."
>  	(call-process epg-gpg-program nil t t "--list-keys" fingerprint))
>        (recenter -1))))
>  
> +(defun notmuch-crypto--async-key-sentinel (process event)
> +  (let ((status (process-status process))
> +	(exit-status (process-exit-status process)))
> +    (when (memq status '(exit signal))
> +      (message "Asynchronous GPG key retrieval %s."
> +	       (if (= exit-status 0)
> +		   "completed"
> +		 "failed")))))
> +
> +(defun notmuch-crypto-sigstatus-error-callback (button)
> +  (let* ((sigstatus (button-get button :notmuch-sigstatus))
> +	 (keyid (concat "0x" (plist-get sigstatus :keyid)))
> +	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")))
> +    (if notmuch-crypto-get-keys-asynchronously
> +	(progn
> +	  (message "Getting the GPG key %s asynchronously..." keyid)
> +	  (make-process :name "notmuch GPG key retrieval"
> +		       :buffer buffer
> +		       :command (list epg-gpg-program "--recv-keys" keyid)
> +		       :sentinel #'notmuch-crypto--async-key-sentinel))
> +      (let ((window (display-buffer buffer t nil)))
> +	(with-selected-window window
> +	  (with-current-buffer buffer
> +	    (goto-char (point-max))
> +	    (call-process epg-gpg-program nil t t "--recv-keys" keyid)
> +	    (insert "\n")
> +	    (call-process epg-gpg-program nil t t "--list-keys" keyid))
> +	  (recenter -1))
> +	(notmuch-show-refresh-view)))))
> +
>  (defun notmuch-crypto-sigstatus-error-callback (button)
>    (let* ((sigstatus (button-get button :notmuch-sigstatus))
>  	 (keyid (concat "0x" (plist-get sigstatus :keyid)))
>
> dme.
> -- 
> For a long time I felt, without style and grace.

dme.
-- 
I'm not living in the real world, no more, no more.

PATCH: fetch missing keys in the background

[Antoine Beaupré]

[-- Attachment #1: Type: text/plain, Size: 246 bytes --]

On 2018-09-03 18:21:32, David Edmondson wrote:
> How about this patch?
>
> You'll need to set “notmuch-crypto-get-keys-asynchronously” to “t” to
> see any benefit.

This is great!

As you said on IRC, the patch is more likely:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: notmuch-async-fetch.patch --]
[-- Type: text/x-diff, Size: 2696 bytes --]

diff --git i/emacs/notmuch-crypto.el w/emacs/notmuch-crypto.el
index fc2b5301..175dc5f2 100644
--- i/emacs/notmuch-crypto.el
+++ w/emacs/notmuch-crypto.el
@@ -43,6 +43,18 @@ mode."
   :package-version '(notmuch . "0.25")
   :group 'notmuch-crypto)
 
+(defcustom notmuch-crypto-get-keys-asynchronously nil
+  "Retrieve gpg keys asynchronously.
+
+If this variable is non-nil, hitting the crypto button will
+trigger network operations in the background. This will also have
+the effect of disabling the automatic refresh on completion. Keep
+this set to nil to make sure the button display is up to date,
+but this will freeze everything until the crypto operation is
+completed, which can take a long time for larger keyrings."
+  :type 'boolean
+  :group 'notmuch-crypto)
+
 (defface notmuch-crypto-part-header
   '((((class color)
       (background dark))
@@ -145,19 +157,35 @@ mode."
 	(call-process epg-gpg-program nil t t "--list-keys" fingerprint))
       (recenter -1))))
 
+(defun notmuch-crypto--async-key-sentinel (process event)
+  (let ((status (process-status process))
+	(exit-status (process-exit-status process)))
+    (when (memq status '(exit signal))
+      (message "Asynchronous GPG key retrieval %s."
+	       (if (= exit-status 0)
+		   "completed"
+		 "failed")))))
+
 (defun notmuch-crypto-sigstatus-error-callback (button)
   (let* ((sigstatus (button-get button :notmuch-sigstatus))
 	 (keyid (concat "0x" (plist-get sigstatus :keyid)))
-	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*"))
-	 (window (display-buffer buffer t nil)))
-    (with-selected-window window
-      (with-current-buffer buffer
-	(goto-char (point-max))
-	(call-process epg-gpg-program nil t t "--recv-keys" keyid)
-	(insert "\n")
-	(call-process epg-gpg-program nil t t "--list-keys" keyid))
-      (recenter -1))
-    (notmuch-show-refresh-view)))
+	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")))
+    (if notmuch-crypto-get-keys-asynchronously
+	(progn
+	  (message "Getting the GPG key %s asynchronously..." keyid)
+	  (make-process :name "notmuch GPG key retrieval"
+		       :buffer buffer
+		       :command (list epg-gpg-program "--recv-keys" keyid)
+		       :sentinel #'notmuch-crypto--async-key-sentinel))
+      (let ((window (display-buffer buffer t nil)))
+	(with-selected-window window
+	  (with-current-buffer buffer
+	    (goto-char (point-max))
+	    (call-process epg-gpg-program nil t t "--recv-keys" keyid)
+	    (insert "\n")
+	    (call-process epg-gpg-program nil t t "--list-keys" keyid))
+	  (recenter -1))
+	(notmuch-show-refresh-view)))))
 
 (defun notmuch-crypto-insert-encstatus-button (encstatus)
   (let* ((status (plist-get encstatus :status))

[-- Attachment #3: Type: text/plain, Size: 904 bytes --]


One thing that's missing is to refresh the view automatically. I
understand this might be difficult to implement however. For example, I
naively tried to add this at the end of the sentinel:

      (when (= exit-status 0)
	(notmuch-show-refresh-view))

Then switch to another buffer (back to the search view, FWIW). The
sentinel then fails with:

error in process sentinel: notmuch-escape-boolean-term: Wrong type argument: stringp, nil
error in process sentinel: Wrong type argument: stringp, nil

Not sure why exactly. But I would certainly take the async update over
automatic refresh any time. It would also be useful to indicate that
will be the effect of the variable in the customize help as well. I gave
that a try in the above patch.

Thanks for the ultra-fast response!

A.

-- 
The United States is a nation of laws:
badly written and randomly enforced.
                        - Frank Zappa

Re: PATCH: fetch missing keys in the background

[David Edmondson]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

On Monday, 2018-09-03 at 13:47:09 -0400, Antoine Beaupré wrote:

> On 2018-09-03 18:21:32, David Edmondson wrote:
>> How about this patch?
>>
>> You'll need to set “notmuch-crypto-get-keys-asynchronously” to “t” to
>> see any benefit.
>
> This is great!
>
> As you said on IRC, the patch is more likely:

Another version, which attempts to redisplay the buffer if it thinks
that it's safe and sensible to do so.

This version *defaults* to async behaviour, as I think that it's safe to
do so.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: async-keys.patch --]
[-- Type: text/x-diff, Size: 3259 bytes --]

diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el
index fc2b5301..6e003f51 100644
--- a/emacs/notmuch-crypto.el
+++ b/emacs/notmuch-crypto.el
@@ -43,6 +43,11 @@ mode."
   :package-version '(notmuch . "0.25")
   :group 'notmuch-crypto)
 
+(defcustom notmuch-crypto-get-keys-asynchronously t
+  "Retrieve gpg keys asynchronously."
+  :type 'boolean
+  :group 'notmuch-crypto)
+
 (defface notmuch-crypto-part-header
   '((((class color)
       (background dark))
@@ -145,19 +150,56 @@ mode."
 	(call-process epg-gpg-program nil t t "--list-keys" fingerprint))
       (recenter -1))))
 
+(defun notmuch-crypto--async-key-sentinel (process event)
+  (let ((status (process-status process))
+	(exit-status (process-exit-status process))
+	(keyid (process-get process :gpg-key-id)))
+    (when (memq status '(exit signal))
+      (message "Getting the GPG key %s asynchronously...%s."
+	       keyid
+	       (if (= exit-status 0)
+		   "completed"
+		 "failed"))
+      ;; If retrieving the key was successful, the original buffer is
+      ;; still alive and point didn't move (i.e. the user didn't move
+      ;; on or away), refresh the buffer.
+      (when (= exit-status 0)
+	(let ((show-buffer (process-get process :notmuch-show-buffer))
+	      (show-point (process-get process :notmuch-show-point)))
+	  (when (and (bufferp show-buffer)
+		     (buffer-live-p show-buffer)
+		     (with-current-buffer show-buffer
+		       (= show-point (point))))
+	    (with-current-buffer show-buffer
+	      (notmuch-show-refresh-view))))))))
+
 (defun notmuch-crypto-sigstatus-error-callback (button)
   (let* ((sigstatus (button-get button :notmuch-sigstatus))
 	 (keyid (concat "0x" (plist-get sigstatus :keyid)))
-	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*"))
-	 (window (display-buffer buffer t nil)))
-    (with-selected-window window
-      (with-current-buffer buffer
-	(goto-char (point-max))
-	(call-process epg-gpg-program nil t t "--recv-keys" keyid)
-	(insert "\n")
-	(call-process epg-gpg-program nil t t "--list-keys" keyid))
-      (recenter -1))
-    (notmuch-show-refresh-view)))
+	 (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")))
+    (if notmuch-crypto-get-keys-asynchronously
+	(let ((p (make-process :name "notmuch GPG key retrieval"
+			       :buffer buffer
+			       :command (list epg-gpg-program "--recv-keys" keyid)
+			       :sentinel #'notmuch-crypto--async-key-sentinel
+			       ;; Create the process stopped so that
+			       ;; we have time to store the key id on
+			       ;; it.
+			       :stop t)))
+	  (process-put p :gpg-key-id keyid)
+	  (process-put p :notmuch-show-buffer (current-buffer))
+	  (process-put p :notmuch-show-point (point))
+	  (message "Getting the GPG key %s asynchronously..." keyid)
+	  (continue-process p))
+      (let ((window (display-buffer buffer t nil)))
+	(with-selected-window window
+	  (with-current-buffer buffer
+	    (goto-char (point-max))
+	    (call-process epg-gpg-program nil t t "--recv-keys" keyid)
+	    (insert "\n")
+	    (call-process epg-gpg-program nil t t "--list-keys" keyid))
+	  (recenter -1))
+	(notmuch-show-refresh-view)))))
 
 (defun notmuch-crypto-insert-encstatus-button (encstatus)
   (let* ((status (plist-get encstatus :status))

[-- Attachment #3: Type: text/plain, Size: 45 bytes --]


dme.
-- 
Please don't stand so close to me.

Re: feature request: fetch missing keys in the background

[Antoine Beaupré]

That's great! It's basically what I was looking for, but unfortunately
it does not work here. I am not sure why, but it never actually updates
the widget, even if I do not move the point.

A.
-- 
Sous le projecteur, on ne voit pas les autres.
                        - Félix Leclerc

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Monday, 2018-09-03 at 15:49:09 -04, Antoine Beaupré wrote:

> That's great! It's basically what I was looking for, but unfortunately
> it does not work here. I am not sure why, but it never actually updates
> the widget, even if I do not move the point.

That's odd. Forcing the connection type to 'pipe improves things on
macOS, but I figured that you were using Linux.

Could you test the patch that I posted and see if it works better?

dme.
-- 
I had my eyes closed in the dark.

Re: feature request: fetch missing keys in the background

[Antoine Beaupré]

On 2018-09-04 14:32:16, David Edmondson wrote:
> On Monday, 2018-09-03 at 15:49:09 -04, Antoine Beaupré wrote:
>
>> That's great! It's basically what I was looking for, but unfortunately
>> it does not work here. I am not sure why, but it never actually updates
>> the widget, even if I do not move the point.
>
> That's odd. Forcing the connection type to 'pipe improves things on
> macOS, but I figured that you were using Linux.
>
> Could you test the patch that I posted and see if it works better?

Err... That *was* with the new patch in cunwos2xx33.fsf@disaster-area.hh.sledj.net

Did i miss something?

-- 
Lorsque l'on range des objets dans des tiroirs, et que l'on a plus
d'objets que de tiroirs, alors un tiroir au moins contient deux
objets.
                        - Lejeune-Dirichlet, Peter Gustav

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Tuesday, 2018-09-04 at 09:34:02 -04, Antoine Beaupré wrote:

> On 2018-09-04 14:32:16, David Edmondson wrote:
>> On Monday, 2018-09-03 at 15:49:09 -04, Antoine Beaupré wrote:
>>
>>> That's great! It's basically what I was looking for, but unfortunately
>>> it does not work here. I am not sure why, but it never actually updates
>>> the widget, even if I do not move the point.
>>
>> That's odd. Forcing the connection type to 'pipe improves things on
>> macOS, but I figured that you were using Linux.
>>
>> Could you test the patch that I posted and see if it works better?
>
> Err... That *was* with the new patch in cunwos2xx33.fsf@disaster-area.hh.sledj.net
>
> Did i miss something?

I just sent another set.

dme.
-- 
I don't care 'bout your other girls, just be good to me.

Re: feature request: fetch missing keys in the background

[Antoine Beaupré]

On 2018-09-04 14:39:57, David Edmondson wrote:
> On Tuesday, 2018-09-04 at 09:34:02 -04, Antoine Beaupré wrote:
>
>> On 2018-09-04 14:32:16, David Edmondson wrote:
>>> On Monday, 2018-09-03 at 15:49:09 -04, Antoine Beaupré wrote:
>>>
>>>> That's great! It's basically what I was looking for, but unfortunately
>>>> it does not work here. I am not sure why, but it never actually updates
>>>> the widget, even if I do not move the point.
>>>
>>> That's odd. Forcing the connection type to 'pipe improves things on
>>> macOS, but I figured that you were using Linux.
>>>
>>> Could you test the patch that I posted and see if it works better?
>>
>> Err... That *was* with the new patch in cunwos2xx33.fsf@disaster-area.hh.sledj.net
>>
>> Did i miss something?
>
> I just sent another set.

Ah yes, in another thread.. I'm not on the list so I missed that,
sorry. :)

So I tried that and it works, if I really don't touch anything. It's
certainly an improvement over the previous behavior. Once the "message"
is gone (ie. as soon as anything else is done in Emacs), it's a little
hard to figure out what is happening without going through the status
buffer.

Maybe that's fine, but it would be nice if the button would immediately
update with that status. For example, the text could change to
"... updating" or something. Of course, we're back to the same place
that it can't be updated without shuffling the buffer around if the
point moved, but at least the UI would "remember" there was some change
in the status (it's not just "Failure" but "We're working on it").

I also quite like the changes to the *..-gpg-out* buffer: it's a little
more readable than it was before.

Anyways, as is, I think it's a great improvement already and would
recommend this for merging.

A.

PS: As it turns out, I had a lot of trouble importing those patches. I
tried first to download the 2018 archive from here:

https://notmuchmail.org/pipermail/notmuch/2018.txt.gz

I tried converting this to a maildir and move it to ~/Maildir/.notmuch/
with spwhitton's mailscripts:

    mbox2maildir notmuch-2018.txt notmuch-2018
    mdmv notmuch-2018/*/* ~/Maildir/.notmuch/
    notmuch new

But the last command did not find any new messages, which left me quite
puzzled. The same thing occured with the full archive at:

https://notmuchmail.org/archives/notmuch.mbox

So I am not sure what I was doing wrong there... I ended up using mutt
(bleeergh! ;)) to extract the patchset...

-- 
We will create a civilization of the Mind in Cyberspace. May it be
more humane and fair than the world your governments have made
before.
                         - John Perry Barlow

Re: feature request: fetch missing keys in the background

[David Bremner]

Antoine Beaupré <anarcat@orangeseeds.org> writes:

>
> https://notmuchmail.org/pipermail/notmuch/2018.txt.gz
>

The short answer is that pipermail does not produce correct mboxes. I'm
not sure all that is wrong with them, but I noticed it mangles the
'From ' and the 'From: ' lines. It is probably some "anti-spam" measure
that was effective in the early 1980s.

The other mbox (full archive) should be correct.

Re: feature request: fetch missing keys in the background

[David Edmondson]

On Tuesday, 2018-09-04 at 10:12:00 -04, Antoine Beaupré wrote:

> So I tried that and it works, if I really don't touch anything. It's
> certainly an improvement over the previous behavior. Once the "message"
> is gone (ie. as soon as anything else is done in Emacs), it's a little
> hard to figure out what is happening without going through the status
> buffer.
>
> Maybe that's fine, but it would be nice if the button would immediately
> update with that status. For example, the text could change to
> "... updating" or something. Of course, we're back to the same place
> that it can't be updated without shuffling the buffer around if the
> point moved, but at least the UI would "remember" there was some change
> in the status (it's not just "Failure" but "We're working on it").

v2 of the patch does this.

dme.
-- 
When I grow up I'll be stable.