[PATCH 0/2] emacs allow save and load of the address-completions

[PATCH 0/2] emacs allow save and load of the address-completions

[Mark Walters]

This series adds a load/save functionality for the address
completions. This avoids the slow and poor completion provided until
the full address-hash has been generated -- this can take some time,
(over a minute on my machine), and is annoying if you frequently
restart emacs (e.g. when debugging things).

Although the patch is quite large it is relatively simple: essentially
a "print" to save, and a "read" to load. I have tried quite hard to
stop over-writing existing files, and to make sure the user does not
get surprising results if they change settings or the address
completion is upgraded in some way (e.g., we store the frequency of
each match in the map).

By default the save/load is switched off: this avoids accidental
privacy leaks, but does make it harder for users to discover.

Best wishes

Mark



Mark Walters (2):
  emacs: address: move address-full-harvest-finished to a function
  emacs: address: save hash

 emacs/notmuch-address.el | 85 ++++++++++++++++++++++++++++++++++++++++++++++--
 emacs/notmuch-company.el |  4 +--
 2 files changed, 84 insertions(+), 5 deletions(-)

-- 
2.1.4

[PATCH 1/2] emacs: address: move address-full-harvest-finished to a function

[Mark Walters]

This makes the code access notmuch-address-full-harvest-finished via a
helper function, notmuch-address--harvest-ready. Later we will use
this to check whether we can load the harvest instead of regenerating
it.
---
 emacs/notmuch-address.el | 8 ++++++--
 emacs/notmuch-company.el | 4 ++--
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el
index b3c56cf..2eaca79 100644
--- a/emacs/notmuch-address.el
+++ b/emacs/notmuch-address.el
@@ -37,7 +37,11 @@
 
 (defvar notmuch-address-full-harvest-finished nil
   "t indicates that full completion address harvesting has been
-finished")
+finished. Use notmuch-address--harvest-ready to access.")
+
+(defun notmuch-address--harvest-ready ()
+  "Return t if there is a full address hash available."
+  notmuch-address-full-harvest-finished)
 
 (defcustom notmuch-address-command 'internal
   "Determines how address completion candidates are generated.
@@ -170,7 +174,7 @@ elisp-based implementation or older implementation requiring
 external commands."
   (cond
    ((eq notmuch-address-command 'internal)
-    (when (not notmuch-address-full-harvest-finished)
+    (unless (notmuch-address--harvest-ready)
       ;; First, run quick synchronous harvest based on what the user
       ;; entered so far
       (notmuch-address-harvest original t))
diff --git a/emacs/notmuch-company.el b/emacs/notmuch-company.el
index ebe2c08..dca6471 100644
--- a/emacs/notmuch-company.el
+++ b/emacs/notmuch-company.el
@@ -41,7 +41,7 @@
 (declare-function notmuch-address-harvest "notmuch-address")
 (declare-function notmuch-address-harvest-trigger "notmuch-address")
 (declare-function notmuch-address-matching "notmuch-address")
-(defvar notmuch-address-full-harvest-finished)
+(declare-function notmuch-address--harvest-ready "notmuch-address")
 (defvar notmuch-address-completion-headers-regexp)
 
 ;;;###autoload
@@ -70,7 +70,7 @@
 				 (line-beginning-position))
 		   (setq notmuch-company-last-prefix (company-grab "[:,][ \t]*\\(.*\\)" 1 (point-at-bol)))))
       (candidates (cond
-		   (notmuch-address-full-harvest-finished
+		   ((notmuch-address--harvest-ready)
 		    ;; Update harvested addressed from time to time
 		    (notmuch-address-harvest-trigger)
 		    (notmuch-address-matching arg))
-- 
2.1.4

[PATCH 2/2] emacs: address: save hash

[Mark Walters]

This allows the user to save the address hash so that it is much
faster for the first completion after a restart. This defaults to off
as there are privacy implications to saving this information.

The code tries hard to avoid overwriting the wrong file. It also notes
if changes have been made to any of the relevant user settings, so
that the user does not get surprising results (i.e., outdated options
being used). Finally it stores some version information so that is
easy for us to update the format of the save file.
---
 emacs/notmuch-address.el | 83 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 79 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el
index 2eaca79..15c709d 100644
--- a/emacs/notmuch-address.el
+++ b/emacs/notmuch-address.el
@@ -37,11 +37,15 @@
 
 (defvar notmuch-address-full-harvest-finished nil
   "t indicates that full completion address harvesting has been
-finished. Use notmuch-address--harvest-ready to access.")
+finished. Use notmuch-address--harvest-ready to access as that
+will load a saved hash if necessary (and available).")
 
 (defun notmuch-address--harvest-ready ()
-  "Return t if there is a full address hash available."
-  notmuch-address-full-harvest-finished)
+  "Return t if there is a full address hash available.
+
+If the hash is not present it attempts to load a saved hash."
+  (or notmuch-address-full-harvest-finished
+      (notmuch-address--load-address-hash)))
 
 (defcustom notmuch-address-command 'internal
   "Determines how address completion candidates are generated.
@@ -91,6 +95,17 @@ This should be a list of the form '(DIRECTION FILTER), where
   :group 'notmuch-send
   :group 'notmuch-external)
 
+(defcustom notmuch-address-save-filename nil
+  "Filename to save the cached completion addresses.
+
+All the addresses notmuch uses for address completion will be
+cached in this file. This has obvious privacy implications so you
+should make sure it is not somewhere publicly readable."
+  :type '(choice (const :tag "Off" nil)
+		 (file :tag "Filename"))
+  :group 'notmuch-send
+  :group 'notmuch-external)
+
 (defcustom notmuch-address-selection-function 'notmuch-address-selection-function
   "The function to select address from given list. The function is
 called with PROMPT, COLLECTION, and INITIAL-INPUT as arguments
@@ -327,6 +342,64 @@ execution, CALLBACK is called when harvesting finishes."
   ;; return value
   nil)
 
+(defvar notmuch-address--save-hash-version 1
+  "Version format of the save hash.")
+
+(defun notmuch-address--get-address-hash ()
+  "Returns the saved address hash as a plist.
+
+Returns nil if the save file does not exist, or it does not seem
+to be a saved address hash."
+  (when notmuch-address-save-filename
+    (condition-case nil
+	(with-temp-buffer
+	  (insert-file-contents notmuch-address-save-filename)
+	  (let ((name (read (current-buffer)))
+		(plist (read (current-buffer))))
+	    ;; We do two simple sanity checks on the loaded file. We just
+	    ;; check a version is specified, not that it is the current
+	    ;; version, as we are allowed to over-write and a save-file with
+	    ;; an older version.
+	    (when (and (string= name "notmuch-address-hash")
+		       (plist-get plist :version))
+	      plist)))
+      ;; The error case catches any of the reads failing.
+      (error nil))))
+
+(defun notmuch-address--load-address-hash ()
+  "Read the saved address hash and set the corresponding variables."
+  (let ((load-plist (notmuch-address--get-address-hash)))
+    (when (and load-plist
+	       ;; If the user's setting have changed, or the version
+	       ;; has changed, return nil to make sure the new settings
+	       ;; take effect.
+	       (equal (plist-get load-plist :completion-settings)
+		      notmuch-address-internal-completion)
+	       (equal (plist-get load-plist :version)
+		      notmuch-address--save-hash-version))
+      (setq notmuch-address-last-harvest (plist-get load-plist :last-harvest)
+	    notmuch-address-completions (plist-get load-plist :completions)
+	    notmuch-address-full-harvest-finished t)
+      ;; Return t to say load was successful.
+      t)))
+
+(defun notmuch-address--save-address-hash ()
+  (when notmuch-address-save-filename
+    (if (or (not (file-exists-p notmuch-address-save-filename))
+	      ;; The file exists, check it is a file we saved
+	    (notmuch-address--get-address-hash))
+	(with-temp-file notmuch-address-save-filename
+	  (let ((save-plist (list :version notmuch-address--save-hash-version
+				  :completion-settings notmuch-address-internal-completion
+				  :last-harvest notmuch-address-last-harvest
+				  :completions notmuch-address-completions)))
+	    (print "notmuch-address-hash" (current-buffer))
+	    (print save-plist (current-buffer))))
+      (message "\
+Warning: notmuch-address-save-filename %s exists but doesn't
+appear to be an address savefile.  Not overwriting."
+	       notmuch-address-save-filename))))
+
 (defun notmuch-address-harvest-trigger ()
   (let ((now (float-time)))
     (when (> (- now notmuch-address-last-harvest) 86400)
@@ -337,7 +410,9 @@ execution, CALLBACK is called when harvesting finishes."
 				 ;; again when the trigger is next
 				 ;; called
 				 (if (string= event "finished\n")
-				     (setq notmuch-address-full-harvest-finished t)
+				     (progn
+				       (notmuch-address--save-address-hash)
+				       (setq notmuch-address-full-harvest-finished t))
 				   (setq notmuch-address-last-harvest 0)))))))
 
 ;;
-- 
2.1.4

Re: [PATCH 0/2] emacs allow save and load of the address-completions

[David Bremner]

Mark Walters <markwalters1009@gmail.com> writes:

> This series adds a load/save functionality for the address
> completions. This avoids the slow and poor completion provided until
> the full address-hash has been generated -- this can take some time,
> (over a minute on my machine), and is annoying if you frequently


Series pushed to master.

By the way, I notice there are many variables starting with
notmuch-address that are not in the notmuch-address customize group.

d