[PATCH v2 0/3] postpone and resume support

[PATCH v2 0/3] postpone and resume support

[Mark Walters]

This is a new version of the WIP patch at
id:1464915472-5669-1-git-send-email-markwalters1009@gmail.com

So far it seems to deal with all cases that I have tried, and the
CAVEATS list is rather smaller than before.

The bindings are C-x C-s to save a draft (in notmuch-message-mode) C-c
C-p to postpone a draft (ie save and exit buffer), and "e" to resume
editing from either show or tree mode. You may want to add "draft" to
your list of excluded tags to hide them unless you specifically search
for them.

CAVEATS

Attachments work, but the attachment that will be sent is the
attachment that was there when the message was postponed. (Attachments
added after resume are obviously not added until the message is sent.)

Each save saves a separate copy of the message. We hide these copies
with the deleted tag but we don't actually delete them. Thus if you
save several copies of message with large attachments you will use a
lot of space.

If you use signing or encryption then I don't know what will happen: 
I have not tested at all.  You might sign a partial message that you
didn't mean too; you might expose plain text to someone.

Finally, and this is not really a caveat, it *may* be possible to resume
a previously sent message. At the moment I haven't tested this, and
have made it so that emacs warns before allowing it.

Best wishes

Mark


Mark Walters (3):
  emacs: tree: move binding for pressing button in message pane to g
  emacs: notmuch-check-exit-status bugfix
  emacs:  postpone/resume support

 emacs/notmuch-lib.el     |  10 +++-
 emacs/notmuch-message.el | 135 +++++++++++++++++++++++++++++++++++++++++++++++
 emacs/notmuch-mua.el     |   4 ++
 emacs/notmuch-show.el    |  13 +++++
 emacs/notmuch-tree.el    |   3 +-
 5 files changed, 162 insertions(+), 3 deletions(-)

-- 
2.1.4

[PATCH v2 1/3] emacs: tree: move binding for pressing button in message pane to g

[Mark Walters]

We want to use "e" for editting postponed messages in show, and in
tree view, so bind the function which does

     (In message pane) Activate BUTTON or button at point

to "g" (go) instead.
---
 emacs/notmuch-tree.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index 4f9ca2d..6c35543 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -254,7 +254,7 @@ FUNC."
     (define-key map (kbd "M-TAB") (notmuch-tree-to-message-pane #'notmuch-show-previous-button))
     (define-key map (kbd "<backtab>")  (notmuch-tree-to-message-pane #'notmuch-show-previous-button))
     (define-key map (kbd "TAB") (notmuch-tree-to-message-pane #'notmuch-show-next-button))
-    (define-key map "e" (notmuch-tree-to-message-pane #'notmuch-tree-button-activate))
+    (define-key map "g" (notmuch-tree-to-message-pane #'notmuch-tree-button-activate))
 
     ;; bindings from show (or elsewhere) but we close the message pane first.
     (define-key map "f" (notmuch-tree-close-message-pane-and #'notmuch-show-forward-message))
-- 
2.1.4

[PATCH v2 2/3] emacs: notmuch-check-exit-status bugfix

[Mark Walters]

This function prints diagnostic information in the event of an
error. However, one of the callers has an optional :stdin-string
keyword argument. This causes the error printing routine to error
itself.

Rather than reach notmuch-check-exit-status about the possible keyword
arguments (currently only one but could be more in the future) this
commit just tells notmuch-check-exit-status how to print non-string arguments.
---

This is an existing bug in the emacs library which the postpone code
is liable to hit (if you try to save when the database is locked).

emacs/notmuch-lib.el | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index f05ded6..36a7262 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -790,9 +790,15 @@ You may need to restart Emacs or upgrade your notmuch package."))
 		    (insert-file-contents err-file)
 		    (unless (eobp)
 		      (buffer-string)))))
+	   (command-string
+	    (mapconcat (lambda (arg)
+			 (shell-quote-argument
+			  (cond ((stringp arg) arg)
+				((symbolp arg) (symbol-name arg))
+				(t "*UNKNOWN ARGUMENT*"))))
+		       command " "))
 	   (extra
-	    (concat
-	     "command: " (mapconcat #'shell-quote-argument command " ") "\n"
+	    (concat "command: " command-string "\n"
 	     (if (integerp exit-status)
 		 (format "exit status: %s\n" exit-status)
 	       (format "exit signal: %s\n" exit-status))
-- 
2.1.4

[PATCH v2 3/3] emacs: postpone/resume support

[Mark Walters]

This provides preliminary support for postponing and resuming in the
emacs frontend. On postponing it uses notmuch insert to put the
message in the notmuch database; resume gets the raw file from notmuch
and using the emacs function mime-to-mml reconstructs the message
(including attachments).

Current bindings are C-x C-s to save a draft, C-c C-p to postpone a
draft (save and exit compose buffer), and e to resume a draft from
show or tree mode.

Previous drafts get tagged deleted on subsequent saves, or on the
message being sent.

Each draft gets its own message-id, and we use the namespace
draft-.... for draft message ids (so, at least for most people, drafts
are easily distinguisable).
---
 emacs/notmuch-message.el | 135 +++++++++++++++++++++++++++++++++++++++++++++++
 emacs/notmuch-mua.el     |   4 ++
 emacs/notmuch-show.el    |  13 +++++
 emacs/notmuch-tree.el    |   1 +
 4 files changed, 153 insertions(+)

diff --git a/emacs/notmuch-message.el b/emacs/notmuch-message.el
index d437b85..0bfb4d3 100644
--- a/emacs/notmuch-message.el
+++ b/emacs/notmuch-message.el
@@ -25,6 +25,8 @@
 (require 'notmuch-tag)
 (require 'notmuch-mua)
 
+(declare-function notmuch-show-get-message-id "notmuch-show" (&optional bare))
+
 (defcustom notmuch-message-replied-tags '("+replied")
   "List of tag changes to apply to a message when it has been replied to.
 
@@ -38,6 +40,32 @@ the \"inbox\" and \"todo\" tags, you would set:
   :type '(repeat string)
   :group 'notmuch-send)
 
+(defcustom notmuch-message-draft-tags '("+draft")
+  "List of tags changes to apply to a draft message when it is saved in the database.
+
+Tags starting with \"+\" (or not starting with either \"+\" or
+\"-\") in the list will be added, and tags starting with \"-\"
+will be removed from the message being stored.
+
+For example, if you wanted to give the message a \"draft\" tag
+but not the (normally added by default) \"inbox\" tag, you would
+set:
+    (\"+draft\" \"-inbox\")"
+  :type '(repeat string)
+  :group 'notmuch-send)
+
+(defcustom notmuch-message-draft-folder "drafts"
+  "Folder to save draft messages in.
+
+This should be specified relative to the root of the notmuch
+database. It will be created if necessary."
+  :type 'string
+  :group 'notmuch-send)
+
+(defvar notmuch-message-draft-id nil
+  "Message-id of the most recent saved draft of this message")
+(make-variable-buffer-local 'notmuch-message-draft-id)
+
 (defun notmuch-message-mark-replied ()
   ;; get the in-reply-to header and parse it for the message id.
   (let ((rep (mail-header-parse-addresses (message-field-value "In-Reply-To"))))
@@ -45,7 +73,114 @@ the \"inbox\" and \"todo\" tags, you would set:
       (notmuch-tag (notmuch-id-to-query (car (car rep)))
 	       (notmuch-tag-change-list notmuch-message-replied-tags)))))
 
+(defun notmuch-message-mark-draft-deleted ()
+  "Tag the last saved draft deleted.
+
+Used when a new version is saved, or the message is sent."
+  (when notmuch-message-draft-id
+    (notmuch-tag notmuch-message-draft-id '("+deleted"))))
+
+(defun notmuch-message-save-draft ()
+  "Save the current draft message in the notmuch database.
+
+This saves the current message in the database with tags
+`notmuch-message-draft-tags` (in addition to any default tags
+applied to newly inserted messages)."
+  (interactive)
+
+  ;; This is based on message-do-fcc but modified for our needs.
+  (let ((case-fold-search t)
+	(buf (current-buffer))
+	(mml-externalize-attachments nil)
+	;; We generate a message id now as we will need it later. Note
+	;; message-make-message-id gives the id inside a "<" ">" pair,
+	;; but notmuch doesn't want that form, so remove them.
+	(id (concat "draft-" (substring (message-make-message-id) 1 -1))))
+    (with-current-buffer (get-buffer-create " *message temp*")
+      (erase-buffer)
+      (insert-buffer-substring buf)
+      ;; We insert a Date header and a Message-ID header, the former
+      ;; so that it is easier to search for the message, and the
+      ;; latter so we have a way of accessing the saved message (for
+      ;; example to delete it at a later time). We check that the
+      ;; user has these in `message-deletable-headers` (the default)
+      ;; as otherwise they are doing something strange and we
+      ;; shouldn't interfere. Note, since we are doing this in a new
+      ;; buffer we don't change the version in the compose buffer.
+      (if (member 'Message-ID message-deletable-headers)
+	  (progn
+	    (message-remove-header "Message-ID")
+	    (message-add-header (concat "Message-ID: <" id ">")))
+	(message "You have customized emacs so Message-ID is not a deletable header, so not changing it")
+	(setq id nil))
+      (if (member 'Date message-deletable-headers)
+	  (progn
+	    (message-remove-header "Date")
+	    (message-add-header (concat "Date: " (message-make-date))))
+	(message "You have customized emacs so Date is not a deletable header, so not changing it"))
+
+      ;; Back to following message-do-fcc
+      (message-encode-message-body)
+      (save-restriction
+	(message-narrow-to-headers)
+	(let ((mail-parse-charset message-default-charset)
+	      (rfc2047-header-encoding-alist
+	       (cons '("Newsgroups" . default)
+		     rfc2047-header-encoding-alist)))
+	  (mail-encode-encoded-word-buffer)))
+      (goto-char (point-min))
+      (when (re-search-forward
+	     (concat "^" (regexp-quote mail-header-separator) "$")
+	     nil t)
+	(replace-match "" t t ))
+
+      (apply 'notmuch-call-notmuch-process :stdin-string (buffer-string)
+	     "insert" "--create-folder"
+	     (concat "--folder=" notmuch-message-draft-folder)
+	     notmuch-message-draft-tags))
+    ;; We are now back in the original compose buffer. Note the
+    ;; function notmuch-call-notmuch-process signals an error on
+    ;; failure, so to get to this point it must have succeeded. Note
+    ;; notmuch-message-draft-id is still the id of the previous draft,
+    ;; so it is safe to mark it deleted.
+    (notmuch-message-mark-draft-deleted)
+    (setq notmuch-message-draft-id (concat "id:" id))
+    (set-buffer-modified-p nil)))
+
+(defun notmuch-message-postpone ()
+  "Save the draft message in the notmuch database and exit buffer."
+  (interactive)
+  (notmuch-message-save-draft)
+  (kill-buffer))
+
+(defun notmuch-message-resume (id)
+  "Resume editting of message with id ID."
+  (switch-to-buffer (get-buffer-create (concat "*notmuch-draft-" id "*")))
+  (setq buffer-read-only nil)
+  (erase-buffer)
+  (let ((coding-system-for-read 'no-conversion))
+      (call-process notmuch-command nil t nil "show" "--format=raw" id))
+  (mime-to-mml)
+  (goto-char (point-min))
+  (when (re-search-forward "^$" nil t)
+    (replace-match mail-header-separator t t))
+  ;; Remove our added Date and Message-ID headers (unless the user has
+  ;; explicitly customized emacs to tell us not to).
+  (save-restriction
+    (message-narrow-to-headers)
+    (when (member 'Message-ID message-deletable-headers)
+      (message-remove-header "Message-ID"))
+    (when (member 'Date message-deletable-headers)
+      (message-remove-header "Date")))
+  (notmuch-message-mode)
+  (set-buffer-modified-p t)
+  ;; Set the draft message-id so that we can delete the current saved draft if the
+  ;; message is resaved or sent.
+  (setq notmuch-message-draft-id id))
+
+
 (add-hook 'message-send-hook 'notmuch-message-mark-replied)
+(add-hook 'message-send-hook 'notmuch-message-mark-draft-deleted)
 
 (provide 'notmuch-message)
 
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 399e138..3118e5d 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -33,6 +33,8 @@
 (declare-function notmuch-show-insert-body "notmuch-show" (msg body depth))
 (declare-function notmuch-fcc-header-setup "notmuch-maildir-fcc" ())
 (declare-function notmuch-fcc-handler "notmuch-maildir-fcc" (destdir))
+(declare-function notmuch-message-postpone "notmuch-message" ())
+(declare-function notmuch-message-save-draft "notmuch-message" ())
 
 ;;
 
@@ -283,6 +285,8 @@ mutiple parts get a header."
 
 (define-key notmuch-message-mode-map (kbd "C-c C-c") #'notmuch-mua-send-and-exit)
 (define-key notmuch-message-mode-map (kbd "C-c C-s") #'notmuch-mua-send)
+(define-key notmuch-message-mode-map (kbd "C-c C-p") #'notmuch-message-postpone)
+(define-key notmuch-message-mode-map (kbd "C-x C-s") #'notmuch-message-save-draft)
 
 (defun notmuch-mua-pop-to-buffer (name switch-function)
   "Pop to buffer NAME, and warn if it already exists and is
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index f33096c..998fd27 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -38,6 +38,7 @@
 (require 'notmuch-mua)
 (require 'notmuch-crypto)
 (require 'notmuch-print)
+(require 'notmuch-message)
 
 (declare-function notmuch-call-notmuch-process "notmuch" (&rest args))
 (declare-function notmuch-search-next-thread "notmuch" nil)
@@ -1425,6 +1426,7 @@ reset based on the original query."
     (define-key map "|" 'notmuch-show-pipe-message)
     (define-key map "w" 'notmuch-show-save-attachments)
     (define-key map "V" 'notmuch-show-view-raw-message)
+    (define-key map "e" 'notmuch-show-resume-message)
     (define-key map "c" 'notmuch-show-stash-map)
     (define-key map "h" 'notmuch-show-toggle-visibility-headers)
     (define-key map "*" 'notmuch-show-tag-all)
@@ -1955,6 +1957,17 @@ to show, nil otherwise."
     (setq buffer-read-only t)
     (view-buffer buf 'kill-buffer-if-not-modified)))
 
+(defun notmuch-show-resume-message ()
+  "Resume EDITING the current draft message.
+
+Resume the current message. Queries if the message does not
+appear to be a draft."
+  (interactive)
+  (let ((tags (notmuch-show-get-tags)))
+    (when (or (equal tags (notmuch-update-tags tags notmuch-message-draft-tags))
+	      (yes-or-no-p "Message does not appear to be a draft: really resume?"))
+      (notmuch-message-resume (notmuch-show-get-message-id)))))
+
 (put 'notmuch-show-pipe-message 'notmuch-doc
      "Pipe the contents of the current message to a command.")
 (put 'notmuch-show-pipe-message 'notmuch-prefix-doc
diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index 6c35543..c759290 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -261,6 +261,7 @@ FUNC."
     (define-key map "r" (notmuch-tree-close-message-pane-and #'notmuch-show-reply-sender))
     (define-key map "R" (notmuch-tree-close-message-pane-and #'notmuch-show-reply))
     (define-key map "V" (notmuch-tree-close-message-pane-and #'notmuch-show-view-raw-message))
+    (define-key map "e" (notmuch-tree-close-message-pane-and #'notmuch-show-resume-message))
 
     ;; The main tree view bindings
     (define-key map (kbd "RET") 'notmuch-tree-show-message)
-- 
2.1.4

Re: [PATCH v2 1/3] emacs: tree: move binding for pressing button in message pane to g

[David Bremner]

Mark Walters <markwalters1009@gmail.com> writes:

> We want to use "e" for editting postponed messages in show, and in
> tree view, so bind the function which does
>
>      (In message pane) Activate BUTTON or button at point
>
> to "g" (go) instead.

IMHO it's a minor irritant that we don't use 'g' for refresh. That's an
almost universal convention in emacs. Of course that problem exists
already, but this somehow makes it worse by precluding using 'g' for
refresh in the future.

d

Re: [PATCH v2 0/3] postpone and resume support

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 3147 bytes --]

On Fri 2016-06-03 13:49:52 -0400, Mark Walters <markwalters1009@gmail.com> wrote:
> This is a new version of the WIP patch at
> id:1464915472-5669-1-git-send-email-markwalters1009@gmail.com
>
> So far it seems to deal with all cases that I have tried, and the
> CAVEATS list is rather smaller than before.
>
> The bindings are C-x C-s to save a draft (in notmuch-message-mode) C-c
> C-p to postpone a draft (ie save and exit buffer), and "e" to resume
> editing from either show or tree mode. You may want to add "draft" to
> your list of excluded tags to hide them unless you specifically search
> for them.

This is a really useful series, even with the caveats Mark lists.  i've
tested it and it behaves as expected for me.

A bit more discussion on the caveats:

> Each save saves a separate copy of the message. We hide these copies
> with the deleted tag but we don't actually delete them. Thus if you
> save several copies of message with large attachments you will use a
> lot of space.

This is a little bit weird, because it means that all these deleted
drafts show up in the thread view when viewing the thread from which the
message was composed.  Is there a reason to not actually delete older
drafts when "re-postponing" or sending?  We created the message
ourselves, so it seems like it's fair to delete it.

> If you use signing or encryption then I don't know what will happen: 
> I have not tested at all.  You might sign a partial message that you
> didn't mean too; you might expose plain text to someone.

I've just tested this for signing, and it's doing the thing i was afraid
it would do :/ When saving a draft marked as a message to be signed, the
message signature is made on the draft.  Even worse, restoring a saved
draft like this leaves the user editing the plaintext while the old
message signature is likely to be appended, which will result in broken
signatures :/

Maybe there's a way to temporarily inhibit the interpretation of all
#secure flags during draft saving (and propagate them through upon
resume)?  I think that would be advisable anyway -- if we ultimately
decide that we want to encrypt drafts, we don't want to encrypt them to
the actual recipients anyway; we only want encrypt-to-self, because we
want to avoid the situation where the person we're sending the message
to gets access to our drafts folder and reads our unfinished/unedited
compositions, so that would be a separate and distinct improvement.

> Finally, and this is not really a caveat, it *may* be possible to resume
> a previously sent message. At the moment I haven't tested this, and
> have made it so that emacs warns before allowing it.

I've tested it.  It's possible, and it even seems to work :) there are
weird interactions between things like DKIM and modified message-ids,
but i think the warning provided is sufficient to let people know that
if they're doing that sort of thing and something breaks, they get to
keep both pieces.  The biggest danger, i think, is in marking the
non-draft for deletion if a user goes ahead with restoring it.

          --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 948 bytes --]