Emacs: postponing messages

Emacs: postponing messages

[Mark Walters]

Dear All

There was some discussion on irc yesterday about a better way of
postponing message when using the emacs frontend. I think getting a
moderately nice interface should be quite easy (see below) but there are
some corner cases on what *should* happen that I would like to resolve
before trying to implement anything.

My broad idea for postpone is to take the partial message, use notmuch
insert to put it in the database with a "postponed" tag, and then on
resume fetch the raw message and go into notmuch-message-mode, and also
either add a deleted tag to the resumed message, or better actually
delete the resumed message. Finally, we would add postponed to the
excluded tags list, so that postponed messages only show up when
searched for.

Now the main difficulty I see is what to do with attachments.

If we just leave the mml #part markers in the message then the file
might have disappeared by the time we resume (or could even be a
different file). This would be a particular problem if people attach
files that are in /tmp say.

Further the way rfc822 forwarding works (ie the default forwarding in
notmuch/emacs) is to put each attachment of the message that is being
forwarded into its own emacs buffer, and include these buffers via the
mml markers. These buffers would all disappear if the user restarts
emacs between postponing and resume.

An alternative would be to attach the attachments with the postponed
message. This is probably doable by writing the message (as if being
fcc'd) to notmuch insert, and then using the mime-to-mml function to
reverse the process. The downside here is that now the attached file is
not the current file in the filesystem when you send -- ie its different
from the normal case.

Any thoughts or comments very gratefully received!

Best wishes

Mark

Re: Emacs: postponing messages

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

On Thu 2016-06-02 14:21:44 -0400, Mark Walters wrote:
> My broad idea for postpone is to take the partial message, use notmuch
> insert to put it in the database with a "postponed" tag, and then on
> resume fetch the raw message and go into notmuch-message-mode, and also
> either add a deleted tag to the resumed message, or better actually
> delete the resumed message. Finally, we would add postponed to the
> excluded tags list, so that postponed messages only show up when
> searched for.

I'd love to see this happen.  The terminology many other mail user
agents use for this workflow is often "draft" or "drafts" so we might do
well to adopt that term, instead of using "postpone", which i don't
think is as widely-used.

> An alternative would be to attach the attachments with the postponed
> message. This is probably doable by writing the message (as if being
> fcc'd) to notmuch insert, and then using the mime-to-mml function to
> reverse the process. The downside here is that now the attached file is
> not the current file in the filesystem when you send -- ie its different
> from the normal case.

I like this approach, and i don't think that the caveat you're
describing is a particuarly bad one, though it depends on how
mime-to-mml works.

I just tested it, and mime-to-mml actually produced something that was
not directly sendable in notmuch-message-mode, because it didn't include
the "--text follows this line--" break between headers and body :/  But
this is probably fixable ;)

However, mime-to-mml actually embedded the content of the included
sub-part upon reconstruction and the filename was only the leaf filename
(i'd included /home/dkg/tmp/test.txt, and in the reconstructed #part it
said filename="test.txt" nofile=yes)

Since this doesn't include the original path of the file (and it said
"nofile=yes"), then i don't think it's a problem.

One thing i should note is that if there's a message-id assigned during
saving of the draft, then we need to think carefully about how a draft
that gets saved multiple times gets indexed.  it'll have the same
message-id, which is good, but there will likely be multiple files
referencing it, each with different content.  ideally, there would only
be one copy indexed, and it would be the latest one.  Also, the
actually-sent mail should be indexed in preference over a previous
draft.

The above pieces would all be really great to have!  In addition, if
they were in place, it'd be good to have an variant notmuch-search view
that lists the recipients instead of the senders, so that i can do a
search for tag:draft and actually see who the messages were sent to,
instead of my own name (as the sender) on each item :)

        --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 948 bytes --]

crypto and draft messages [was: Re: Emacs: postponing messages]

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

On Thu 2016-06-02 14:21:44 -0400, Mark Walters wrote:
> There was some discussion on irc yesterday about a better way of
> postponing message when using the emacs frontend. I think getting a
> moderately nice interface should be quite easy (see below) but there are
> some corner cases on what *should* happen that I would like to resolve
> before trying to implement anything.

one other corner case worth thinking about here (it can probably be
postponed until we have base cases handled, but i wanted to bring it up)
is how per-message cryptographic operations (mml-secure-*) interact with
drafts.

In particular, i think that any sort of message signing should *not*
happen during saving of a draft, but the intent to sign should be
preserved.  That is, we should save and restore the #secure tag when
saving a draft or restoring a draft, but the saved draft itself should
*not* be signed.

for encryption, i have a different (and arguably opposite) intuition.
if the sender has the ability to *decrypt* mails, i'd argue that saving
a draft should encrypt the draft, regardless of the draft's stated
intent to encrypt.

These cases matter because i know many people use tools like
offline-imap to sync their mail store with a remote mailserver.

if the remote mailserver can get a copy of the signed draft, it could
replay it (effectively making use of an unintentional signature).
Likewise, if the user doesn't think about encrypting a message until
they're they're ready to send it, then an intermediate/draft version of
the message might end up in cleartext on the remote server.

    --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 948 bytes --]

[WIP PATCH] emacs: postpone/resume support

[Mark Walters]

This provides preliminary support for postponing and resuming in the
emacs frontend. On postponing it uses notmuch insert to put the
message in the notmuch database; resume gets the raw file from notmuch
and using the emacs function mime-to-mml reconstructs the message
(including attachments).
---

This is a WIP patch enabling postpone/resume. On very light testing so
far it seems to work surprisingly well.

Please note that it is an early version, so it may eat your draft
message, mangle (or mysteriously corrupt) your draft message etc; and
it each draft will remain in your message store unless you manually
delete it.

To use: to postpone just do M-x notmuch-message-postpone in your
compose buffer. To resume fo M-x notmuch-message-resume while on the
correct message in a show buffer. You may want to add "draft" to your
exclude tags in .notmuch-config so that drafts only show up if you
search for them explicitly.

CAVEATS:

Attachments work, but the attachment that is sent is the attachment
that was there when the message was postponed. (Attachments added
after resume are obviously not added until the message is sent.)

If the database is locked when you try and postpone it will
mysteriously fail. I think this is actually caused by a bug elsewhere
in notmuch. But in any case the compose buffer should not be killed in
this case.

It would probably be nice to delete the messages on resume. Since we
don't really have that functionality, we just add a deleted tag. This
will hide them in search buffers but, if you postpone a reply, it ill
still show up (collapsed) in the show buffer for the thread.

If you resume a postponed message and then postpone again without
editting then the deleted tag will not be removed so the message may
be hard to find. (see below for possible fixed for this)

Messages don't get a date header as emacs adds that when it sends it
so notmuch will list all postponed message as 1 Jan 1970. 

If you use signing or encryption then I don't know what will happen --
I have not tested at all.  You might sign a partial message that you
didn't mean too; you might expose plain text to someone.

WHAT NEEDS DOING

Trivially we would want some keybindings for the two commands (any
suggested keybindings gratefully received). More significantly, I
would either like a flag to notmuch insert to say "make these tag
changes even if the message already exists"; or perhaps a notmuch
delete command that (genuinely) deletes a single message.

Best wishes

Mark



emacs/notmuch-message.el | 90 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)

diff --git a/emacs/notmuch-message.el b/emacs/notmuch-message.el
index d437b85..fdd91c0 100644
--- a/emacs/notmuch-message.el
+++ b/emacs/notmuch-message.el
@@ -25,6 +25,8 @@
 (require 'notmuch-tag)
 (require 'notmuch-mua)
 
+(declare-function notmuch-show-get-message-id "notmuch-show" (&optional bare))
+
 (defcustom notmuch-message-replied-tags '("+replied")
   "List of tag changes to apply to a message when it has been replied to.
 
@@ -38,6 +40,28 @@ the \"inbox\" and \"todo\" tags, you would set:
   :type '(repeat string)
   :group 'notmuch-send)
 
+(defcustom notmuch-message-draft-tags '("+draft")
+  "List of tags changes to apply to a draft message when it is saved in the database.
+
+Tags starting with \"+\" (or not starting with either \"+\" or
+\"-\") in the list will be added, and tags starting with \"-\"
+will be removed from the message being stored.
+
+For example, if you wanted to give the message a \"draft\" tag
+but not the (normally added by default) \"inbox\" tag, you would
+set:
+    (\"+draft\" \"-inbox\")"
+  :type '(repeat string)
+  :group 'notmuch-send)
+
+(defcustom notmuch-message-draft-folder "drafts"
+  "Folder to save draft messages in.
+
+This should be specified relative to the root of the notmuch
+database. It will be created if necessary."
+  :type 'string
+  :group 'notmuch-send)
+
 (defun notmuch-message-mark-replied ()
   ;; get the in-reply-to header and parse it for the message id.
   (let ((rep (mail-header-parse-addresses (message-field-value "In-Reply-To"))))
@@ -45,6 +69,72 @@ the \"inbox\" and \"todo\" tags, you would set:
       (notmuch-tag (notmuch-id-to-query (car (car rep)))
 	       (notmuch-tag-change-list notmuch-message-replied-tags)))))
 
+(defun notmuch-message-postpone ()
+  "Save the current draft message in the notmuch database.
+
+This saves the current message in the database with tags
+`notmuch-message-draft-tags` (in addition to any default tags
+applied to newly inserted messages)."
+  ;; This is taken from message-do-fcc but modified for our needs.  In
+  ;; particular we don't want to process other fcc lines, nor remove
+  ;; them. Note some of the following may not be needed -- I tried to
+  ;; stay close to the original function.
+  (interactive)
+  (let ((case-fold-search t)
+	(buf (current-buffer))
+	(mml-externalize-attachments nil))
+    (with-current-buffer (get-buffer-create " *message temp*")
+      (erase-buffer)
+      (insert-buffer-substring buf)
+      (message-encode-message-body)
+      (save-restriction
+	(message-narrow-to-headers)
+	(let ((mail-parse-charset message-default-charset)
+	      (rfc2047-header-encoding-alist
+	       (cons '("Newsgroups" . default)
+		     rfc2047-header-encoding-alist)))
+	  (mail-encode-encoded-word-buffer)))
+      (goto-char (point-min))
+      (when (re-search-forward
+	     (concat "^" (regexp-quote mail-header-separator) "$")
+	     nil t)
+	(replace-match "" t t ))
+
+      ;; Unless the user has entered a message-id manually the message
+      ;; does not have one -- thus notmuch will use the sha1 of the
+      ;; message. Hence we should only get a collision if the message
+      ;; is identical to a previous draft. In this case we should
+      ;; remove the deleted tag, but it is not clear howto.
+      (apply 'notmuch-call-notmuch-process :stdin-string (buffer-string)
+	     "insert" "--create-folder"
+	     (concat "--folder=" notmuch-message-draft-folder)
+	     notmuch-message-draft-tags)))
+  ;; The function notmuch-call-notmuch-process signals an error on failure, so
+  ;; to get to this point it must have succeeded.
+  (set-buffer-modified-p nil)
+  (kill-buffer))
+
+(defun notmuch-message-resume (&optional id)
+  "View the original source of the current message."
+  (interactive)
+  (let* ((id (or id (notmuch-show-get-message-id)))
+	 (buf (get-buffer-create (concat "*notmuch-draft-" id "*")))
+	 (inhibit-read-only t))
+    (switch-to-buffer buf)
+    (setq buffer-read-only nil)
+    (erase-buffer)
+    (let ((coding-system-for-read 'no-conversion))
+      (call-process notmuch-command nil t nil "show" "--format=raw" id))
+    (mime-to-mml)
+    (goto-char (point-min))
+    (when (re-search-forward "^$" nil t)
+      (replace-match mail-header-separator t t))
+    (notmuch-message-mode)
+    (set-buffer-modified-p t)
+    ;; Delete the message (since we can't do this yet just tag it deleted).
+    (notmuch-tag id '("+deleted"))))
+
+
 (add-hook 'message-send-hook 'notmuch-message-mark-replied)
 
 (provide 'notmuch-message)
-- 
2.1.4