[PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[Jameson Graef Rollins]

---
 NEWS |   15 ++++++++++++++-
 1 files changed, 14 insertions(+), 1 deletions(-)

diff --git a/NEWS b/NEWS
index ee84e9a..f688bd0 100644
--- a/NEWS
+++ b/NEWS
@@ -39,7 +39,20 @@ Ruby bindings changes
   Wrap new library functions notmuch_database_{begin,end}_atomic, 
   and add new exception Notmuch::UnbalancedAtomicError. Rename destroy 
   to destroy! according to Ruby naming conventions.
-  
+
+Emacs improvements
+------------------
+
+  * Add gpg callback to crypto sigstatus buttons to retrieve/refresh
+    signing key.
+  * Add notmuch-show-refresh-view function (and corresponding binding)
+    to refresh the view of a notmuch-show buffer.
+
+Reply formatting cleanup
+------------------------
+
+  "notmuch reply" no longer includes notification that non-leafnode
+  MIME parts are being suppressed.
 
 Notmuch 0.8 (2011-09-10)
 ========================
-- 
1.7.6.3

Re: [PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

Ok, I just realized why these messages aren't showing up in reply to the
messages I intend: it's the "id:" prefix that's there when I use "c i"
to copy the message ID to the kill-ring!  I'm then in-reply-to'ing

id:87ipnzd4ds.fsf@zancas.localnet

instead of just

87ipnzd4ds.fsf@zancas.localnet

Patch to fix that to follow!

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[David Bremner]

On Sat,  8 Oct 2011 20:47:47 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> ---
>  NEWS |   15 ++++++++++++++-
>  1 files changed, 14 insertions(+), 1 deletions(-)

Pushed, thanks.

Re: [PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]

Hi!

On Sat, 08 Oct 2011 20:55:59 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> Ok, I just realized why these messages aren't showing up in reply to the
> messages I intend: it's the "id:" prefix that's there when I use "c i"
> to copy the message ID to the kill-ring!  I'm then in-reply-to'ing
> 
> id:87ipnzd4ds.fsf@zancas.localnet
> 
> instead of just
> 
> 87ipnzd4ds.fsf@zancas.localnet

Well, yes.  :-)

> Patch to fix that to follow!

Hmm, don't know.  Of course, c f also doesn't stash From:"Some One
<someone@some.where>", but just Some One <someone@some.where>.  On the
other hand, the id:"[Message ID]" format has the advantage that it's
directly usable as a notmuch search term.  I'm ambivalent.


Grüße,
 Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [PATCH] NEWS: add notes about emacs improvements and reply formating cleanup

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On Sun, 09 Oct 2011 19:54:17 +0200, Thomas Schwinge <thomas@schwinge.name> wrote:
> Hmm, don't know.  Of course, c f also doesn't stash From:"Some One
> <someone@some.where>", but just Some One <someone@some.where>.  On the
> other hand, the id:"[Message ID]" format has the advantage that it's
> directly usable as a notmuch search term.  I'm ambivalent.

I agree that it's sort of nice to be able to paste the result directly
on the command line, but I think it's really not that big of a deal to
just type "id:" before pasting on the command line.  I think that the
argument of consistency and the confusion it causes in non-notmuch
contexts makes it a stronger argument to remove the search term prefix.

My patch follows.  If anyone has any really vehement objection please
raise them, but ultimately I think this makes things cleaner and less
likely to cause problems in the long run.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

[PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Jameson Graef Rollins]

This modifies the notmuch-show "c i" binding to yank just the
message-id to the kill-ring, without the "id:" prefix.  This makes
this yank binding consistent with the rest of the yank bindings in
that they don't include any search term prefixes.  This is a more
natural interface and will cause less confusion when pasting into
non-notmuch contexts.

Internal functions are modified to prepend the "id:" prefix as needed,
including the notmuch-show-get-bodypart-internal and
notmuch-show-save-part internal functions which are modified to accept
just the bare message-id as the primary argument.
---
 emacs/notmuch-show.el |   27 ++++++++++++++-------------
 test/emacs            |    2 +-
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 8db5435..be85b12 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -99,7 +99,7 @@ any given message."
 (defmacro with-current-notmuch-show-message (&rest body)
   "Evaluate body with current buffer set to the text of current message"
   `(save-excursion
-     (let ((id (notmuch-show-get-message-id)))
+     (let ((id (concat "id:" (notmuch-show-get-message-id))))
        (let ((buf (generate-new-buffer (concat "*notmuch-msg-" id "*"))))
          (with-current-buffer buf
 	    (call-process notmuch-command nil t nil "show" "--format=raw" id)
@@ -412,7 +412,7 @@ current buffer, if possible."
 	  ;; times (hundreds!), which results in many calls to
 	  ;; `notmuch part'.
 	  (unless content
-	    (setq content (notmuch-show-get-bodypart-internal (concat "id:" message-id)
+	    (setq content (notmuch-show-get-bodypart-internal message-id
 							      part-number))
 	    (with-current-buffer w3m-current-buffer
 	      (notmuch-show-w3m-cid-store-internal url
@@ -615,12 +615,13 @@ current buffer, if possible."
 ;; Uses the buffer-local variable notmuch-show-process-crypto to
 ;; determine if parts should be decrypted first.
 (defun notmuch-show-get-bodypart-internal (message-id part-number)
-  (let ((args '("show" "--format=raw"))
+  (let ((id (concat "id:" message-id))
+	(args '("show" "--format=raw"))
 	(part-arg (format "--part=%s" part-number)))
     (setq args (append args (list part-arg)))
     (if notmuch-show-process-crypto
 	(setq args (append args '("--decrypt"))))
-    (setq args (append args (list message-id)))
+    (setq args (append args (list id)))
     (with-temp-buffer
       (let ((coding-system-for-read 'no-conversion))
 	(progn
@@ -629,7 +630,7 @@ current buffer, if possible."
 
 (defun notmuch-show-get-bodypart-content (msg part nth)
   (or (plist-get part :content)
-      (notmuch-show-get-bodypart-internal (concat "id:" (plist-get msg :id)) nth)))
+      (notmuch-show-get-bodypart-internal (plist-get msg :id) nth)))
 
 ;; \f
 
@@ -1050,7 +1051,7 @@ All currently available key bindings:
 
 (defun notmuch-show-get-message-id ()
   "Return the message id of the current message."
-  (concat "id:\"" (notmuch-show-get-prop :id) "\""))
+  (concat "\"" (notmuch-show-get-prop :id) "\""))
 
 ;; dme: Would it make sense to use a macro for many of these?
 
@@ -1195,7 +1196,7 @@ any effects from previous calls to
 (defun notmuch-show-reply (&optional prompt-for-sender)
   "Reply to the current message."
   (interactive "P")
-  (notmuch-mua-new-reply (notmuch-show-get-message-id) prompt-for-sender))
+  (notmuch-mua-new-reply (concat "id:" (notmuch-show-get-message-id)) prompt-for-sender))
 
 (defun notmuch-show-forward-message (&optional prompt-for-sender)
   "Forward the current message."
@@ -1242,7 +1243,7 @@ any effects from previous calls to
 (defun notmuch-show-view-raw-message ()
   "View the file holding the current message."
   (interactive)
-  (let* ((id (notmuch-show-get-message-id))
+  (let* ((id (concat "id:" (notmuch-show-get-message-id)))
 	 (buf (get-buffer-create (concat "*notmuch-raw-" id "*"))))
     (call-process notmuch-command nil buf nil "show" "--format=raw" id)
     (switch-to-buffer buf)
@@ -1266,11 +1267,11 @@ than only the current message."
 	(setq shell-command 
 	      (concat notmuch-command " show --format=mbox "
 		      (shell-quote-argument
-		       (mapconcat 'identity (notmuch-show-get-message-ids-for-open-messages) " OR "))
+		       (mapconcat 'identity "id:" (notmuch-show-get-message-ids-for-open-messages) " OR "))
 		      " | " command))
       (setq shell-command
 	    (concat notmuch-command " show --format=raw "
-		    (shell-quote-argument (notmuch-show-get-message-id)) " | " command)))
+		    (shell-quote-argument (concat "id:" (notmuch-show-get-message-id))) " | " command)))
     (let ((buf (get-buffer-create (concat "*notmuch-pipe*"))))
       (with-current-buffer buf
 	(setq buffer-read-only nil)
@@ -1312,7 +1313,7 @@ the result."
 	 (new-tags (notmuch-show-add-tags-worker current-tags toadd)))
 
     (unless (equal current-tags new-tags)
-      (apply 'notmuch-tag (notmuch-show-get-message-id)
+      (apply 'notmuch-tag (concat "id:" (notmuch-show-get-message-id))
 	     (mapcar (lambda (s) (concat "+" s)) toadd))
       (notmuch-show-set-tags new-tags))))
 
@@ -1320,13 +1321,13 @@ the result."
   "Remove a tag from the current message."
   (interactive
    (list (notmuch-select-tag-with-completion
-	  "Tag to remove: " (notmuch-show-get-message-id))))
+	  "Tag to remove: " (concat "id:" (notmuch-show-get-message-id)))))
 
   (let* ((current-tags (notmuch-show-get-tags))
 	 (new-tags (notmuch-show-del-tags-worker current-tags toremove)))
 
     (unless (equal current-tags new-tags)
-      (apply 'notmuch-tag (notmuch-show-get-message-id)
+      (apply 'notmuch-tag (concat "id:" (notmuch-show-get-message-id))
 	     (mapcar (lambda (s) (concat "-" s)) toremove))
       (notmuch-show-set-tags new-tags))))
 
diff --git a/test/emacs b/test/emacs
index 6d9cfd9..deb5808 100755
--- a/test/emacs
+++ b/test/emacs
@@ -262,7 +262,7 @@ test_expect_equal_file attachment1.gz "$EXPECTED/attachment"
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-part"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment2.gz\""))
-	      (notmuch-show-save-part "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com" 5))' > /dev/null 2>&1
+	      (notmuch-show-save-part "cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com" 5))' > /dev/null 2>&1
 test_expect_equal_file attachment2.gz "$EXPECTED/attachment"
 
 test_begin_subtest "View raw message within emacs"
-- 
1.7.6.3

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

Hi!

On Sun,  9 Oct 2011 15:35:48 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>  (defun notmuch-show-get-message-id ()
>    "Return the message id of the current message."
> -  (concat "id:\"" (notmuch-show-get-prop :id) "\""))
> +  (concat "\"" (notmuch-show-get-prop :id) "\""))

Shouldn't the double quotes be removed here, too?  (And be re-added in
the other places where id: is added.)


Grüße,
 Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

On Tue, 11 Oct 2011 01:46:43 +0200, Thomas Schwinge <thomas@schwinge.name> wrote:
> On Sun,  9 Oct 2011 15:35:48 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> >  (defun notmuch-show-get-message-id ()
> >    "Return the message id of the current message."
> > -  (concat "id:\"" (notmuch-show-get-prop :id) "\""))
> > +  (concat "\"" (notmuch-show-get-prop :id) "\""))
> 
> Shouldn't the double quotes be removed here, too?  (And be re-added in
> the other places where id: is added.)

Yes, I think you're right Thomas.  Afaik emacs doesn't need any kind of
escaping when storing strings internally.  I'll tweak that and resubmit
the patch.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

On Tue, 11 Oct 2011 01:46:43 +0200, Thomas Schwinge <thomas@schwinge.name> wrote:
> On Sun,  9 Oct 2011 15:35:48 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> >  (defun notmuch-show-get-message-id ()
> >    "Return the message id of the current message."
> > -  (concat "id:\"" (notmuch-show-get-prop :id) "\""))
> > +  (concat "\"" (notmuch-show-get-prop :id) "\""))
> 
> Shouldn't the double quotes be removed here, too?  (And be re-added in
> the other places where id: is added.)

In point of fact, do we need these quotes around message IDs at all?
Can message IDs have characters that would need to be escaped for the
shell?

This seems to also be related to the issue brought up by Erlend Simonsen
in [0], where emacs 24 seems to be shell-quoting things as needed.

So maybe these quotes aren't actually needed at all?...

jamie.

[0] id:87y5w9w6dm.fsf@fudge.hue.no

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Tomi Ollila]

On Thu 27 Oct 2011 05:52, Jameson Graef Rollins <jrollins@finestructure.net> writes:

> On Tue, 11 Oct 2011 01:46:43 +0200, Thomas Schwinge
> <thomas@schwinge.name> wrote:
>> On Sun, 9 Oct 2011 15:35:48 -0700, Jameson Graef Rollins
>> <jrollins@finestructure.net> wrote:
>> >  (defun notmuch-show-get-message-id ()
>> >    "Return the message id of the current message."
>> > -  (concat "id:\"" (notmuch-show-get-prop :id) "\""))
>> > +  (concat "\"" (notmuch-show-get-prop :id) "\""))
>> 
>> Shouldn't the double quotes be removed here, too?  (And be re-added in
>> the other places where id: is added.)
>
> In point of fact, do we need these quotes around message IDs at all?
> Can message IDs have characters that would need to be escaped for the
> shell?

Is there intermediate shell run when emacs run these commands, if not,
then shell expansion isn't happen. If there is, then the "simplest" way
to avoid shell expansion is to put strings in '' -quotes and replace
all intermediate ':s with '\'' (meaning single ' expands to ''\''' ;)

>
> This seems to also be related to the issue brought up by Erlend Simonsen
> in [0], where emacs 24 seems to be shell-quoting things as needed.
>
> So maybe these quotes aren't actually needed at all?...

These quotes are needed if emacs uses intermediate shell and emacs 
doesn't quote itself. if there is functionality change between emacs
23 and emacs 24 here, then that's interesting.

>
> jamie.
>

Tomi

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 2249 bytes --]

On Wed, 26 Oct 2011 19:52:59 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> In point of fact, do we need these quotes around message IDs at all?
> Can message IDs have characters that would need to be escaped for the
> shell?

Message ID syntax is defined in RFC 2822[1]:
> message-id      =       "Message-ID:" msg-id CRLF
> 
> in-reply-to     =       "In-Reply-To:" 1*msg-id CRLF
> 
> references      =       "References:" 1*msg-id CRLF
> 
> msg-id          =       [CFWS] "<" id-left "@" id-right ">" [CFWS]
> 
> id-left         =       dot-atom-text / no-fold-quote / obs-id-left
> 
> id-right        =       dot-atom-text / no-fold-literal / obs-id-right
> 
> no-fold-quote   =       DQUOTE *(qtext / quoted-pair) DQUOTE
>
> no-fold-literal =       "[" *(dtext / quoted-pair) "]"

And since
> quoted-pair     =       ("\" text) / obs-qp
>
> text            =       %d1-9 /         ; Characters excluding CR and LF
>                        %d11 /
>                        %d12 /
>                        %d14-127 /
>                        obs-text

It can theoretically include pretty much any character, including
quotes, if they are properly escaped. Hence, the query parser would have
to take that into account, but surrounding them by quotes probably isn't
useful anyway.

As far passing things to the shell goes:

> atext           =       ALPHA / DIGIT / ; Any character except controls,
>                         "!" / "#" /     ;  SP, and specials.
>                         "$" / "%" /     ;  Used for atoms
>                         "&" / "'" /
>                         "*" / "+" /
>                         "-" / "/" /
>                         "=" / "?" /
>                         "^" / "_" /
>                         "`" / "{" /
>                         "|" / "}" /
>                         "~"
> 
> atom            =       [CFWS] 1*atext [CFWS]
> 
> dot-atom        =       [CFWS] dot-atom-text [CFWS]
> 
> dot-atom-text   =       1*atext *("." 1*atext)

So, they can contain things like "$foo@$bar", which will be mangled if
passed to the shell, even with double quotes.

[1] http://tools.ietf.org/html/rfc2822#section-3.6.4

Cheers,
Daniel

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Tomi Ollila]

On Fri 28 Oct 2011 14:06, Daniel Schoepe <daniel@schoepe.org> writes:

> On Wed, 26 Oct 2011 19:52:59 -0700, Jameson Graef Rollins
> <jrollins@finestructure.net> wrote:
>> In point of fact, do we need these quotes around message IDs at all?
>> Can message IDs have characters that would need to be escaped for the
>> shell?
>
> Message ID syntax is defined in RFC 2822[1]:
...
> As far passing things to the shell goes:
>
>> atext           =       ALPHA / DIGIT / ; Any character except controls,
>>                         "!" / "#" /     ;  SP, and specials.
>>                         "$" / "%" /     ;  Used for atoms
>>                         "&" / "'" /
>>                         "*" / "+" /
>>                         "-" / "/" /
>>                         "=" / "?" /
>>                         "^" / "_" /
>>                         "`" / "{" /
>>                         "|" / "}" /
>>                         "~"
>> 
>> atom            =       [CFWS] 1*atext [CFWS]
>> 
>> dot-atom        =       [CFWS] dot-atom-text [CFWS]
>> 
>> dot-atom-text   =       1*atext *("." 1*atext)
>
> So, they can contain things like "$foo@$bar", which will be mangled if
> passed to the shell, even with double quotes.

atext includes ' -- how inconvenient for shell's sake. Indeed, the
"":s does noe disallog $var expansion, but avoids potential problems
with ', |, & and ` (and smaller problems with ~, *,  + and ?).
Without ' allowed in msg-id writing arg between '':s in shell command
line would have been convenient.

Now that ":s are removed from the id: argument, maybe also '<' and
">" could be removed... (or was it xapian which handles this?)

>
> [1] http://tools.ietf.org/html/rfc2822#section-3.6.4
>
> Cheers,
> Daniel

Tomi

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[David Bremner]

On Sun,  9 Oct 2011 15:35:48 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> This modifies the notmuch-show "c i" binding to yank just the
> message-id to the kill-ring, without the "id:" prefix.  This makes
> this yank binding consistent with the rest of the yank bindings in
> that they don't include any search term prefixes.

Hi Jamie, Hi all.

What do you think about the approach in

     id:"1309422050-23241-1-git-send-email-pieter@praet.org"

I sortof like the idea of having with and without prefix bindings.

Or maybe we could provide a prefix argument (getit? ;) ) so that C-u c i
adds the prefix.

David

Re: [PATCH] emacs: Modify notmuch-show-get-message-id to return message-id unprefixed with "id:".

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Sat, 12 Nov 2011 09:52:51 -0500, David Bremner <david@tethera.net> wrote:
> What do you think about the approach in
> 
>      id:"1309422050-23241-1-git-send-email-pieter@praet.org"
> 
> I sortof like the idea of having with and without prefix bindings.

That seems like a totally reasonable solution to me.

> Or maybe we could provide a prefix argument (getit? ;) ) so that C-u c i
> adds the prefix.

I'd rather just use a separate key ("I") instead of getting into the
whole prefix business.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]