[PATCH 1/2] test: emacs mailto: URI handling

[PATCH 1/2] test: emacs mailto: URI handling

[Jameson Graef Rollins]

This adds a test for proposed rfc6068 "mailto:" URI handling.  The
proposed function would be called 'notmuch-mua-mailto'.  The test
provides an example mailto: string that should test some subset of the
rfc6068 specification: http://tools.ietf.org/html/rfc6068
---
 test/emacs |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 8ca4c8a..c6afa14 100755
--- a/test/emacs
+++ b/test/emacs
@@ -273,6 +273,35 @@ On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> w
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "Compose to \"mailto:\" URI"
+test_subtest_known_broken
+# This *should* test the ability to parse rfc6068 "mailto:" URIs:
+# http://tools.ietf.org/html/rfc6068
+# this test is based on:
+# http://shadow2531.com/opera/testcases/mailto/rfc2368-0.html
+# which is itself based on the precursor rfc2368, so it could probably
+# be updated.
+test_emacs "(let ((mailto \"mailto:to1%40example.net%2C%20to2%40example.net?to=to3%40example.net%2C%20to4%40example.net&amp;subject=last%20call%20for%20cats%20%26%20dogs%2E&amp;subject=You%20should%20not%20see%20me%2E&amp;body=line1%0D%0Aline2&amp;body=line3&amp;body=line4%0D%0AColumn1A%09Column1B%09Column1C%09Column1D%0D%0A%E2%88%9A&amp;cc=cc1%40example.net%2C%20cc2%40example.net&amp;to=to5%40example.net%2C%20to6%40example.net&amp;foo=IGNORE&amp;bcc=bcc1%40example.net%2C%20bcc2%40example.net&amp;bcc=bcc3%40example.net%2C%20bcc4%40example.net&amp;in-reply-to=<orig1@example.net>\")
+		  (notmuch-fcc-dirs nil))
+	      (notmuch-mua-mailto mailto)
+	      (test-output))"
+cat <<EOF >EXPECTED
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+To: to1@example.net, to2@example.net, to3@example.net, to4@example.net
+Cc: cc1@example.net, cc2@example.net
+Subject: last call for cats & dogs.
+In-Reply-To: <orig1@example.net>
+Bcc: bcc1@example.net, bcc2@example.net
+--text follows this line--
+line1
+line2
+line3
+line4
+Column1A	Column1B	Column1C	Column1D
+√
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-attachments"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment1.gz\""))
-- 
1.7.8.3

[PATCH 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

The new function 'notmuch-mua-mailto' provides an interactive handler
for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
specification: http://tools.ietf.org/html/rfc6068

More decoding of the mailto string needs to be done, as is evident by
the fact that the mailto test remains broken.
---
Unfortunately I'm not sure how best to do the URI decoding, so I've
left a FIXME in the code, and the test as known_broken.  I'm hoping an
elisp/encodings expert out there will pick this up.

 emacs/notmuch-mua.el |   62 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 62 insertions(+), 0 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 023645e..750e8d6 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -24,6 +24,10 @@
 (require 'notmuch-lib)
 (require 'notmuch-address)
 
+(require 'rfc2368)
+(require 'rfc2047)
+(require 'mailheader)
+
 ;;
 
 (defcustom notmuch-mua-send-hook '(notmuch-mua-message-send-hook)
@@ -131,6 +135,64 @@ list."
 
   (message-goto-to))
 
+(defun notmuch-mua-mailto (mailto)
+  "Invoke the notmuch mail composition window for a `mailto:' URI."
+  ;; this should implement implement rfc6068: http://tools.ietf.org/html/rfc6068
+  ;; which obsoleted: http://tools.ietf.org/html/rfc2368
+  ;; this function is based on previous work: http://www.emacswiki.org/emacs/MailtoHandler
+  (interactive)
+  (when (and (stringp mailto)
+	     (string-match "\\`mailto:" mailto))
+    (let* (
+	   ;; FIXME: need to decode all html encodings in uri.
+	   (mailto (replace-regexp-in-string "&" "&" mailto))
+	   (hdr-alist (rfc2368-parse-mailto-url mailto))
+	   to subject other-headers body
+	   (allowed-xtra-hdrs '(cc bcc in-reply-to)))
+
+      (with-temp-buffer
+	;; extract body if it's defined
+	(when (assoc "Body" hdr-alist)
+	  (dolist (hdr hdr-alist)
+	    (when (equal "Body" (car hdr))
+	      (insert (format "%s\n" (cdr hdr)))))
+	  (rfc2047-decode-region (point-min) (point-max))
+	  (setq body (buffer-substring-no-properties
+		      (point-min) (point-max)))
+	  (erase-buffer))
+
+	;; extract headers
+	(dolist (hdr hdr-alist)
+	  (unless (equal "Body" (car hdr))
+	    (insert (format "%s: %s\n" (car hdr) (cdr hdr)))))
+	(rfc2047-decode-region (point-min) (point-max))
+	(goto-char (point-min))
+	(setq hdr-alist (mail-header-extract-no-properties)))
+
+      (setq to (cdr (assoc 'to hdr-alist)))
+      (setq subject (cdr (assoc 'subject hdr-alist)))
+
+      ;; extract allowed other headers, taking only first defined
+      ;; value
+      (dolist (hdr hdr-alist)
+	(if (and (member (car hdr) allowed-xtra-hdrs)
+		 (not (assoc (car hdr) other-headers)))
+	    (add-to-list 'other-headers hdr)))
+
+      (notmuch-mua-mail to subject other-headers)
+
+      ;; insert the message body - but put it in front of the signature
+      ;; if one is present
+      (goto-char (point-max))
+      (if (re-search-backward message-signature-separator nil t)
+	  (forward-line -1)
+	(goto-char (point-max)))
+      (insert body)
+      (push-mark))
+    (set-buffer-modified-p nil)
+
+    (message-goto-body)))
+
 (defun notmuch-mua-mail (&optional to subject other-headers &rest other-args)
   "Invoke the notmuch mail composition window.
 
-- 
1.7.8.3

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[David Edmondson]

[-- Attachment #1: Type: text/plain, Size: 431 bytes --]

On Sun, 29 Jan 2012 11:33:44 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> The new function 'notmuch-mua-mailto' provides an interactive handler
> for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
> specification: http://tools.ietf.org/html/rfc6068

How is this expected to be used? I know generally about how mailto: URIs
are used, but not how they are currently plumbed in with an Emacs MUA.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

On Mon, 30 Jan 2012 15:43:10 +0000, David Edmondson <dme@dme.org> wrote:
> On Sun, 29 Jan 2012 11:33:44 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > The new function 'notmuch-mua-mailto' provides an interactive handler
> > for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
> > specification: http://tools.ietf.org/html/rfc6068
> 
> How is this expected to be used? I know generally about how mailto: URIs
> are used, but not how they are currently plumbed in with an Emacs MUA.

I use a variant of the following as my browser mailto url handler:

  #!/bin/sh
  emacsclient --eval "(notmuch-mua-mailto \"$@\")"

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

On Sun, Jan 29 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> The new function 'notmuch-mua-mailto' provides an interactive handler
> for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
> specification: http://tools.ietf.org/html/rfc6068

I am using and finding this patch useful.  It has no conflict with
anything when not used.  Is there any other objection with pushing it?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jani Nikula]

Jameson Graef Rollins <jrollins@finestructure.net> writes:

> On Sun, Jan 29 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> The new function 'notmuch-mua-mailto' provides an interactive handler
>> for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
>> specification: http://tools.ietf.org/html/rfc6068
>
> I am using and finding this patch useful.  It has no conflict with
> anything when not used.  Is there any other objection with pushing it?

I'm going through the review queue right now, tagging notmuch::stale any
patches that don't apply on master. I was just about to nmbug push stale
on this one (2/2). Is that in error? Care to verify?

BR,
Jani.

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 650 bytes --]

On Sat, Apr 14 2012, Jani Nikula <jani@nikula.org> wrote:
> I'm going through the review queue right now, tagging notmuch::stale any
> patches that don't apply on master. I was just about to nmbug push stale
> on this one (2/2). Is that in error? Care to verify?

Yes, it is an error.  This patch applies to master.  Did you in fact
verify that it doesn't?  You should not be tagging patches stale if they
in fact still apply to master.  If they still need review or something
else that's a different matter.  Please don't mark things as stale if
they are not.

Obviously I still care about this patch, or I wouldn't have re-commented
on it.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jani Nikula]

Jameson Graef Rollins <jrollins@finestructure.net> writes:

> On Sat, Apr 14 2012, Jani Nikula <jani@nikula.org> wrote:
>> I'm going through the review queue right now, tagging notmuch::stale any
>> patches that don't apply on master. I was just about to nmbug push stale
>> on this one (2/2). Is that in error? Care to verify?
>
> Yes, it is an error.  This patch applies to master.  Did you in fact
> verify that it doesn't?  You should not be tagging patches stale if they
> in fact still apply to master.  If they still need review or something
> else that's a different matter.  Please don't mark things as stale if
> they are not.

My nmfirehose script flagged the patch as not applying, and I verified
this (and all of them) by hand, using the emacs interface to save the
patch. No, I'm not intentionally tagging patches stale if they apply,
but I'll stop for now in case there's a bug somewhere (incl. PEBKAC)
breaking the patches and thus making them not apply.

> Obviously I still care about this patch, or I wouldn't have re-commented
> on it.

I was doing this purely based on whether the patches apply or not.


BR,
Jani.

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

On Sat, Apr 14 2012, Jani Nikula <jani@nikula.org> wrote:
> I was doing this purely based on whether the patches apply or not.

I'm so sorry, Jani.  You are correct that this patch requires a small
rebase fix to apply to master.  I'm not sure how I missed it previously.

Updated version on the way.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/2] emacs: new mua mailto: URI handler

[Jani Nikula]

Jameson Graef Rollins <jrollins@finestructure.net> writes:

> On Sat, Apr 14 2012, Jani Nikula <jani@nikula.org> wrote:
>> I was doing this purely based on whether the patches apply or not.
>
> I'm so sorry, Jani.  You are correct that this patch requires a small
> rebase fix to apply to master.  I'm not sure how I missed it previously.

No problem. Thanks for checking again; I was worried we might have a bug
somewhere. :)

Jani.

[PATCH v2 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

The new function 'notmuch-mua-mailto' provides an interactive handler
for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
specification: http://tools.ietf.org/html/rfc6068

More decoding of the mailto string needs to be done, as is evident by
the fact that the mailto test remains broken.
---
Rebased against current master.

 emacs/notmuch-mua.el |   62 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..59b4cf4 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -26,6 +26,10 @@
 (require 'notmuch-lib)
 (require 'notmuch-address)
 
+(require 'rfc2368)
+(require 'rfc2047)
+(require 'mailheader)
+
 (eval-when-compile (require 'cl))
 
 ;;
@@ -198,6 +202,64 @@ list."
 
   (message-goto-to))
 
+(defun notmuch-mua-mailto (mailto)
+  "Invoke the notmuch mail composition window for a `mailto:' URI."
+  ;; this should implement implement rfc6068: http://tools.ietf.org/html/rfc6068
+  ;; which obsoleted: http://tools.ietf.org/html/rfc2368
+  ;; this function is based on previous work: http://www.emacswiki.org/emacs/MailtoHandler
+  (interactive)
+  (when (and (stringp mailto)
+	     (string-match "\\`mailto:" mailto))
+    (let* (
+	   ;; FIXME: need to decode all html encodings in uri.
+	   (mailto (replace-regexp-in-string "&" "&" mailto))
+	   (hdr-alist (rfc2368-parse-mailto-url mailto))
+	   to subject other-headers body
+	   (allowed-xtra-hdrs '(cc bcc in-reply-to)))
+
+      (with-temp-buffer
+	;; extract body if it's defined
+	(when (assoc "Body" hdr-alist)
+	  (dolist (hdr hdr-alist)
+	    (when (equal "Body" (car hdr))
+	      (insert (format "%s\n" (cdr hdr)))))
+	  (rfc2047-decode-region (point-min) (point-max))
+	  (setq body (buffer-substring-no-properties
+		      (point-min) (point-max)))
+	  (erase-buffer))
+
+	;; extract headers
+	(dolist (hdr hdr-alist)
+	  (unless (equal "Body" (car hdr))
+	    (insert (format "%s: %s\n" (car hdr) (cdr hdr)))))
+	(rfc2047-decode-region (point-min) (point-max))
+	(goto-char (point-min))
+	(setq hdr-alist (mail-header-extract-no-properties)))
+
+      (setq to (cdr (assoc 'to hdr-alist)))
+      (setq subject (cdr (assoc 'subject hdr-alist)))
+
+      ;; extract allowed other headers, taking only first defined
+      ;; value
+      (dolist (hdr hdr-alist)
+	(if (and (member (car hdr) allowed-xtra-hdrs)
+		 (not (assoc (car hdr) other-headers)))
+	    (add-to-list 'other-headers hdr)))
+
+      (notmuch-mua-mail to subject other-headers)
+
+      ;; insert the message body - but put it in front of the signature
+      ;; if one is present
+      (goto-char (point-max))
+      (if (re-search-backward message-signature-separator nil t)
+	  (forward-line -1)
+	(goto-char (point-max)))
+      (insert body)
+      (push-mark))
+    (set-buffer-modified-p nil)
+
+    (message-goto-body)))
+
 (defun notmuch-mua-mail (&optional to subject other-headers &rest other-args)
   "Invoke the notmuch mail composition window.
 
-- 
1.7.9.5

Re: [PATCH v2 2/2] emacs: new mua mailto: URI handler

[Mark Walters]

On Sat, 14 Apr 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> The new function 'notmuch-mua-mailto' provides an interactive handler
> for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
> specification: http://tools.ietf.org/html/rfc6068
>
> More decoding of the mailto string needs to be done, as is evident by
> the fact that the mailto test remains broken.
> ---
> Rebased against current master.

Hi I have just been playing with this and it seems to work well. Indeed,
I have not come across  any "real" links  that it fails on. (All the
links on http://www.mailto.co.uk/ seem to work for example.)

What is your experience? If it only fails on contrived examples then I
am much happier about including it.

I only had one comment on the lisp:
 
> +      (notmuch-mua-mail to subject other-headers)

If you make this
   
   (notmuch-mua-mail to subject other-headers nil (notmuch-mua-get-switch-function))

it respects the users preference on where to open the new message.

Best wishes

Mark

Re: [PATCH v2 2/2] emacs: new mua mailto: URI handler

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

On Tue, Jun 19 2012, Mark Walters <markwalters1009@gmail.com> wrote:
> On Sat, 14 Apr 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> The new function 'notmuch-mua-mailto' provides an interactive handler
>> for rfc6068 "mailto:" URIs.  It attempts to implement the rfc6068
>> specification: http://tools.ietf.org/html/rfc6068
>>
>> More decoding of the mailto string needs to be done, as is evident by
>> the fact that the mailto test remains broken.
>> ---
>> Rebased against current master.
>
> Hi I have just been playing with this and it seems to work well. Indeed,
> I have not come across  any "real" links  that it fails on. (All the
> links on http://www.mailto.co.uk/ seem to work for example.)

Cool!  I haven't seen that site before, but it will certainly be useful
for me to extend the tests for this function.

> What is your experience? If it only fails on contrived examples then I
> am much happier about including it.

It's been working fine for me for many months.  I don't click on mailto
links that frequently, but it always seems to work when I do.

> I only had one comment on the lisp:
>  
>> +      (notmuch-mua-mail to subject other-headers)
>
> If you make this
>    
>    (notmuch-mua-mail to subject other-headers nil (notmuch-mua-get-switch-function))
>
> it respects the users preference on where to open the new message.

Great idea!  I'll add this to the next version.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 1/2] test: emacs mailto: URI handling

[Ethan Glasser-Camp]

Jameson Graef Rollins <jrollins@finestructure.net> writes:

> This adds a test for proposed rfc6068 "mailto:" URI handling.  The
> proposed function would be called 'notmuch-mua-mailto'.  The test
> provides an example mailto: string that should test some subset of the
> rfc6068 specification: http://tools.ietf.org/html/rfc6068

Hi! Just going through the patch queue. This test, at least, looks
sane, so why not apply it?

As for the mailto: handler itself, you wrote in
id:"87vcin2fo6.fsf@servo.finestructure.net" that you would add something
to the next version, thereby implying that there would be a next
version. Do you still intend to release such a new version soon? I would
love to review it. This version looks fine to me, with one minor concern.

> +	   ;; FIXME: need to decode all html encodings in uri.
> +	   (mailto (replace-regexp-in-string "&amp;" "&" mailto))

This is odd. Shouldn't the navigator be decoding all that stuff while
parsing, before you even get to it? How often do you get mailto links
with &amp; in them? You obviously wouldn't have put it in there if it
weren't necessary.

I would be +1 on merging this patch or the next version.

Ethan