* [PATCH 0/2] Better id: link buttonization @ 2012-10-31 2:29 Austin Clements 2012-10-31 2:29 ` [PATCH 1/2] test: Test buttonization of id: links Austin Clements 2012-10-31 2:29 ` [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages Austin Clements 0 siblings, 2 replies; 6+ messages in thread From: Austin Clements @ 2012-10-31 2:29 UTC (permalink / raw) To: notmuch Jani pointed out to me that the new id stashing with less quoting is more likely to result in emails containing id: links that get buttonized wrong. Since id: links without quotes are hip, this series fixes the buttonization regexp to match Xapian query syntax, rather than being ad hoc. ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] test: Test buttonization of id: links 2012-10-31 2:29 [PATCH 0/2] Better id: link buttonization Austin Clements @ 2012-10-31 2:29 ` Austin Clements 2012-10-31 2:29 ` [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages Austin Clements 1 sibling, 0 replies; 6+ messages in thread From: Austin Clements @ 2012-10-31 2:29 UTC (permalink / raw) To: notmuch This matches the current behavior of the buttonizer, so it passes, but many of these cases are not what you'd want (and some of them aren't even valid Xapian queries). The next patch will fix the handling of these cases and update the test. --- test/emacs-show | 40 ++++++++++++++++++++++++++++++++++++++++ test/test-lib.el | 14 ++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/test/emacs-show b/test/emacs-show index 64c38d3..c4f3cf4 100755 --- a/test/emacs-show +++ b/test/emacs-show @@ -106,5 +106,45 @@ test_emacs '(notmuch-search "from:lars@seas.harvard.edu and subject:\"Maildir st (test-visible-output)' test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-indent-thread-content-off +test_begin_subtest "id buttonization" +add_message '[body]=" +id:abc +id:abc.def. id:abc,def, id:abc;def; +id:abc)def +id:ab\"c def +id:\"abc\"def +id:\"ab\"\"c\"def +id:\"ab c\"def +id:\"abc\".def +id:\"abc +\" +id:) +id: +cid:xxx"' +test_emacs '(notmuch-show "id:'$gen_msg_id'") + (notmuch-test-mark-links) + (test-visible-output)' +cat <<EOF >EXPECTED +Notmuch Test Suite <test_suite@notmuchmail.org> (2001-01-05) (inbox) +Subject: id buttonization +To: Notmuch Test Suite <test_suite@notmuchmail.org> +Date: Fri, 05 Jan 2001 15:43:57 +0000 + +<<id:abc>> +<<id:abc.def.>> <<id:abc,def,>> <<id:abc;def;>> +<<id:abc)def>> +<<id:ab>>"c def +<<id:"abc">>def +<<id:"ab">>"c"def +id:"ab c"def +<<id:"abc">>.def +id:"abc +" +<<id:)>> +id: +c<<id:xxx>> +EOF +test_expect_equal_file OUTPUT EXPECTED + test_done diff --git a/test/test-lib.el b/test/test-lib.el index fa3380c..dece811 100644 --- a/test/test-lib.el +++ b/test/test-lib.el @@ -107,6 +107,20 @@ nothing." (ad-set-arg 1 (char-to-string char)) ad-do-it)))) +(defun notmuch-test-mark-links () + "Enclose links in the current buffer with << and >>." + ;; Links are often created by jit-lock functions + (jit-lock-fontify-now) + (save-excursion + (let ((inhibit-read-only t)) + (goto-char (point-min)) + (let ((button)) + (while (setq button (next-button (point))) + (goto-char (button-start button)) + (insert "<<") + (goto-char (button-end button)) + (insert ">>")))))) + (defmacro notmuch-test-run (&rest body) "Evaluate a BODY of test expressions and output the result." `(with-temp-buffer -- 1.7.10 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages 2012-10-31 2:29 [PATCH 0/2] Better id: link buttonization Austin Clements 2012-10-31 2:29 ` [PATCH 1/2] test: Test buttonization of id: links Austin Clements @ 2012-10-31 2:29 ` Austin Clements 2012-10-31 10:00 ` Jani Nikula 2012-11-01 14:01 ` Sascha Silbe 1 sibling, 2 replies; 6+ messages in thread From: Austin Clements @ 2012-10-31 2:29 UTC (permalink / raw) To: notmuch This regexp agrees with Xapian query syntax much more closely, though we specifically disallow various cases that would be confusing in the context of an email body (e.g., punctuation at the end of an id: link is not considered part of the id: link because it's probably part of the surrounding text). In particular, this handles id: links that are not surrounded by quotes much better, which stash is much more likely to generate now that we don't quote id's that don't need to be quoted. It also handles quoted id: links better. We update the buttonization test to reflect the new pattern. --- emacs/notmuch-show.el | 20 +++++++++++++++++++- test/emacs-show | 14 +++++++------- 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index f273eb4..e96e099 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -991,6 +991,24 @@ message at DEPTH in the current thread." "Insert the forest of threads FOREST." (mapc (lambda (thread) (notmuch-show-insert-thread thread 0)) forest)) +(defvar notmuch-id-regexp + (concat + ;; Match the id: prefix only if it begins a word (to disallow, for + ;; example, matching cid:). + "\\<id:\\(" + ;; If the term starts with a ", then parse Xapian's quoted boolean + ;; term syntax, which allows for anything as long as embedded + ;; double quotes escaped by doubling them. We also disallow + ;; newlines (which Xapian allows) to prevent runaway terms. + "\"\\([^\"\n]\\|\"\"\\)*\"" + ;; Otherwise, parse Xapian's unquoted syntax, which goes up to the + ;; next space or ). We disallow [.,;] as the last character + ;; because these are probably part of the surrounding text, and not + ;; part of the id. This doesn't match single character ids; meh. + "\\|[^\"[:space:])][^[:space:])]*[^[:space:]).,;]" + "\\)") + "The regexp used to match id: links in messages.") + (defun notmuch-show-buttonise-links (start end) "Buttonise URLs and mail addresses between START and END. @@ -999,7 +1017,7 @@ a corresponding notmuch search." (goto-address-fontify-region start end) (save-excursion (goto-char start) - (while (re-search-forward "id:\\(\"?\\)[^[:space:]\"]+\\1" end t) + (while (re-search-forward notmuch-id-regexp end t) ;; remove the overlay created by goto-address-mode (remove-overlays (match-beginning 0) (match-end 0) 'goto-address t) (make-text-button (match-beginning 0) (match-end 0) diff --git a/test/emacs-show b/test/emacs-show index c4f3cf4..e16483c 100755 --- a/test/emacs-show +++ b/test/emacs-show @@ -131,18 +131,18 @@ To: Notmuch Test Suite <test_suite@notmuchmail.org> Date: Fri, 05 Jan 2001 15:43:57 +0000 <<id:abc>> -<<id:abc.def.>> <<id:abc,def,>> <<id:abc;def;>> -<<id:abc)def>> -<<id:ab>>"c def +<<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; +<<id:abc>>)def +<<id:ab"c>> def <<id:"abc">>def -<<id:"ab">>"c"def -id:"ab c"def +<<id:"ab""c">>def +<<id:"ab c">>def <<id:"abc">>.def id:"abc " -<<id:)>> +id:) id: -c<<id:xxx>> +cid:xxx EOF test_expect_equal_file OUTPUT EXPECTED -- 1.7.10 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages 2012-10-31 2:29 ` [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages Austin Clements @ 2012-10-31 10:00 ` Jani Nikula 2012-11-01 14:01 ` Sascha Silbe 1 sibling, 0 replies; 6+ messages in thread From: Jani Nikula @ 2012-10-31 10:00 UTC (permalink / raw) To: Austin Clements, notmuch On Wed, 31 Oct 2012, Austin Clements <amdragon@MIT.EDU> wrote: > This regexp agrees with Xapian query syntax much more closely, though > we specifically disallow various cases that would be confusing in the > context of an email body (e.g., punctuation at the end of an id: link > is not considered part of the id: link because it's probably part of > the surrounding text). > > In particular, this handles id: links that are not surrounded by > quotes much better, which stash is much more likely to generate now > that we don't quote id's that don't need to be quoted. It also > handles quoted id: links better. > > We update the buttonization test to reflect the new pattern. Hi Austin, all of this looks good as-is, but I propose the changes below on top. (With the relevant comment changed too.) BR, Jani. diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index e96e099..117eb0e 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -1005,7 +1005,7 @@ message at DEPTH in the current thread." ;; next space or ). We disallow [.,;] as the last character ;; because these are probably part of the surrounding text, and not ;; part of the id. This doesn't match single character ids; meh. - "\\|[^\"[:space:])][^[:space:])]*[^[:space:]).,;]" + "\\|[^\"[:space:])][^[:space:])]*[^])[:space:].,:;?!]" "\\)") "The regexp used to match id: links in messages.") diff --git a/test/emacs-show b/test/emacs-show index e16483c..e2d7c70 100755 --- a/test/emacs-show +++ b/test/emacs-show @@ -109,7 +109,12 @@ test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-indent-thread-content-off test_begin_subtest "id buttonization" add_message '[body]=" id:abc -id:abc.def. id:abc,def, id:abc;def; +id:abc.def. id:abc,def, id:abc;def; id:abc:def: +id:foo@bar.?baz? id:foo@bar!.baz! +(id:foo@bar.baz) [id:foo@bar.baz] +id:foo@bar.baz... +id:2+2=5 +id:=_-:/.[]@$%+ id:abc)def id:ab\"c def id:\"abc\"def @@ -131,7 +136,12 @@ To: Notmuch Test Suite <test_suite@notmuchmail.org> Date: Fri, 05 Jan 2001 15:43:57 +0000 <<id:abc>> -<<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; +<<id:abc.def>>. <<id:abc,def>>, <<id:abc;def>>; <<id:abc:def>>: +<<id:foo@bar.?baz>>? <<id:foo@bar!.baz>>! +(<<id:foo@bar.baz>>) [<<id:foo@bar.baz>>] +<<id:foo@bar.baz>>... +<<id:2+2=5>> +<<id:=_-:/.[]@$%+>> <<id:abc>>)def <<id:ab"c>> def <<id:"abc">>def ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages 2012-10-31 2:29 ` [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages Austin Clements 2012-10-31 10:00 ` Jani Nikula @ 2012-11-01 14:01 ` Sascha Silbe 2012-11-02 19:27 ` Jameson Graef Rollins 1 sibling, 1 reply; 6+ messages in thread From: Sascha Silbe @ 2012-11-01 14:01 UTC (permalink / raw) To: Austin Clements, notmuch [-- Attachment #1: Type: text/plain, Size: 559 bytes --] Austin Clements <amdragon@MIT.EDU> writes: > +(defvar notmuch-id-regexp > + (concat > + ;; Match the id: prefix only if it begins a word (to disallow, for > + ;; example, matching cid:). > + "\\<id:\\(" [...] Since you're already at it, please extend it to accept RFC2392 [1] syntax for message URLs: mid-url = "mid" ":" message-id [ "/" content-id ] The id: syntax is non-standard (i.e. specific to notmuch). Sascha [1] http://www.ietf.org/rfc/rfc2392.txt -- http://sascha.silbe.org/ http://www.infra-silbe.de/ [-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages 2012-11-01 14:01 ` Sascha Silbe @ 2012-11-02 19:27 ` Jameson Graef Rollins 0 siblings, 0 replies; 6+ messages in thread From: Jameson Graef Rollins @ 2012-11-02 19:27 UTC (permalink / raw) To: Sascha Silbe, Austin Clements, notmuch [-- Attachment #1: Type: text/plain, Size: 625 bytes --] On Thu, Nov 01 2012, Sascha Silbe <sascha-ml-reply-to-2012-4@silbe.org> wrote: > Since you're already at it, please extend it to accept RFC2392 [1] > syntax for message URLs: > > mid-url = "mid" ":" message-id [ "/" content-id ] > > The id: syntax is non-standard (i.e. specific to notmuch). > > Sascha > > [1] http://www.ietf.org/rfc/rfc2392.txt Holy crap! There's an RFC that specifies a URL scheme for message-ids?? From 1998?? Very cool. We definitely need to support that. Does anything else support this? I don't see any reason we couldn't support the "cid" schemes as well. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-11-02 19:27 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-10-31 2:29 [PATCH 0/2] Better id: link buttonization Austin Clements 2012-10-31 2:29 ` [PATCH 1/2] test: Test buttonization of id: links Austin Clements 2012-10-31 2:29 ` [PATCH 2/2] emacs: Improve the regexp used to match id:'s in messages Austin Clements 2012-10-31 10:00 ` Jani Nikula 2012-11-01 14:01 ` Sascha Silbe 2012-11-02 19:27 ` Jameson Graef Rollins
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).