[BUG] Can't search for some square brackets in From:

[Justin Humm <justin.humm@posteo.de>]

[-- Attachment #1: Type: text/plain, Size: 5563 bytes --]

Hello,

I struggle with searching the From field of mails, that have square brackets
in it. The first instance of that problem I found is

  From: =?UTF-8?Q?Shedhalle_T=c3=bcbingen_[Festival]?= <festival@shedhalle.de>

In this case, on 0.27 I'm unable to find any mail with

  from:"Shedhalle"
  from:"Tübingen"
  from:"Festival"

However I can find mails with 

  From: "dependabot[bot]" <notifications@github.com>

by searching for

  from:"dependabot"
  from:"bot"

I managed to write a short test, that reproduces the problem. It fails on
master, but should pass imho. It is attached below this mail. For me it
generates the following output:


  $ ./T080-search.sh
  
  T080-search: Testing "notmuch search" in several variations
   PASS   Search body
   PASS   Search by from:
   PASS   Search by to:
   PASS   Search by subject:
   PASS   Search by subject (utf-8):
   PASS   Search by id:
   PASS   Search by mid:
   PASS   Search by tag:
   PASS   Search by thread:
   PASS   Search body (phrase)
   PASS   Search by from: (address)
   PASS   Search by from: (name)
   FAIL   Search by from: (name with brackets)
          --- T080-search.13.expected     2018-10-08 22:02:44.157369241 +0000
          +++ T080-search.13.output       2018-10-08 22:02:44.158369260 +0000
          @@ -1 +1 @@
          -thread:XXX   2000-01-01 [1/1] Search By From Name [with brackets]; search by from (name with brackets) (inbox unread)
          +
   PASS   Search by from: (name and address)
   PASS   Search by from: without prefix (name and address)
   PASS   Search by to: (address)
   PASS   Search by to: (name)
   PASS   Search by to: (name and address)
   PASS   Search by to: without prefix (name and address)
   PASS   Search by subject: (phrase)
   FAIL   Search for all messages ("*")
          --- T080-search.21.EXPECTED     2018-10-08 22:02:44.462375008 +0000
          +++ T080-search.21.OUTPUT       2018-10-08 22:02:44.463375027 +0000
          @@ -35,7 +35,7 @@
           thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; negative result (inbox unread)
           thread:XXX   2000-01-01 [1/1] searchbyfrom@example.com; search by from (address) (inbox unread)
           thread:XXX   2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)
          -thread:XXX   2000-01-01 [1/1] Search By From Name [with brackets]; search by from (name with brackets) (inbox unread)
          +thread:XXX   2000-01-01 [1/1] ; search by from (name with brackets) (inbox unread)
           thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread)
           thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)
           thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; subject search test (phrase) (inbox unread)
   PASS   Search body (utf-8):
   PASS   headers do not have adjacent term positions
   PASS   parts have non-overlapping term positions
   PASS   parts do not have adjacent term positions


I could imagine, that the problem occurs not just with square brackets, but
with other special characters as well, but I didn't test for it. Unfortunately,
I have no knowledge of C and notmuch development. If anybody can give me a hint
or a general direction how to fix this, I'd be glad and could try it for myself.

I hope that I formatted this email correctly, never send a git commit via mail before…

Best,
Justin



---
 test/T080-search.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/test/T080-search.sh b/test/T080-search.sh
index a3f0dead..f2d16f74 100755
--- a/test/T080-search.sh
+++ b/test/T080-search.sh
@@ -67,6 +67,11 @@ add_message '[subject]="search by from (name)"' '[date]="Sat, 01 Jan 2000 12:00:
 output=$(notmuch search 'from:"Search By From Name"' | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)"

+test_begin_subtest "Search by from: (name with brackets)"
+add_message '[subject]="search by from (name with brackets)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[from]="Search By From Name [with brackets] <test@example.com>"'
+output=$(notmuch search 'from:"Search By From Name [with brackets]"' | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Search By From Name [with brackets]; search by from (name with brackets) (inbox unread)"
+
 test_begin_subtest "Search by from: (name and address)"
 output=$(notmuch search 'from:"Search By From Name <test@example.com>"' | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)"
@@ -139,6 +144,7 @@ thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; body search (phrase) (inbox un
 thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; negative result (inbox unread)
 thread:XXX   2000-01-01 [1/1] searchbyfrom@example.com; search by from (address) (inbox unread)
 thread:XXX   2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)
+thread:XXX   2000-01-01 [1/1] Search By From Name [with brackets]; search by from (name with brackets) (inbox unread)
 thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread)
 thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)
 thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; subject search test (phrase) (inbox unread)
--
2.18.1



[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEUHHCsJMTrgs+YKQ6XyTjvVZhfrIFAlu72DUACgkQXyTjvVZh
frIlsgf+L37ou8W94JPAz4NOZRoDmTyPYaJGgB6TVoz1OQbioIzQ4Vfjcpb9k0Gx
AdlTnaq/JVzOZ53D3aFCcsD77tK2laazdRf7empbytOA1DfbfCbzpGpSWRQDHzXw
kTvbbxOAhkqG7ynUgxMf6OFo6iJEmZWy6OPPDSPJKGGg7wRw/kpYdwX0ijQ04AgV
idFokdispSd20CYjx7qIjXUwnGGTiU5LqLHMdaUd4v7O0fvFsBdz00XrKSzanA4W
SEQ7UGLBtmMvLiOvHGkaKSP/J6uuU+lBag61xG5VGJA2w9ffoOO34SVWLOEmOMWl
VU/9BJXMHwHmtmeASDBH9kfPsMyb5w==
=4IY/
-----END PGP SIGNATURE-----