Query emails sent to undisclosed-recipients

Query emails sent to undisclosed-recipients

[Firmin Martin]

Hi,

I have emails whose the "To" field is undisclosed recipients. In JSON:

```
"To": "undisclosed-recipients: ;"
```

I would want to tag such email as spam, but I can't query them
using 

```
 notmuch show --format=json to:"undisclosed-recipients: ;"
```

or any variation (regex etc.).

This question has already been addressed in 2013 [1]. Are there any plan
to implement this feature or available workaround ?

Thanks,

Firmin Martin

[1] https://notmuchmail.org/pipermail/notmuch/2013/015516.html

Re: Query emails sent to undisclosed-recipients

[Tomi Ollila]

On Tue, Mar 23 2021, Firmin Martin wrote:

> Hi,
>
> I have emails whose the "To" field is undisclosed recipients. In JSON:
>
> ```
> "To": "undisclosed-recipients: ;"
> ```
>
> I would want to tag such email as spam, but I can't query them
> using 
>
> ```
>  notmuch show --format=json to:"undisclosed-recipients: ;"
> ```
>
> or any variation (regex etc.).
>
> This question has already been addressed in 2013 [1]. Are there any plan
> to implement this feature or available workaround ?

Tried. many things. did not work. notmuch-search-terms(7) tells

     to:<name-or-address>

(so no regex syntax...)

I don't know why that doesn't work. IIRC no plan, but patches welcome >;D

Tomi

PS: I tried

    1  20:21  0:00  notmuch search to:undisclosed-recipients
    2  20:21  0:00  notmuch search to:/undisclosed-recipients/
    6  20:22  0:00  notmuch search id:msg-w-ur@not.an.example
    9  20:23  0:00  notmuch search 'to:undisclosed*'
   10  20:23  0:00  notmuch search 'to:undisclosed'
   11  20:23  0:17  notmuch search 'to:tomi.ollila'
   12  20:24  0:01  notmuch search 'to:/undisclosed/'
   13  20:24  0:00  notmuch search 'to:/undisclosed*/'
   14  20:24  0:00  notmuch search 'to:/undisclosed.*/'
   15  20:24  0:00  notmuch search 'to:/.*undisclosed.*/'
   16  20:25  0:14  notmuch help search
   17  20:25  0:00  notmuch help notmuch-search-terms
   18  20:25  0:02  notmuch help search
   19  20:25  0:00  notmuch help notmuch-search-terms
   20  20:25  0:07  notmuch help search
   21  20:25  0:53  notmuch help search-terms
   22  20:26  0:00  notmuch search 'to:undisclosed-recipients:'
   23  20:26  0:00  notmuch search 'to:undisclosed-recipients'
   24  20:27  0:00  notmuch search 'to:tomi.oll*'
   25  20:27  0:03  notmuch search 'to:tomi.'
   26  20:27  0:02  notmuch search 'to:tomi.*'
   27  20:27  0:00  notmuch search 'to:tomi.o*'
   28  20:27  0:00  notmuch search 'to:undisclosed-recipients:'
   29  20:28  1:35  notmuch help search-terms
   30  20:30  0:00  notmuch search 'to:undisclosed-recipients:;'

>
> Thanks,
>
> Firmin Martin
>
> [1] https://notmuchmail.org/pipermail/notmuch/2013/015516.html

Re: Query emails sent to undisclosed-recipients

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Tue, Mar 23 2021, Firmin Martin wrote:
>
>> Hi,
>>
>> I have emails whose the "To" field is undisclosed recipients. In JSON:
>>
>> ```
>> "To": "undisclosed-recipients: ;"
>> ```
>>
>> I would want to tag such email as spam, but I can't query them
>> using 
>>
>> ```
>>  notmuch show --format=json to:"undisclosed-recipients: ;"
>> ```
>>
>> or any variation (regex etc.).
>>
>> This question has already been addressed in 2013 [1]. Are there any plan
>> to implement this feature or available workaround ?
>
> Tried. many things. did not work. notmuch-search-terms(7) tells
>
>      to:<name-or-address>
>
> (so no regex syntax...)
>
> I don't know why that doesn't work. IIRC no plan, but patches welcome >;D

The (light) technical background is that regex syntax in notmuch
requires value slots, and someone (TM) would need to evaluate how much
adding a value slot for to: would cost in terms of database size / speed
of queries.

I think there's a separate question about address groups being ignored,
discussed in the linked thread.

d

Re: Query emails sent to undisclosed-recipients

[Tomi Ollila]

On Tue, Mar 23 2021, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>
>> On Tue, Mar 23 2021, Firmin Martin wrote:
>>
>>> Hi,
>>>
>>> I have emails whose the "To" field is undisclosed recipients. In JSON:
>>>
>>> ```
>>> "To": "undisclosed-recipients: ;"
>>> ```
>>>
>>> I would want to tag such email as spam, but I can't query them
>>> using 
>>>
>>> ```
>>>  notmuch show --format=json to:"undisclosed-recipients: ;"
>>> ```
>>>
>>> or any variation (regex etc.).
>>>
>>> This question has already been addressed in 2013 [1]. Are there any plan
>>> to implement this feature or available workaround ?
>>
>> Tried. many things. did not work. notmuch-search-terms(7) tells
>>
>>      to:<name-or-address>
>>
>> (so no regex syntax...)
>>
>> I don't know why that doesn't work. IIRC no plan, but patches welcome >;D
>
> The (light) technical background is that regex syntax in notmuch
> requires value slots, and someone (TM) would need to evaluate how much
> adding a value slot for to: would cost in terms of database size / speed
> of queries.
>
> I think there's a separate question about address groups being ignored,
> discussed in the linked thread.

But the question if why doesn't to:undisclosed-recipients:
or to:undisclosed-recipients work

>
> d

[PATCH] test: add known broken test for indexing RFC822 group names

[David Bremner]

Austin Clements diagnosed this indexing problem in [1].

[1]: id:20130711215207.GR2214@mit.edu
---

Hi Tomi;

Here's a test that demonstrates the bug / missing feature.


 test/T050-new.sh | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/test/T050-new.sh b/test/T050-new.sh
index 2985e24c..109ca4ef 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -339,6 +339,13 @@ test_expect_code 1 "NOTMUCH_NEW --debug 2>&1"
 
 notmuch config set new.tags $OLDCONFIG
 
+test_begin_subtest "RFC822 group names are indexed"
+test_subtest_known_broken
+generate_message [to]="undisclosed-recipients:"
+NOTMUCH_NEW > OUTPUT
+output=$(notmuch search --output=messages to:undisclosed-recipients)
+test_expect_equal "${output}" "${gen_msg_id}"
+
 test_begin_subtest "Long directory names don't cause rescan"
 test_subtest_known_broken
 printf -v name 'z%.0s' {1..234}
-- 
2.30.2

Re: [PATCH] test: add known broken test for indexing RFC822 group names

[David Bremner]

David Bremner <david@tethera.net> writes:

> Austin Clements diagnosed this indexing problem in [1].
>
> [1]: id:20130711215207.GR2214@mit.edu

BTW, I followed Austin's suggestion in the linked message, and confirmed
that the database has no XTO terms for the test message.

d

Re: Query emails sent to undisclosed-recipients

[NeilBrown]

[-- Attachment #1.1: Type: text/plain, Size: 2543 bytes --]

On Tue, Mar 23 2021, Tomi Ollila wrote:

> On Tue, Mar 23 2021, David Bremner wrote:
>
>> Tomi Ollila <tomi.ollila@iki.fi> writes:
>>
>>> On Tue, Mar 23 2021, Firmin Martin wrote:
>>>
>>>> Hi,
>>>>
>>>> I have emails whose the "To" field is undisclosed recipients. In JSON:
>>>>
>>>> ```
>>>> "To": "undisclosed-recipients: ;"
>>>> ```
>>>>
>>>> I would want to tag such email as spam, but I can't query them
>>>> using 
>>>>
>>>> ```
>>>>  notmuch show --format=json to:"undisclosed-recipients: ;"
>>>> ```
>>>>
>>>> or any variation (regex etc.).
>>>>
>>>> This question has already been addressed in 2013 [1]. Are there any plan
>>>> to implement this feature or available workaround ?
>>>
>>> Tried. many things. did not work. notmuch-search-terms(7) tells
>>>
>>>      to:<name-or-address>
>>>
>>> (so no regex syntax...)
>>>
>>> I don't know why that doesn't work. IIRC no plan, but patches welcome >;D
>>
>> The (light) technical background is that regex syntax in notmuch
>> requires value slots, and someone (TM) would need to evaluate how much
>> adding a value slot for to: would cost in terms of database size / speed
>> of queries.
>>
>> I think there's a separate question about address groups being ignored,
>> discussed in the linked thread.
>
> But the question if why doesn't to:undisclosed-recipients:
> or to:undisclosed-recipients work

Because "undisclosed-recipient:" is not an address or a comment (in
RFC822 / RFC5322 syntax).  It is a label (a name for a group of addresses).
It is not syntactically valid to have an empty "to:" field, or to have
no "to:" field.  The only valid syntax which doesn't actually give any
address is "label:;".

These messages don't actually have any "to" address.
So
   notmuch search "not to:*"
should work... except that it doesn't.

    notmuch search --output=files "not (to:a* OR to:b* OR to:c* OR to:d* \
    OR to:e* OR to:f* OR to:g* OR to:h* OR  to:i* OR to:j* OR to:k* \
    OR \to:l* OR to:m* OR to:n* OR to:o* OR to:p* OR to:q* OR to:r* \
    OR to:s* OR to:t* OR to:u* OR to:v* OR to:w* OR to:x* OR to:y* OR to:z*)"

does work (as long as no addressed start with a non-alpha character).

I piped the above in
    xargs grep -i '^to:' | grep -v -i ': *;'

Some of the matches had an empty 'to:' which is syntactically invalid.
Others had "<>" as the address.  I don't think this is legal, but I've
seen it used in Return-path: a lot.  RFC5322 doesn't mention it.
The rest was in the noise.

NeilBrown

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH] test: add known broken test for indexing RFC822 group names

[David Bremner]

David Bremner <david@tethera.net> writes:

> Austin Clements diagnosed this indexing problem in [1].
>
> [1]: id:20130711215207.GR2214@mit.edu

Applied to master.

d