notmuch-search not excluding excluded tags

notmuch-search not excluding excluded tags

[Richard Stanton]

I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command

notmuch search tag:unread

I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:

(notmuch-search “tag:unread”)

I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?

Thanks.

Richard Stanton
\r

Re: notmuch-search not excluding excluded tags

[David Bremner]

Richard Stanton <rhstanton@berkeley.edu> writes:

> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>
> notmuch search tag:unread
>
> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>
> (notmuch-search “tag:unread”)
>
> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?

Hi Richard;

As for as I know it _should_ work (modulo the curly quotes being invalid
syntax). I don't have any real hypothesis for what is going wrong, but a
few ideas for gather data.

1) As a start, maybe try evaluating

        (notmuch-config-get "search.exclude_tags")

in emacs and make sure it matches the corresponding

   notmuch config get search.exclude_tags

2) If possible, run notmuch with a minimal configuration (i.e. just loading
notmuch, no personal configuration or other packages). There is a script
./devel/try-emacs-mua in the source if you have a self-built notmuch.\r

Re: notmuch-search not excluding excluded tags

[Richard H. Stanton]

Thanks, David.

It all seems to be working fine on my work machine! 

I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.

> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
> 
> Richard Stanton <rhstanton@berkeley.edu> writes:
> 
>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>> 
>> notmuch search tag:unread
>> 
>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>> 
>> (notmuch-search “tag:unread”)
>> 
>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
> 
> Hi Richard;
> 
> As for as I know it _should_ work (modulo the curly quotes being invalid
> syntax). I don't have any real hypothesis for what is going wrong, but a
> few ideas for gather data.
> 
> 1) As a start, maybe try evaluating
> 
>        (notmuch-config-get "search.exclude_tags")
> 
> in emacs and make sure it matches the corresponding
> 
>   notmuch config get search.exclude_tags
> 
> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
> notmuch, no personal configuration or other packages). There is a script
> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
\r

Re: notmuch-search not excluding excluded tags

[Richard H. Stanton]

Oh, wait… It's not working on my office machine either. I’ll start tracing things and see what happens.

By the way,

(notmuch-config-get "search.exclude_tags”)

returns

"spam
trash"

which is what it should be returning.

> On Apr 16, 2024, at 12:00 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
> 
> Thanks, David.
> 
> It all seems to be working fine on my work machine! 
> 
> I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.
> 
>> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
>> 
>> Richard Stanton <rhstanton@berkeley.edu> writes:
>> 
>>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>>> 
>>> notmuch search tag:unread
>>> 
>>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>>> 
>>> (notmuch-search “tag:unread”)
>>> 
>>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
>> 
>> Hi Richard;
>> 
>> As for as I know it _should_ work (modulo the curly quotes being invalid
>> syntax). I don't have any real hypothesis for what is going wrong, but a
>> few ideas for gather data.
>> 
>> 1) As a start, maybe try evaluating
>> 
>>       (notmuch-config-get "search.exclude_tags")
>> 
>> in emacs and make sure it matches the corresponding
>> 
>>  notmuch config get search.exclude_tags
>> 
>> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
>> notmuch, no personal configuration or other packages). There is a script
>> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
> 
\r

Re: notmuch-search not excluding excluded tags

[Richard H. Stanton]

Running 

(notmuch-search "tag:unread" t t)

gives the correct results, so it looks like notmuch-search-hide-excluded is not getting set to the correct default value.

> On Apr 16, 2024, at 3:15 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
> 
> Oh, wait… It's not working on my office machine either. I’ll start tracing things and see what happens.
> 
> By the way,
> 
> (notmuch-config-get "search.exclude_tags”)
> 
> returns
> 
> "spam
> trash"
> 
> which is what it should be returning.
> 
>> On Apr 16, 2024, at 12:00 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>> 
>> Thanks, David.
>> 
>> It all seems to be working fine on my work machine! 
>> 
>> I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.
>> 
>>> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
>>> 
>>> Richard Stanton <rhstanton@berkeley.edu> writes:
>>> 
>>>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>>>> 
>>>> notmuch search tag:unread
>>>> 
>>>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>>>> 
>>>> (notmuch-search “tag:unread”)
>>>> 
>>>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
>>> 
>>> Hi Richard;
>>> 
>>> As for as I know it _should_ work (modulo the curly quotes being invalid
>>> syntax). I don't have any real hypothesis for what is going wrong, but a
>>> few ideas for gather data.
>>> 
>>> 1) As a start, maybe try evaluating
>>> 
>>>      (notmuch-config-get "search.exclude_tags")
>>> 
>>> in emacs and make sure it matches the corresponding
>>> 
>>> notmuch config get search.exclude_tags
>>> 
>>> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
>>> notmuch, no personal configuration or other packages). There is a script
>>> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
>> 
> 
\r

Re: notmuch-search not excluding excluded tags

[Richard H. Stanton]

I used edebug to trace through the execution of notmuch-search in notmuch.el.

At line 1096, we have the code

	(let ((proc (notmuch-start-notmuch
		     "notmuch-search" buffer #'notmuch-search-process-sentinel
		     "search" "--format=sexp" "--format-version=5"
		     (if oldest-first
			 "--sort=oldest-first"
		       "--sort=newest-first")
		     (if hide-excluded
			 "--exclude=true"
		       "--exclude=false")
		     query)))

I think the problem is the line

(If hide-excluded

This is the (optional) argument passed to notmuch-search. If it was not passed, this test fails and “—exclude=false” gets selected. I think we’re testing the wrong variable here. It looks like it should be 

(If  notmuch-search-hide-excluded

instead, since this variable is set to its default value if no argument is passed to notmuch-search.

[Similarly with the preceding test for oldest-first, I think.]





> On Apr 16, 2024, at 3:22 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
> 
> Running 
> 
> (notmuch-search "tag:unread" t t)
> 
> gives the correct results, so it looks like notmuch-search-hide-excluded is not getting set to the correct default value.
> 
>> On Apr 16, 2024, at 3:15 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>> 
>> Oh, wait… It's not working on my office machine either. I’ll start tracing things and see what happens.
>> 
>> By the way,
>> 
>> (notmuch-config-get "search.exclude_tags”)
>> 
>> returns
>> 
>> "spam
>> trash"
>> 
>> which is what it should be returning.
>> 
>>> On Apr 16, 2024, at 12:00 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>> 
>>> Thanks, David.
>>> 
>>> It all seems to be working fine on my work machine! 
>>> 
>>> I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.
>>> 
>>>> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
>>>> 
>>>> Richard Stanton <rhstanton@berkeley.edu> writes:
>>>> 
>>>>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>>>>> 
>>>>> notmuch search tag:unread
>>>>> 
>>>>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>>>>> 
>>>>> (notmuch-search “tag:unread”)
>>>>> 
>>>>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
>>>> 
>>>> Hi Richard;
>>>> 
>>>> As for as I know it _should_ work (modulo the curly quotes being invalid
>>>> syntax). I don't have any real hypothesis for what is going wrong, but a
>>>> few ideas for gather data.
>>>> 
>>>> 1) As a start, maybe try evaluating
>>>> 
>>>>     (notmuch-config-get "search.exclude_tags")
>>>> 
>>>> in emacs and make sure it matches the corresponding
>>>> 
>>>> notmuch config get search.exclude_tags
>>>> 
>>>> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
>>>> notmuch, no personal configuration or other packages). There is a script
>>>> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
>>> 
>> 
> 
\r

Re: notmuch-search not excluding excluded tags

[Richard H. Stanton]

Actually, I think the code might be working as intended. Calling notmuch-search interactively things work just fine, and in the comments I see the note 

"When called interactively, this will prompt for a query and use the configured default sort order.”

This sounds like it deliberately ignores the default settings when the function is called non-interactively, but why…?





> On Apr 16, 2024, at 3:40 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
> 
> I used edebug to trace through the execution of notmuch-search in notmuch.el.
> 
> At line 1096, we have the code
> 
> (let ((proc (notmuch-start-notmuch
>     "notmuch-search" buffer #'notmuch-search-process-sentinel
>     "search" "--format=sexp" "--format-version=5"
>     (if oldest-first
> "--sort=oldest-first"
>       "--sort=newest-first")
>     (if hide-excluded
> "--exclude=true"
>       "--exclude=false")
>     query)))
> 
> I think the problem is the line
> 
> (If hide-excluded
> 
> This is the (optional) argument passed to notmuch-search. If it was not passed, this test fails and “—exclude=false” gets selected. I think we’re testing the wrong variable here. It looks like it should be 
> 
> (If  notmuch-search-hide-excluded
> 
> instead, since this variable is set to its default value if no argument is passed to notmuch-search.
> 
> [Similarly with the preceding test for oldest-first, I think.]
> 
> 
> 
> 
> 
>> On Apr 16, 2024, at 3:22 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>> 
>> Running 
>> 
>> (notmuch-search "tag:unread" t t)
>> 
>> gives the correct results, so it looks like notmuch-search-hide-excluded is not getting set to the correct default value.
>> 
>>> On Apr 16, 2024, at 3:15 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>> 
>>> Oh, wait… It's not working on my office machine either. I’ll start tracing things and see what happens.
>>> 
>>> By the way,
>>> 
>>> (notmuch-config-get "search.exclude_tags”)
>>> 
>>> returns
>>> 
>>> "spam
>>> trash"
>>> 
>>> which is what it should be returning.
>>> 
>>>> On Apr 16, 2024, at 12:00 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>>> 
>>>> Thanks, David.
>>>> 
>>>> It all seems to be working fine on my work machine! 
>>>> 
>>>> I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.
>>>> 
>>>>> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
>>>>> 
>>>>> Richard Stanton <rhstanton@berkeley.edu> writes:
>>>>> 
>>>>>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>>>>>> 
>>>>>> notmuch search tag:unread
>>>>>> 
>>>>>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>>>>>> 
>>>>>> (notmuch-search “tag:unread”)
>>>>>> 
>>>>>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
>>>>> 
>>>>> Hi Richard;
>>>>> 
>>>>> As for as I know it _should_ work (modulo the curly quotes being invalid
>>>>> syntax). I don't have any real hypothesis for what is going wrong, but a
>>>>> few ideas for gather data.
>>>>> 
>>>>> 1) As a start, maybe try evaluating
>>>>> 
>>>>>    (notmuch-config-get "search.exclude_tags")
>>>>> 
>>>>> in emacs and make sure it matches the corresponding
>>>>> 
>>>>> notmuch config get search.exclude_tags
>>>>> 
>>>>> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
>>>>> notmuch, no personal configuration or other packages). There is a script
>>>>> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
>>>> 
>>> 
>> 
> 
\r

Re: notmuch-search not excluding excluded tags

[Richard Stanton]

In case anyone else is affected by this, the problem I was having was that notmuch-search org links (defined in ol-notmuch.el) were returning results that included spam and trash messages even though I had asked to exclude these. To get around this, redefine org-notmuch-search-open to explicitly pass some additional options to notmuch-search:

(defun org-notmuch-search-open (path _)
  "Follow a notmuch search link specified by PATH."
  (notmuch-search path notmuch-search-oldest-first notmuch-search-hide-excluded))


> On Apr 16, 2024, at 3:53 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
> 
> Actually, I think the code might be working as intended. Calling notmuch-search interactively things work just fine, and in the comments I see the note 
> 
> "When called interactively, this will prompt for a query and use the configured default sort order.”
> 
> This sounds like it deliberately ignores the default settings when the function is called non-interactively, but why…?
> 
> 
> 
> 
> 
>> On Apr 16, 2024, at 3:40 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>> 
>> I used edebug to trace through the execution of notmuch-search in notmuch.el.
>> 
>> At line 1096, we have the code
>> 
>> (let ((proc (notmuch-start-notmuch
>>    "notmuch-search" buffer #'notmuch-search-process-sentinel
>>    "search" "--format=sexp" "--format-version=5"
>>    (if oldest-first
>> "--sort=oldest-first"
>>      "--sort=newest-first")
>>    (if hide-excluded
>> "--exclude=true"
>>      "--exclude=false")
>>    query)))
>> 
>> I think the problem is the line
>> 
>> (If hide-excluded
>> 
>> This is the (optional) argument passed to notmuch-search. If it was not passed, this test fails and “—exclude=false” gets selected. I think we’re testing the wrong variable here. It looks like it should be 
>> 
>> (If  notmuch-search-hide-excluded
>> 
>> instead, since this variable is set to its default value if no argument is passed to notmuch-search.
>> 
>> [Similarly with the preceding test for oldest-first, I think.]
>> 
>> 
>> 
>> 
>> 
>>> On Apr 16, 2024, at 3:22 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>> 
>>> Running 
>>> 
>>> (notmuch-search "tag:unread" t t)
>>> 
>>> gives the correct results, so it looks like notmuch-search-hide-excluded is not getting set to the correct default value.
>>> 
>>>> On Apr 16, 2024, at 3:15 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>>> 
>>>> Oh, wait… It's not working on my office machine either. I’ll start tracing things and see what happens.
>>>> 
>>>> By the way,
>>>> 
>>>> (notmuch-config-get "search.exclude_tags”)
>>>> 
>>>> returns
>>>> 
>>>> "spam
>>>> trash"
>>>> 
>>>> which is what it should be returning.
>>>> 
>>>>> On Apr 16, 2024, at 12:00 PM, Richard H. Stanton <rhstanton@berkeley.edu> wrote:
>>>>> 
>>>>> Thanks, David.
>>>>> 
>>>>> It all seems to be working fine on my work machine! 
>>>>> 
>>>>> I can’t decide if that’s good or bad news… I suspect it’ll take me longer to track down what’s going on than if the behavior were consistent on my two machines.
>>>>> 
>>>>>> On Apr 16, 2024, at 7:23 AM, David Bremner <david@tethera.net> wrote:
>>>>>> 
>>>>>> Richard Stanton <rhstanton@berkeley.edu> writes:
>>>>>> 
>>>>>>> I have spam and trash defined as excluded tags for notmuch searches and when I run (at the command line) the command
>>>>>>> 
>>>>>>> notmuch search tag:unread
>>>>>>> 
>>>>>>> I get a list of unread messages that does *not* include unread spam or trash emails. But when I put the following into my *scratch* buffer and execute it:
>>>>>>> 
>>>>>>> (notmuch-search “tag:unread”)
>>>>>>> 
>>>>>>> I now get ALL unread messages, including those tagged as spam or trash. Am I missing something?
>>>>>> 
>>>>>> Hi Richard;
>>>>>> 
>>>>>> As for as I know it _should_ work (modulo the curly quotes being invalid
>>>>>> syntax). I don't have any real hypothesis for what is going wrong, but a
>>>>>> few ideas for gather data.
>>>>>> 
>>>>>> 1) As a start, maybe try evaluating
>>>>>> 
>>>>>>   (notmuch-config-get "search.exclude_tags")
>>>>>> 
>>>>>> in emacs and make sure it matches the corresponding
>>>>>> 
>>>>>> notmuch config get search.exclude_tags
>>>>>> 
>>>>>> 2) If possible, run notmuch with a minimal configuration (i.e. just loading
>>>>>> notmuch, no personal configuration or other packages). There is a script
>>>>>> ./devel/try-emacs-mua in the source if you have a self-built notmuch.
>>>>> 
>>>> 
>>> 
>> 
> 
\r