notmuch_query_add_tag_excluded and notmuch_query_search_threads

notmuch_query_add_tag_excluded and notmuch_query_search_threads

[Gaute Hope]

hi,

I can't quite figure out how to make excluded threads show up in
searches explicitly including the tag.

If I add the 'muted' tag to all messages in a thread, and set up my
query as follows:


    notmuch_query_t * query = notmuch_query_create (nm_db,
    "tag:muted");
    notmuch_query_add_tag_exclude (query, "muted");
    notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_TRUE);
    threads = notmuch_query_search_threads (query);

I would expect to have the thread matching, I get nothing however. It
does show up in: $ notmuch search tag:muted (muted added to excluded
tags), but does notmuch search operate with
notmuch_query_search_messages? 

Regards, Gaute

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[David Bremner]

Gaute Hope <eg@gaute.vetsj.com> writes:

>
> I would expect to have the thread matching, I get nothing however. It
> does show up in: $ notmuch search tag:muted (muted added to excluded
> tags), but does notmuch search operate with
> notmuch_query_search_messages? 
>

I expanded your sample into a full program, and it seems to work ok for
me. Not sure what the difference might be.

#include <notmuch.h>
#include <stdlib.h>
#include <stdio.h>
int main(int argc, char **argv){

  notmuch_database_t *nm_db;
  notmuch_query_t *query;
  notmuch_threads_t *threads;
  notmuch_thread_t *thread;

  if (notmuch_database_open ("/home/bremner/Maildir/",
			     NOTMUCH_DATABASE_MODE_READ_ONLY, &nm_db))
    return 1;


  
  query = notmuch_query_create (nm_db, "tag:muted");
  notmuch_query_add_tag_exclude (query, "muted");
  notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_TRUE);

  for (threads = notmuch_query_search_threads (query);
       notmuch_threads_valid (threads);
       notmuch_threads_move_to_next (threads))
    {
      thread = notmuch_threads_get (threads);
      printf("%d\n", notmuch_thread_get_total_messages(thread));
      notmuch_thread_destroy (thread);
    }

}

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[Gaute Hope]

Excerpts from David Bremner's message of December 7, 2014 12:52:
> Gaute Hope <eg@gaute.vetsj.com> writes:
> 
>>
>> I would expect to have the thread matching, I get nothing however. It
>> does show up in: $ notmuch search tag:muted (muted added to excluded
>> tags), but does notmuch search operate with
>> notmuch_query_search_messages? 
>>
> 
> I expanded your sample into a full program, and it seems to work ok for
> me. Not sure what the difference might be.
> 
> [...]
>   query = notmuch_query_create (nm_db, "tag:muted");
>   notmuch_query_add_tag_exclude (query, "muted");
>   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_TRUE);

  printf("%d\n", notmuch_query_count_threads (query));

> 
>   for (threads = notmuch_query_search_threads (query);
>        notmuch_threads_valid (threads);
>        notmuch_threads_move_to_next (threads))
> 
> [...]

Thanks,

I also did an notmuch_query_count_threads (query) as illustrated
above, this returns 0 and also results in the erroneous behavior below. Your code works as expected without the _count_threads() call.

regards, gaute

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[David Bremner]

Gaute Hope <eg@gaute.vetsj.com> writes:

>
> I also did an notmuch_query_count_threads (query) as illustrated
> above, this returns 0 and also results in the erroneous behavior
> below. Your code works as expected without the _count_threads() call.

I think there's a documentation error; notmuch_query_count_threads is
destructive. As to why your's returns 0 that's still a mystery to me. I
get 84, which matches the output from "notmuch search".

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[Gaute Hope]

Excerpts from David Bremner's message of December 7, 2014 14:54:
> Gaute Hope <eg@gaute.vetsj.com> writes:
> 
>>
>> I also did an notmuch_query_count_threads (query) as illustrated
>> above, this returns 0 and also results in the erroneous behavior
>> below. Your code works as expected without the _count_threads() call.
> 
> I think there's a documentation error; notmuch_query_count_threads is
> destructive. As to why your's returns 0 that's still a mystery to me. I
> get 84, which matches the output from "notmuch search".
> 

Ok, that explains it. You are right - I mixed up, the count is correct.

regards, gaute

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[David Bremner]

Gaute Hope <eg@gaute.vetsj.com> writes:
>> 
>
> Ok, that explains it. You are right - I mixed up, the count is correct.
>
> regards, gaute

At the end of this message is a simpler example, that I think
demonstrates a bug. At the very list it's not very nice that the
destructiveness only shows up when excludes are involved.

Here is the output; note the second exclude query!

,----
| ╭─ maritornes:~/tmp 
| ╰─% NOTMUCH_DEBUG_QUERY=1 ./a.out
| Query string is:
| tag:muted
| Exclude query is:
| Xapian::Query()
| Final query is:
| Xapian::Query((Tmail AND 0 * Kmuted))
| ret1=84
| Exclude query is:
| Xapian::Query(<alldocuments>)
| Final query is:
| Xapian::Query(((Tmail AND 0 * Kmuted) AND_NOT <alldocuments>))
| ret2=0
`----



#include <notmuch.h>
#include <stdlib.h>
#include <stdio.h>

int main(int argc, char **argv){

  notmuch_database_t *nm_db;
  notmuch_query_t *query;

  if (notmuch_database_open ("/home/bremner/Maildir/",
			     NOTMUCH_DATABASE_MODE_READ_ONLY, &nm_db))
    return 1;

  query = notmuch_query_create (nm_db, "tag:muted");
  notmuch_query_add_tag_exclude (query, "muted");
  notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_TRUE);

  printf("ret1=%d\n", notmuch_query_count_threads (query));
  printf("ret2=%d\n", notmuch_query_count_threads (query));

}

Re: notmuch_query_add_tag_excluded and notmuch_query_search_threads

[Mark Walters]

Hi

>
> At the end of this message is a simpler example, that I think
> demonstrates a bug. At the very list it's not very nice that the
> destructiveness only shows up when excludes are involved.
>
> Here is the output; note the second exclude query!

I think I know what is causing this and can (almost) give a fix.

The problem is that _notmuch_exclude_tags in lib/query.cc is not
idempotent: it iterates through the list of excluded tag attached to
query and changes any which match a tag in the query to "". But this
gets confused if applied a second time as it sees an empty string rather
than Ktag (where "K" is the prefix for 'tag'). 

We could just check in _notmuch_exclude_tags whether the excluded item
is "" and if so, just skip it. (I am not sure what the right syntax for
this in the mix of C and C++ that happens in query.cc)

Alternatively we could try and modify the list of excluded tags directly
when doing the iteration (ie link the previous tag to the next
tag). 

Since both of these do modify the actual query (as we do currently)
there is a possibility that a user could be surprised. I think the user
can't modify the actual query string so I don't think this actually
occurs.

Any thoughts?

Mark