Tag search peculiarities

Tag search peculiarities

[Ben Gamari]

Hey all,

While trying perfect my initial tagging script, I have run into a very strange
set of issue. In my script, I use the following to exclude mailing list traffic
from my inbox,

notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new

However, I found that mailing list traffic was still getting through.

After investigating further, I found that any query in the form of "tag:inbox
and tag:$TAG" would return no results. Strangely, all other combinations of tag
searches (i.e. "tag:lkml and tag:unread") seem to work just fine.

Has anyone else noticed this sort of behavior? Does the inbox tag have some
special signifigance that I should know of? Is my index just FUBAR? (the ladder
would be very strange considering it's only a few days old and I can't think of
any crashes, etc. that might have corrupted it) Any ideas for debugging?

Thanks!

- Ben

Re: Tag search peculiarities

[Michal Sojka]

On Wed, 17 Mar 2010, Ben Gamari wrote:
> While trying perfect my initial tagging script, I have run into a very strange
> set of issue. In my script, I use the following to exclude mailing list traffic
> from my inbox,
> 
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new
> 
> However, I found that mailing list traffic was still getting through.
> 
> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of? Is my index just FUBAR? (the ladder
> would be very strange considering it's only a few days old and I can't think of
> any crashes, etc. that might have corrupted it) Any ideas for debugging?

Hi Ben,

I've never had similar problems. AFAIK there is nothing special on inbox
tag. It seems you use your configurable tags patch so I suspected that
patch, but it seems OK.

I'd try the following:
notmuch dump|grep "inbox.*${TAG}"
notmuch dump|grep "${TAG}.*inbox"

It if outputs something, then it's really strange.

-Michal

Re: Tag search peculiarities

[Sandra Snan]

Hey, Ben.
Ben Gamari <bgamari.foss@gmail.com> wrote:
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new

Is there a new line between the calls? Like:
notmuch tag -new tag:new and tag:list # removes new from list
notmuch tag -new +inbox tag:new # replaces those that still are new with inbox

> However, I found that mailing list traffic was still getting through.

What do you mean by “through”? Do you mean that there still are
messages tagged both inbox and list? Even though you can’t search for
them?

If a message is tagged both inbox and list, but not new then the two
lines of your script that you posted so far wouldn’t change it, and it
would still bo both inbox and list.

It’s risky being so dependent on the new tag.

> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of?

Yes, many versions of the reading client (for example, the version of
notmuch.el that ships with the notmuch package in debian) remove the
inbox tag from messages once you scroll past them or when you press
the key that’s bound to notmuch-show-archive-thread (default is “a”).

> Is my index just FUBAR? (the ladder would be very strange
> considering it's only a few days old and I can't think of any
> crashes, etc. that might have corrupted it) Any ideas for debugging?

Add the tags manually to a few messages, search for them again, and be
sure to not run your tagging script while looking at this problem.

Michal’s tip, looking at a few lines of notmuch dump, is a good idea
too. Notmuch dump is fast, so don’t be afraid (notmuch restore on the
other hand… but even that completes fast enough.)

I’ve often been surprised at weird tag situations then realized that
it was old versions of my own scripts that had ran in the background
and I’ve forgotten about it.

Boolean algebra and set theory is tricky stuff! Good luck.

Sandra

Re: Tag search peculiarities

[Ben Gamari]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2781 bytes --]

On Thu, 18 Mar 2010 15:27:12 +0100, Sandra Snan <sandra.snan@handgranat.org> wrote:
> Hey, Ben.
> Ben Gamari <bgamari.foss@gmail.com> wrote:
> > notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new
> 
> Is there a new line between the calls? Like:
> notmuch tag -new tag:new and tag:list # removes new from list
> notmuch tag -new +inbox tag:new # replaces those that still are new with inbox
> 
Eh? They are two separate notmuch invocations.

> > However, I found that mailing list traffic was still getting through.
> 
> What do you mean by “through”? Do you mean that there still are
> messages tagged both inbox and list? Even though you can’t search for
> them?
> 
Yeah, poor choice of words. After the two commands above ran, my mailing list
messages were tagged with new, inbox.

> If a message is tagged both inbox and list, but not new then the two
> lines of your script that you posted so far wouldn’t change it, and it
> would still bo both inbox and list.
> 
> It’s risky being so dependent on the new tag.
> 
Eh? I fail to see why. It's no more risky than depending on the inbox tag.

> > After investigating further, I found that any query in the form of "tag:inbox
> > and tag:$TAG" would return no results. Strangely, all other combinations of tag
> > searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> > 
> > Has anyone else noticed this sort of behavior? Does the inbox tag have some
> > special signifigance that I should know of?
> 
> Yes, many versions of the reading client (for example, the version of
> notmuch.el that ships with the notmuch package in debian) remove the
> inbox tag from messages once you scroll past them or when you press
> the key that’s bound to notmuch-show-archive-thread (default is “a”).
> 
I guess I should have been more explicit. I meant in notmuch itself, not the
clients.

> > Is my index just FUBAR? (the ladder would be very strange
> > considering it's only a few days old and I can't think of any
> > crashes, etc. that might have corrupted it) Any ideas for debugging?
> 
> Add the tags manually to a few messages, search for them again, and be
> sure to not run your tagging script while looking at this problem.
> 
> Michal’s tip, looking at a few lines of notmuch dump, is a good idea
> too. Notmuch dump is fast, so don’t be afraid (notmuch restore on the
> other hand… but even that completes fast enough.)
> 
> I’ve often been surprised at weird tag situations then realized that
> it was old versions of my own scripts that had ran in the background
> and I’ve forgotten about it.
> 
Indeed. Hopefully it will be something so simple. I'm currently blissfully
unaware of the internals of Xapian, and would really like to remain that way.

Thanks,

- Ben

Re: Tag search peculiarities

[Sandra Snan]

On Sat, 20 Mar 2010 16:09:00 -0700 (PDT), Ben Gamari <bgamari.foss@gmail.com> wrote:
> On Thu, 18 Mar 2010 15:27:12 +0100, Sandra Snan <sandra.snan@handgranat.org> wrote:
> > Hey, Ben.
> > Ben Gamari <bgamari.foss@gmail.com> wrote:
> > > notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new
> > 
> > Is there a new line between the calls? Like:
> > notmuch tag -new tag:new and tag:list # removes new from list
> > notmuch tag -new +inbox tag:new # replaces those that still are new with inbox
> > 
> Eh? They are two separate notmuch invocations.

In my mail (quoted above) it showed up as one single line, a single
invocation. Guess there was mail problem somewhere. Thanks for the clarification.

> Yeah, poor choice of words. After the two commands above ran, my mailing list
> messages were tagged with new, inbox.

And not list? Huh. Does seem weird that after explicitly replacing new
with inbox, as you did, you still have them new.