Problems with Maildir and files moving from new/ to cur/

Problems with Maildir and files moving from new/ to cur/

[Brett Viren]

Hi,

I use procmail to prefilter mail into various sub groups of a Maildir.
 This Maildir is served by dovecot IMAP and is also visible to
notmuch.  I run "notmuch new" followed by "notmuch tag -inbox -unread
tag:inbox" (I use GNUS to do my reading and don't want these tags in
the way the few times I run notmuch for reading) every 10 minutes from
cron.  I often will check my email before the cron job gets a chance
to run and vice versa.

The problem I have is that this race condition means that sometimes
notmuch will index a message as being under the Maildir/.<topic>/new/
subdirectory, I then read it and it is moved to Maildir/.<topic>/cur/
(I think that is what is happening).  Then when I do a search, notmuch
returns a file that is no longer around.

This may not be an issue with notmuch, per se, but does anyone have
any suggestions to make this work more smoothly?

Thanks,
-Brett.

Re: Problems with Maildir and files moving from new/ to cur/

[Michal Sojka]

On Tue, 9 Feb 2010 10:38:04 -0500, Brett Viren <brett.viren@gmail.com> wrote:
> Hi,
> 
> I use procmail to prefilter mail into various sub groups of a Maildir.
>  This Maildir is served by dovecot IMAP and is also visible to
> notmuch.  I run "notmuch new" followed by "notmuch tag -inbox -unread
> tag:inbox" (I use GNUS to do my reading and don't want these tags in
> the way the few times I run notmuch for reading) every 10 minutes from
> cron.  I often will check my email before the cron job gets a chance
> to run and vice versa.
> 
> The problem I have is that this race condition means that sometimes
> notmuch will index a message as being under the Maildir/.<topic>/new/
> subdirectory, I then read it and it is moved to Maildir/.<topic>/cur/
> (I think that is what is happening).  Then when I do a search, notmuch
> returns a file that is no longer around.
> 
> This may not be an issue with notmuch, per se, but does anyone have
> any suggestions to make this work more smoothly?

Hi,

I'm afraid that with current version there is no solution for this.
There were some proposals to tell notmuch which files you want to index
and use this functionality to run indexing right on delivery on when a
rename is detected (e.g. by inotify). See
id:1264719647-sup-9540@ben-laptop for details. AFAIK nobody have sent
patches for this yet.

Cheers,
Michal

Re: Problems with Maildir and files moving from new/ to cur/

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

On Tue, 9 Feb 2010 10:38:04 -0500, Brett Viren <brett.viren@gmail.com> wrote:
> The problem I have is that this race condition means that sometimes
> notmuch will index a message as being under the Maildir/.<topic>/new/
> subdirectory, I then read it and it is moved to Maildir/.<topic>/cur/
> (I think that is what is happening).  Then when I do a search, notmuch
> returns a file that is no longer around.

The problem here is due to some drifting assumptions.

When I made the "notmuch new" command, I was assuming a mailstore that
wouldn't change except for new messages being added, (such that it
likely wouldn't be inconvenient to run "notmuch new" when that mail was
added).

But now that we have support within notmuch for a mailstore that does
change, (with messages being deleted and renamed), then "notmuch new" is
too blunt a tool for what you want here.

One option would be to implement some new interface in notmuch for
indexing specific file paths. Then you could arrange for that to be
called when your mail client does the renaming, (either by modifying the
mail client itself or by hooking into something like inotify).

> This may not be an issue with notmuch, per se, but does anyone have
> any suggestions to make this work more smoothly?

Otherwise, the only way I see for you to get the behavior you want
without having to write any new code is to just run "notmuch new;
notmuch search <foo>" whenever you want to search. That's not ideal, but
might be a usable workaround for now as it would hopefully add just a
second or two to your search times.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]