From: Austin Clements <amdragon@MIT.EDU>
To: Carl Worth <cworth@cworth.org>
Cc: notmuch@notmuchmail.org
Subject: Re: [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run
Date: Wed, 26 Jan 2011 10:07:28 -0500 [thread overview]
Message-ID: <20110126150728.GT13226@mit.edu> (raw)
In-Reply-To: <87mxmn27w6.fsf@yoom.home.cworth.org>
Quoth Carl Worth on Jan 26 at 9:59 pm:
> On Wed, 26 Jan 2011 19:15:21 +1000, Carl Worth <cworth@cworth.org> wrote:
> > But then I'm also wondering if perhaps we could do the processing undeferred
> > in all cases?
> >
> > I haven't thought that through, but I'd be glad to hear your ideas.
>
> This is still an open question if anyone wants to pursue it, (it might
> make it simpler to then fix the atomicity bugs with adding new messages
> to the database, and only later adjusting the maildir filename).
I believe you're right that synchronization could always be done
eagerly. In fact, I believe this is true even with the addition of
conjunctive and disjunctive flags.
When adding or removing messages, flag synchronization fully dictates
all tags in flag2tag (that is, the tags' current states don't matter).
It also always examines *all* file names backing the message. This is
a stateless process. With eager synchronization, the tags may go
through some transient states during notmuch new, but will always
settle down to the correct set as of the final add or remove for a
given message ID.
> On that topic, if we decide we do need to defer the tags/flags mapping,
> then the real fix is to probably also defer the addition of the filename
> to the message document in the database. If we change these things only
> at the same time, then we should be able to avoid any problems with
> things getting out of synchronization.
I'd been thinking about this as a way to break up notmuch new into
small, consistent transactions, but I don't think it's necessary since
flags can be synchronized eagerly. In fact, with eager
synchronization and one other change, I believe notmuch new can be
completely correctly performed in lots of small transactions.
The one other change is that currently, if notmuch is interrupted
after updating a directory mtime but before processing file removals,
a subsequent notmuch new will miss those removals. This could be
brushed under the rug, since notmuch will notice the removal after any
other change in that directory. Or it could be handled correctly by
giving directories a "removal mtime" in addition to their current
"addition mtime" that is only updated after removals are handled.
next prev parent reply other threads:[~2011-01-26 15:07 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-01-21 9:59 [PATCH 0/3] Speedups and enhancements of notmuch new Michal Sojka
2011-01-21 9:59 ` [PATCH 1/3] Do not defer maildir flag synchronization during the first " Michal Sojka
2011-01-21 9:59 ` [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run Michal Sojka
2011-01-25 22:42 ` Austin Clements
2011-01-26 9:15 ` Carl Worth
2011-01-26 11:59 ` Carl Worth
2011-01-26 15:07 ` Austin Clements [this message]
2011-01-26 16:50 ` Michal Sojka
2011-01-27 5:04 ` Carl Worth
2011-01-27 5:43 ` Austin Clements
2011-01-30 0:21 ` Rob Browning
2011-01-21 9:59 ` [PATCH 2/3] new: Add all initial tags at once Michal Sojka
2011-01-26 12:10 ` Carl Worth
2011-01-26 16:52 ` Thomas Schwinge
2011-01-27 5:03 ` Carl Worth
2011-01-27 7:14 ` Carl Worth
2011-01-27 11:08 ` Michal Sojka
2011-01-21 9:59 ` [PATCH 3/3] new: Enhance progress reporting Michal Sojka
2011-01-26 12:23 ` Carl Worth
2011-01-26 13:16 ` Michal Sojka
2011-01-26 13:06 ` [PATCH] new: Print progress estimates only when we have sufficient information Michal Sojka
2011-01-26 13:49 ` Carl Worth
-- strict thread matches above, loose matches on Subject: below --
2011-01-28 0:08 [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run Austin Clements
2011-01-28 8:51 ` Sebastian Spaeth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110126150728.GT13226@mit.edu \
--to=amdragon@mit.edu \
--cc=cworth@cworth.org \
--cc=notmuch@notmuchmail.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).