From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id EFBC0431FBF for ; Wed, 23 Apr 2014 14:28:58 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VcMfwEmVbsEp for ; Wed, 23 Apr 2014 14:28:51 -0700 (PDT) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 54BEA431FBD for ; Wed, 23 Apr 2014 14:28:51 -0700 (PDT) X-AuditID: 1209190f-f790b6d000000c3a-5c-53583092726f Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id 7D.67.03130.29038535; Wed, 23 Apr 2014 17:28:50 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s3NLSmnf002716; Wed, 23 Apr 2014 17:28:49 -0400 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s3NLSk48019621 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 23 Apr 2014 17:28:47 -0400 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) (envelope-from ) id 1Wd4ij-0004yg-2B; Wed, 23 Apr 2014 17:28:45 -0400 Date: Wed, 23 Apr 2014 17:28:43 -0400 From: Austin Clements To: David Mazieres expires 2014-07-05 CEST Subject: Re: [PATCH] Add configurable changed tag to messages that have been changed on disk Message-ID: <20140423212843.GN25817@mit.edu> References: <1396800683-9164-1-git-send-email-eg@gaute.vetsj.com> <87wqf2gqig.fsf@ta.scs.stanford.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87wqf2gqig.fsf@ta.scs.stanford.edu> User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFuplleLIzCtJLcpLzFFi42IR4hRV1p1kEBFssOA1v0XT50usFqsWxVtc vzmT2YHZ48e/ZjaPZ6tuMXtc+ruNKYA5issmJTUnsyy1SN8ugSvj57bTLAWzhSounj7K0sB4 hK+LkZNDQsBEYvLFD0wQtpjEhXvr2boYuTiEBGYzSdyctwbK2cgo0X96IjOEc5pJYtb1lawQ zhJGicPnLoH1swioSky7t5gRxGYT0JDYtn85mC0iUCwx8fMFZhCbWcBGYsLH12wgtrBAnMS0 yzvAenkFdCSevl4AViMkkCFxYsYhFoi4oMTJmU9YIHq1JG78ewlUzwFkS0ss/8cBEuYUMJR4 tLqdFcQWFVCRmHJyG9sERqFZSLpnIemehdC9gJF5FaNsSm6Vbm5iZk5xarJucXJiXl5qka6J Xm5miV5qSukmRnCoS/LvYPx2UOkQowAHoxIPr8Tl8GAh1sSy4srcQ4ySHExKorzJmhHBQnxJ +SmVGYnFGfFFpTmpxYcYJTiYlUR4N2sD5XhTEiurUovyYVLSHCxK4rxvra2ChQTSE0tSs1NT C1KLYLIyHBxKErxL9YEaBYtS01Mr0jJzShDSTBycIMN5gIbPB6nhLS5IzC3OTIfIn2JUlBLn vagHlBAASWSU5sH1wlLRK0ZxoFeEeZtA2nmAaQyu+xXQYCagwQUTwkEGlyQipKQaGAXir9hG mq/sWHJoN3vHPBMG4+NMZqKyNVtn7Pb3fdxQb2hbHXHgTeGR61L3hIr/XJk6W5nx+e7zL1nN dt5cFXnfbVpfAqPp36g/6wriPmxhF/cw9jqjcii6crKwUbga+8pWHyPDFE2f5E8Rrdde/9h5 IPn+1dD9z14Zz+XQ6TlzlPXaGxG7v0osxRmJhlrMRcWJAFmmB+ggAwAA Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 21:28:59 -0000 Quoth David Mazieres on Apr 06 at 10:19 pm: > Gaute Hope writes: > > > When one of the source files for a message is changed on disk, renamed, > > deleted or a new source file is added. A configurable changed tag is > > is added. The tag can be configured under the option 'changed_tags' in > > the [new] section, the default is none. Tests have been updated to > > accept the new config option. > > > > notmuch-setup now asks for a changed tag after the new tags question. > > > > This could be useful for for example 'afew' to detect remote changes in > > IMAP folders and update the FolderNameFilter to also add tags or remove > > tags when a _existing_ message has been added to or removed from a > > maildir. > > I think this is the wrong way to achieve such functionality, because > then the change tag A) is expensive to remove, B) is easy to misuse > (remember to call fsync everywhere before deleting the change tag), and > C) can be used by only one application. > > A better approach would be to add a new "modtime" xapian value that is > updated whenever the tags or any other terms (such as XFDIRENTRY) are > added to or deleted from a docid. If it's a Xapian value, rather than a > term, then modtime will be queriable just like date, allowing multiple > applications to query all docids modified since the last time they ran. I'd like to have efficient change detection, too. In my case, I'd like to use it to support efficient live search and show updates. The design I'd sketched out for that used a log rather than ctimes, and I'm curious if you have thoughts on the relative merits and suitability for tag sync of these approaches. I'd been leaning toward logging because it can capture changes to things that aren't represented as documents in the database, such as thread membership. This probably doesn't matter for synchronization, but it makes it much easier to figure out which threads in thread search results need to be refreshed. A log can also capture message deletion easily, while ctimes would require tombstones (which may be a good idea for other reasons [1]). On the other hand, logging is obviously more mechanism than ctimes, and probably requires some garbage collection story. [1] id:20140421162058.GE25817@mit.edu