From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 5CC74431FC3 for ; Mon, 1 Mar 2010 08:27:20 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -1.442 X-Spam-Level: X-Spam-Status: No, score=-1.442 tagged_above=-999 required=5 tests=[AWL=-0.332, BAYES_05=-1.11] autolearn=ham Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SNGkimUFunby for ; Mon, 1 Mar 2010 08:27:19 -0800 (PST) Received: from mail-vw0-f53.google.com (mail-vw0-f53.google.com [209.85.212.53]) by olra.theworths.org (Postfix) with ESMTP id 6A573431FBD for ; Mon, 1 Mar 2010 08:27:19 -0800 (PST) Received: by vws5 with SMTP id 5so22534vws.26 for ; Mon, 01 Mar 2010 08:27:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:subject:from:to :in-reply-to:references:date:message-id:user-agent :content-transfer-encoding; bh=h3zzXFxabL7VDuMTj3FTFcd5bQHl1wapUjdxZlQQcPw=; b=VDtVBKL5kusFzrX24CsrViP0jDtQHqfIgUjXcC7iHTpEKlxTzeKacrlEdtun3x0Mq8 pBblDVcVSCEcIlgctzFJiRE0dpPKn4nk28u0IM99VskVNhFwtLPaZPi0eUabT8P0Dxw3 9T8cgOQwN9KetxIXE7ta1QJPfM+RiAVU0mN/s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:subject:from:to:in-reply-to:references:date:message-id :user-agent:content-transfer-encoding; b=JoV3f4Ia0cl8DZxC9Xcf5QHnNqy4V3iWdR4zBF9J9yO0GvdWFsIx/6xJ232j40RZBN nL1AOENvVQEAb4ZDibXDoXkh7tDPdbou4YUHIJ7i5VmQOu1ls+k69n36ITiyquwfCJ6Y vR3rb2OIZ1fCZyJtrLp5termgG9Py63dlbiPM= Received: by 10.220.124.24 with SMTP id s24mr3315080vcr.182.1267460831289; Mon, 01 Mar 2010 08:27:11 -0800 (PST) Received: from localhost (umass-959-186.wireless.umass.edu [128.119.77.186]) by mx.google.com with ESMTPS id 29sm31434003vws.3.2010.03.01.08.27.09 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 01 Mar 2010 08:27:09 -0800 (PST) Content-Type: text/plain; charset=UTF-8 From: Ben Gamari To: notmuch In-reply-to: <87ljecmnbd.fsf@steelpick.localdomain> References: <87pr57jvkz.fsf@SSpaeth.de> <87wry2wl7p.fsf@yoom.home.cworth.org> <87ljecmnbd.fsf@steelpick.localdomain> Date: Mon, 01 Mar 2010 11:27:07 -0500 Message-Id: <1267459947-sup-6640@ben-laptop> User-Agent: Sup/git Content-Transfer-Encoding: 8bit Subject: Re: Introducing notmuchsync X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Mar 2010 16:27:20 -0000 Excerpts from Michal Sojka's message of Mon Mar 01 03:57:26 -0500 2010: > Hi, when you are on this topic, I'll put my two cents in. Currently > notmuch works only with mail-store comprised of files. People want to > work with their mails on multiple computers so there are several > ideas/solutions how to achieve that. Notmuchsync, which plays well with > offlineimap, is one of them. Another idea is Git based mail-store, which > I would really like to have. > > If we want to make notmuch work with Git-based mail-store, it will be > necessary to make the interface between notmuch core and mail-store > handling code a bit more abstract so that it will be possible to > configure mail-store type to be used. I agree that we should make notmuch less tied to maildirs as the only possible mailstore. In the interest of simplicity, however, it's important to keep the surface area of the interface down. > > This abstract mail-store interface will contain methods for the > following operations: > 1. read mail identified by a path from mail-store > (current notmuch uses fread() for this, for git this will be > something like system("git cat-file ...")) > Yep. Seems reasonable. > 2. get the list of new mails which need to be indexed > (current notmuch does recursive file traversal, for git-based store > it will be something like system("git diff-tree --name-status ...") > Is this really necessary? Another option (which I believe has been mentioned here in the past) is to simply pass notmuch new a list of message "paths" to add (although I'm not sure if many mail delivery programs give you that sort of information). How do you propose that the backends keep track of what mail has been indexed? > 3. add/remove tags for a given message (this will be NULL for current > notmuch functionality) Yep, makes sense. > > This way the mail-store will be able to store both mails and > corresponding tags and in case of git, it will be easy to synchronize > mail-stores on multiple computers. Sounds great. Can't wait! > > Now, if we have this, it would be very easy to add support for > Maildir-based mail-store. The implementation of the first two methods > would be the same as what is currently in notmuch and the third method > would rename files in mailstore if certain tags (such as unread) are > added or removed. In case of rename, notmuch database would be > immediately updated to reflect the change. > The interface here seems a little vague. How would the backend notify notmuch that the filename has changed? > So to summarize, I think there should be an abstract layer for > handling different types of mail-store. I can see at least three > possible implementations of this abstract interface: 1) immutable > mail-store (as currently implemented in notmuch) 2) Maildir-based > mail-store for limited synchronization with other Maildir tools and > 3) git-based mail-store for full synchronization. Don't forget mbox. It seems like it would be pretty trivial to implement (although I'm not sure what performance would look like). > > I've already started working on this, but I'm still in the state where I > mainly study how notmuch works in order not to break its current > functionality. With all of this infrastructure, it seems like it wouldn't be too difficult to support messages from multiple backends in a single index. Not sure if people would be interested enough to warrant the added complexity though. > > I'd like to hear what others think about this idea. > That's my uninformed opinion. Take it with an appropriately sized grain of salt. I do think that some sort of abstraction will be necessary though. I'm glad you're taking a look at this. I've been wanting to start hacking away at some sort of prototype git backend, but I haven't wanted to start before we have the appropriate infrastructure in notmuch. Cheers, - Ben