From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id A047E431FBC for ; Wed, 17 Feb 2010 17:01:25 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.878 X-Spam-Level: X-Spam-Status: No, score=-0.878 tagged_above=-999 required=5 tests=[AWL=-0.879, BAYES_50=0.001] autolearn=ham Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iRb4c94B65Y3 for ; Wed, 17 Feb 2010 17:01:24 -0800 (PST) Received: from mail-vw0-f53.google.com (mail-vw0-f53.google.com [209.85.212.53]) by olra.theworths.org (Postfix) with ESMTP id D5779431FAE for ; Wed, 17 Feb 2010 17:01:24 -0800 (PST) Received: by vws16 with SMTP id 16so675183vws.26 for ; Wed, 17 Feb 2010 17:01:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:subject:from:to :in-reply-to:references:date:message-id:user-agent :content-transfer-encoding; bh=79qGzAW8CB8B3lAy8Uq1trXhZpbapnshxR7UGcfBja0=; b=ePs60i1tHOS3YGPKLlkExSG4jv8Hgt8zGN8ATspJOl3td05EOsHLGMhYjJk07ZIHJr rFtOBo3KEPjfQktvfS6d9kC8sBgd+hnRaXbE4f4Jg0mubYhFbLzY3SjpYV+zn5PZrTwX C+KHgC67s1Nt1oAC/tSyMXmCRqbWt/RWyE2pk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:subject:from:to:in-reply-to:references:date:message-id :user-agent:content-transfer-encoding; b=HL/1gZ2jTjyVI1BjcE2AzYcjvZBgX3ZXHsUWfEH88hsps3Hs7nPgvHaHFNob/jP+WK nfzxnlqRc6GPZ95H/c6UBhXYS9QToFJfgxgDO0t9iBJikqEGHhf4KqsocUFUdIhw997g M/QuBuivkqBG4DLU9NGRpZsolsKRiSuR6IIT8= Received: by 10.220.126.208 with SMTP id d16mr6376379vcs.20.1266454884464; Wed, 17 Feb 2010 17:01:24 -0800 (PST) Received: from localhost (umass-959-129.wireless.umass.edu [128.119.77.129]) by mx.google.com with ESMTPS id 30sm22175501vws.1.2010.02.17.17.01.22 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 17 Feb 2010 17:01:23 -0800 (PST) Content-Type: text/plain; charset=UTF-8 From: Ben Gamari To: notmuch In-reply-to: <87ljerju0q.fsf@willster.local.flamingspork.com> References: <20100215002914.GA22402@flamingspork.com> <20100217012101.GD8249@lapse.rw.madduck.net> <87ljerju0q.fsf@willster.local.flamingspork.com> Date: Wed, 17 Feb 2010 20:01:21 -0500 Message-Id: <1266453575-sup-3653@ben-laptop> User-Agent: Sup/git Content-Transfer-Encoding: 8bit Subject: Re: Mail in git X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 01:01:25 -0000 Excerpts from Stewart Smith's message of Wed Feb 17 18:56:53 -0500 2010: > On Wed, 17 Feb 2010 14:21:01 +1300, martin f krafft wrote: > > What I am wondering is if (explicit) tags couldn't be represented as > > tree-objects with this. > > I think it could get expensive for tags with lots of messages. > > As far as I understand it, the tree object is stored in full and space > is only reclaimed during repack (due to delta compression). > > So if you, say, had the entire history of a high volume list such as > linux-kernel, adding messages could get rather expensive if you > auto-tagged (or autotagged messages with patches or whatever). > Well, it's tough to say, but I don't think it's as bad as you think. I proposed that we could use a tree structure like the following, ╭─msg1 ╭tagA.list1╶┼─msg2 │ ╰─msg3 │ │ ╭─msg4 ├tagA.list2╶┼─msg5 │ ╰─msg6 tagA ╶┤ │ ╭─msg7 ├tagA.list3╶┼─msg8 │ ╰─msg9 │ │ ╭─msg10 ╰tagA.list4╶┼─msg11 ╰─msg12 This way, adding a message to, say list3, would only require rewriting list3 and tagA, which seems pretty reasonable to me. Moreover, we could make the tree structure as deep as necessary, although we would need to rewrite a node at every level of the tree, so its tough saying how many levels is too many. It could simply be adaptive (e.g. bisect any nodes with more than N children). This certainly isn't as simple as the naive approach, but I think it's the only reasonable approach performance-wise and I don't believe it shouldn't be too tricky. > > messages would then be deleted whenever using git-gc. > > > > No idea how this would sync if we don't keep ancestry. Otoh, it > > would probably not be very expensive to do just that. > > If we keep ancestry though, we are reusing existing working code for > backup (git-pull :) This is one of the reasons I feel it's important we keep it. And as is stated below, the storage overhead is minimal. > > Keep in mind that with my tests, the Maildir in git is about a quarter > to a fifth of the size of it in Maildir... so a bit of extra usage per > message isn't as dramatic as it may sound. >