From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 64DE2431FCF for ; Sun, 15 Jan 2012 15:40:24 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -2.3 X-Spam-Level: X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8+hn0dcE9SYI for ; Sun, 15 Jan 2012 15:40:23 -0800 (PST) Received: from tempo.its.unb.ca (tempo.its.unb.ca [131.202.1.21]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id BC4B5431FC0 for ; Sun, 15 Jan 2012 15:40:23 -0800 (PST) Received: from zancas.localnet (fctnnbsc36w-156034076032.pppoe-dynamic.High-Speed.nb.bellaliant.net [156.34.76.32]) (authenticated bits=0) by tempo.its.unb.ca (8.13.8/8.13.8) with ESMTP id q0FNeIIW030647 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Sun, 15 Jan 2012 19:40:19 -0400 Received: from bremner by zancas.localnet with local (Exim 4.77) (envelope-from ) id 1RmZgQ-0000sP-8B; Sun, 15 Jan 2012 19:40:18 -0400 From: David Bremner To: Austin Clements Subject: Re: [PATCH v3 04/10] notmuch-dump: add --format=(notmuch|sup) In-Reply-To: <20120115203511.GB9385@mit.edu> References: <874nwxbkhr.fsf@zancas.localnet> <1326591624-15493-1-git-send-email-david@tethera.net> <1326591624-15493-5-git-send-email-david@tethera.net> <20120115203511.GB9385@mit.edu> User-Agent: Notmuch/0.11+61~gbaa2c97 (http://notmuchmail.org) Emacs/23.3.1 (x86_64-pc-linux-gnu) Date: Sun, 15 Jan 2012 19:40:18 -0400 Message-ID: <87k44silgd.fsf@zancas.localnet> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2012 23:40:24 -0000 On Sun, 15 Jan 2012 15:35:11 -0500, Austin Clements wrote: > We definitely need a round-trip-able dump format. Did you consider > using JSON to allow for future flexibility (e.g., expansion of what we > store in the database) and so we don't have to invent our own > encodings? A JSON format wouldn't necessarily be a reason *not* to > also have this format, especially considering how > shell-script-friendly this is (versus how shell-script-unfriendly JSON > is), I'm just curious what trade-offs you're considering. I was looking for something fairly close to what we have, to allow people to migrate their various scripts (e.g. nmbug) to the new format without too much pain. Maybe some small amount of header information at the start of the file would support extensibility, while still being shell script friendly. I'm also not too sure how much overhead the JSON quoting would induce. My tags file is currently about 10M, and on my old laptop takes about 15s to dump. That's a long 15s when I'm trying to sync my mail. For "normal" backup use, a little more overhead doesn't matter, although the stories of non-linear slowdowns that people report suggest we shouldn't get too cavalier about that. > You might want to call this format something more self-descriptive > like "text" or "hextext" or something in case we do want to expand in > the future. "sup" is probably fine for the legacy format since that's > set in stone at this point. yeah, I'm definitely open to better suggestions for a name