From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 11D93431FB6 for ; Mon, 21 Mar 2011 07:31:36 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ktu6a8Ihivi0 for ; Mon, 21 Mar 2011 07:31:35 -0700 (PDT) Received: from mail-wy0-f181.google.com (mail-wy0-f181.google.com [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 33234431FB5 for ; Mon, 21 Mar 2011 07:31:35 -0700 (PDT) Received: by wyf23 with SMTP id 23so5770386wyf.26 for ; Mon, 21 Mar 2011 07:31:31 -0700 (PDT) Received: by 10.216.56.65 with SMTP id l43mr5079358wec.113.1300717891612; Mon, 21 Mar 2011 07:31:31 -0700 (PDT) Received: from localhost (203.56-242-81.adsl-dyn.isp.belgacom.be [81.242.56.203]) by mx.google.com with ESMTPS id t11sm2579666wes.41.2011.03.21.07.31.27 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 21 Mar 2011 07:31:28 -0700 (PDT) From: Pieter Praet To: Mueen Nawaz , notmuch@notmuchmail.org Subject: Re: Questions about importing mail (mbox) In-Reply-To: <87bp15m9oz.fsf@fester.com> References: <87bp15m9oz.fsf@fester.com> User-Agent: Notmuch/0.5-49-g4aeb291 (http://notmuchmail.org) Emacs/23.1.50.1 (x86_64-pc-linux-gnu) Date: Mon, 21 Mar 2011 15:31:26 +0100 Message-ID: <87zkooo88x.fsf@A7GMS.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Mar 2011 14:31:36 -0000 On Sun, 20 Mar 2011 20:30:52 -0700, Mueen Nawaz wrote: > > Hi, > > I'm trying to experiment with notmuch. > > As I understand it, notmuch does not handle mbox for input. The problem > is that all my mail is currently in mbox format. > > So I first tried converting mbox to maildir using mb2md. > > It didn't do a good job. When I subsequently tried importing to notmuch, > notmuch complained about lots of non-mail files - I confirmed that > indeed mb2md had botched converting those emails. > > So then I tried to convert to mh format using Sylpheed. This seemed to > go well, but then when importing to notmuch, it complained again for > about 20 emails, and a manual check confirmed that some messages did not > get converted properly to mh (they don't show up in Sylpheed). > > And then I noticed another discrepancy. mutt shows that I started with > 44473 messages in mbox. When I imported into Sylpheed, it showed 44482 > messages (no idea where the extra 9 came from). However, notmuch is > reporting that it processed 44482 files, but that it added 35602 > messages. > > Why only 35602 (it complained for only about 20 messages)? A search > confirmed that some messages that show up in both mutt (in mbox) and > Sylpheed (in mh format) were not indexed. > > So I want to know: When you guys switched to notmuch, how did you ensure > you did not miss any emails. I really, really, really don't want to lose > any emails in this process! > > Thanks. > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch It would've been a no-brainer if you'd been using Maildir all along (mbox is evil incarnate), but... I'd suggest keeping your original mbox file safe in git [1], and consistently commiting every step of the way, so even if messages were to get lost in translation, you still have a way to get them back, with negligible storage overhead (just remember to "git gc --aggressive --prune=now" when you're finished). Compacting the mbox file, i.e. purging all stale messages (sync-mailbox in mutt?) and diffing to HEAD could then possibly give you an indication as to the origin of the 9 surplus files. For the actual conversion to Maildir (and any type of mail fetching in general), I'd suggest using FDM [2], you'll never look back. Regarding the significant discrepancy between processed and added files in Notmuch: Could be dupes (e.g. mail to/cc/bcc yourself or mailing lists, ending up in both Inbox and Sent), which are automatically suppressed by Notmuch. [1] http://git-scm.com/ [2] http://fdm.sourceforge.net/