From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 947BE431FBC for ; Mon, 12 Aug 2013 07:34:41 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jlc2fYe5+FOD for ; Mon, 12 Aug 2013 07:34:35 -0700 (PDT) Received: from dmz-mailsec-scanner-3.mit.edu (dmz-mailsec-scanner-3.mit.edu [18.9.25.14]) by olra.theworths.org (Postfix) with ESMTP id 4A18F431FAF for ; Mon, 12 Aug 2013 07:34:35 -0700 (PDT) X-AuditID: 1209190e-b7f988e0000009a7-80-5208f277a9ec Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP id 79.10.02471.872F8025; Mon, 12 Aug 2013 10:34:32 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id r7CEYUoX006416; Mon, 12 Aug 2013 10:34:31 -0400 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r7CEYSnq030359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 12 Aug 2013 10:34:29 -0400 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) (envelope-from ) id 1V8tCV-0006nu-KX; Mon, 12 Aug 2013 10:34:27 -0400 Date: Mon, 12 Aug 2013 10:34:26 -0400 From: Austin Clements To: Vladimir Marek Subject: Re: Possible addtions to notmuch new ? Message-ID: <20130812143426.GA13257@mit.edu> References: <20130812093443.GB16684@virt.cz.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130812093443.GB16684@virt.cz.oracle.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupjleLIzCtJLcpLzFFi42IRYrdT1634xBFkMPkVq8X1mzOZLTpu72Zz YPJ4tuoWs8fHp7dYApiiuGxSUnMyy1KL9O0SuDJ6Vk5kKWiXqDj5zruB8Z1QFyMnh4SAicSE Q8+YIWwxiQv31rN1MXJxCAnsY5SY+PstlLORUWJazwk2kCohgdNMEr23IiESSxglpnQvBmtn EVCV2DNzHVgRm4CGxLb9yxlBbBEBPYlNm46xgtjMAtIS3343M4HYwgL6Ejev9rKA2LwCOhLz JrczQSwwlzi4cjkTRFxQ4uTMJywQvVoSN/69BIpzgM1Z/o8DJMwpYCExc1YD2CpRARWJKSe3 sU1gFJqFpHsWku5ZCN0LGJlXMcqm5Fbp5iZm5hSnJusWJyfm5aUW6Rrr5WaW6KWmlG5iBAU1 pyTfDsavB5UOMQpwMCrx8GZ+5AgSYk0sK67MPcQoycGkJMq78wNQiC8pP6UyI7E4I76oNCe1 +BCjBAezkgiv2jygHG9KYmVValE+TEqag0VJnPfZ07OBQgLpiSWp2ampBalFMFkZDg4lCV4L kD2CRanpqRVpmTklCGkmDk6Q4TxAw31AaniLCxJzizPTIfKnGHU5/qyc+4lRiCUvPy9VSpzX A6RIAKQoozQPbg4sGb1iFAd6S5g3EKSKB5jI4Ca9AlrCBLTEqBlsSUkiQkqqgXHD2sOmllsk 3mz3nGeSGeoed1mkomPelpUWqxfXfXDpmNV0VOiPfur0aul134Qkf3Yv2WnROq3lU/w11zfr HCYKyOcWuszhOLNUsVlKT+xA9KSJBtsnfpKq+aFx6MLe93Ez5aS3iGaVvTx3oiRu43rr0uV/ vAwUzzC9ucHQeN92odkUyykzoluUWIozEg21mIuKEwHNfNWjIQMAAA== Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Aug 2013 14:34:41 -0000 Quoth Vladimir Marek on Aug 12 at 11:34 am: > Hi, > > My mail setup is a directory containing several subdirectories each > subdirectory corresponds to one real mail account I am using. Each mail > account is synchronized differently - I am using offlineimap, fetchmeail > or even synthetically created emails (I am writing very simple jabber<-> > mail gate).Every now and then I am running 'notmuch new' to discover new > emails and make them available in my MUA. > > That works pretty well, but has some disadvantages too > - notmuch new takes very long time (30s) during which the notmuch > database seems to be locked for any other updates from my MUA > - notmuch new takes long time because it always processes my archive > dir containing many files. That's mostly un-necessary as typically > there's no new mail delivered Could you try this patch? It's basically untested other than passing the test suite, though in principle the worst harm it could do is make notmuch new miss new messages or think renames are deletions. If it helps significantly with your performance problems, I'll clean it up and add a test. diff --git a/notmuch-new.c b/notmuch-new.c index faa33f1..196c5cb 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -323,6 +323,9 @@ add_files (notmuch_database_t *notmuch, } db_mtime = directory ? notmuch_directory_get_mtime (directory) : 0; + if (directory && db_mtime == fs_mtime && st.st_nlink == 2) + goto DONE; + /* If the database knows about this directory, then we sort based * on strcmp to match the database sorting. Otherwise, we can do * inode-based sorting for faster filesystem operation. */ > - I don't have the possibility of passing new mails through procmail. > That would be useful for example for changing cron mail subjects, > putting related automated mails into threads (bugzilla, etc.). > > > I was thinking that if we could split the new mail discovery from > it's processing, it would solve the disadvantages I'm facing. Something > like > > notmuch new --verbose --dry-run [dir] | my_filter | notmuch insert - > > It would work > - --dry-run would not lock and change the database > - --verbose would print the changes to stdout/stderr. Something like: > > new mail/file.1 > new mail/file.2 > deleted mail/file.3 > renamed mail/file.4 mail/file.5 > ... > > [dir] would limit the scope of 'notmuch' new search to dir and it's > subdirectories. Alternatively we could have set of include or exclude > rules similarly to rsync (for example), but [dir] is easier to > implement. > > 'my_filter' would be my script which could change the contents of emails > before they are inserted into notmuch database. > > Notmuch insert would be able to not only add new mails, but also remove > old ones or note that the file was renamed. > > How would this sound? > > I'm not saying I would implement this, I'm rather curious where would > you like to see notmuch in the future. > > Cheers