From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 978E4431FAF for ; Sun, 24 Jun 2012 09:29:43 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T0rOlydDZeIF for ; Sun, 24 Jun 2012 09:29:43 -0700 (PDT) Received: from smtp.chost.de (setoy.chost.de [217.160.209.225]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id A79AA431FAE for ; Sun, 24 Jun 2012 09:29:42 -0700 (PDT) Received: (qmail 14457 invoked by uid 5015); 24 Jun 2012 16:29:40 -0000 Received: (nullmailer pid 3401 invoked by uid 123); Sun, 24 Jun 2012 16:29:39 -0000 Received: from twin.sascha.silbe.org (twin.sascha.silbe.org [192.168.1.2]) by flatty.sascha.silbe.org ([192.168.1.252]) with SMTP via TCP; 24 Jun 2012 16:29:39 -0000 Received: (nullmailer pid 25943 invoked by uid 8193); Sun, 24 Jun 2012 16:29:39 -0000 From: Sascha Silbe To: notmuch Subject: [PATCH 0/3] Speed up notmuch new for unchanged directories Date: Sun, 24 Jun 2012 18:29:23 +0200 Message-Id: <1340555366-25891-1-git-send-email-sascha-pgp@silbe.org> X-Mailer: git-send-email 1.7.10 Mail-Followup-To: notmuch X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list Reply-To: Sascha Silbe List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jun 2012 16:29:43 -0000 All the time I thought what makes "notmuch new" so abysmally slow is the stat() for each maildir. But as it continued to be slow even after I moved most mails out of 'new' (into 'new-20120624'), I strace'd notmuch and noticed it listed even unchanged directories, thereby listing and iterating over each and every single of the 900k mails in my mail store. There's still quite some room for further improvements as it continues to take several minutes to scan < 100 new mails in changed directories containing < 1000 mails in total. Even the rsync run that fetches the new mails is faster. Sascha Silbe (3): lib: fix NULL checks for filenames iterators lib: add support for rewinding a filenames iterator new: don't read unchanged directories from disk lib/filenames.c | 15 ++++++- lib/notmuch.h | 8 ++++ notmuch-new.c | 130 +++++++++++++++++++++++++++++++++++++++++-------------- 3 files changed, 119 insertions(+), 34 deletions(-) -- 1.7.10