From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id CB42F431FB6 for ; Mon, 15 Oct 2012 17:13:07 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.799 X-Spam-Level: X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yYOY5iwFZpo9 for ; Mon, 15 Oct 2012 17:13:05 -0700 (PDT) Received: from mail-qa0-f53.google.com (mail-qa0-f53.google.com [209.85.216.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id B6660431FAE for ; Mon, 15 Oct 2012 17:13:05 -0700 (PDT) Received: by mail-qa0-f53.google.com with SMTP id s11so2163371qaa.5 for ; Mon, 15 Oct 2012 17:13:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=t8EmuOexxqL8YFO6aVX4zN4D0lw/eaVpkR1H7q3exH4=; b=MoJZj9Dr+pBL9MQneIwxvkgyZs4RhAs+A6oaGoW4mZhA5nmE2HNVhWN2gB0g3q01pH KVLCklvKSbcVyFA2m7G3CV17KDw6psHNlYlnH4TZKcDCHd1CPID0jiQ8cqEVK0n1KQKH nFQeZlRTd5MXGbt0FZNQ6DfaJR1I4EKkmiKzGOxkZ3OnXEs/rxDCoovRqGpn0n+sDY66 OdhKnkHerDRMjeKp8I3TXAh50OHA9bMfHADWQ8EdDqlyVHufGEc/3r+pQEOpOl0W/aMH 9m/FK12r7R3T+T+8CJ9D9MPkoLVHmnKzxOSCLICTlTDEXXS+y0Aew2y+BqyOpNDVylw/ 1FMg== Received: by 10.224.107.5 with SMTP id z5mr23458461qao.41.1350346384972; Mon, 15 Oct 2012 17:13:04 -0700 (PDT) Received: from smtp.gmail.com (p70-80.acedsl.com. [66.114.70.80]) by mx.google.com with ESMTPS id cz8sm11134483qab.21.2012.10.15.17.13.03 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 15 Oct 2012 17:13:04 -0700 (PDT) From: Ethan Glasser-Camp To: Sascha Silbe , notmuch Subject: Re: [PATCH 3/3] new: don't read unchanged directories from disk In-Reply-To: <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org> References: <1340555366-25891-1-git-send-email-sascha-pgp@silbe.org> <1340555366-25891-4-git-send-email-sascha-pgp@silbe.org> User-Agent: Notmuch/0.14+45~g6ea9330 (http://notmuchmail.org) Emacs/23.3.1 (x86_64-pc-linux-gnu) Date: Mon, 15 Oct 2012 20:13:01 -0400 Message-ID: <87y5j78eb6.fsf@betacantrips.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2012 00:13:07 -0000 Sascha Silbe writes: > Previously, notmuch new listed all directories on disk, even if they > were unchanged from the state recorded in the database. This could take > a huge amount of time for large numbers of mails as it would list each > individual mail. > > By iterating over the subdirectories recorded in the database we can > avoid accessing the file system for each unchanged directory. If the > modification time does not match we fall back to a full file system scan > so new subdirectories will get picked up and scanned recursively. Hi! Just going through the patch queue. These patches look fine to me, but I think architecturally, notmuch needs to decide what it wants to do -- if it wants to optimize directory-rescanning, or if it wants to provide a better way to just add files known to be new, and let mail delivery hook into that system. I'd prefer the second, but I think it's OK to merge working code that does the first in the meantime. The perfect is the enemy of the good. Austin raised some concerns on this patch set that I'm not sure were completely addressed. For my part, it seems intuitive that a stat() would be faster than a scandir() and so I would have no problem with merging this series. Ethan