From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 51075404947 for ; Tue, 16 Mar 2010 10:10:28 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -3.943 X-Spam-Level: X-Spam-Status: No, score=-3.943 tagged_above=-999 required=5 tests=[AWL=0.056, BAYES_50=0.001, RCVD_IN_DNSWL_MED=-4] autolearn=ham Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XdhZlTgZUa5F for ; Tue, 16 Mar 2010 10:10:25 -0700 (PDT) Received: from e23smtp02.au.ibm.com (e23smtp02.au.ibm.com [202.81.31.144]) by olra.theworths.org (Postfix) with ESMTP id EE385404942 for ; Tue, 16 Mar 2010 10:10:24 -0700 (PDT) Received: from d23relay05.au.ibm.com (d23relay05.au.ibm.com [202.81.31.247]) by e23smtp02.au.ibm.com (8.14.3/8.13.1) with ESMTP id o2GH77Yg012514 for ; Wed, 17 Mar 2010 04:07:07 +1100 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay05.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o2GH4S0q1454158 for ; Wed, 17 Mar 2010 04:04:28 +1100 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o2GHAMKQ027341 for ; Wed, 17 Mar 2010 04:10:22 +1100 Received: from skywalker.linux.vnet.ibm.com ([9.77.214.188]) by d23av03.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVin) with ESMTP id o2GHAHqS027185; Wed, 17 Mar 2010 04:10:19 +1100 From: "Aneesh Kumar K. V" To: Ben Gamari , Olly Betts , martin f krafft In-Reply-To: <4b9fa5d2.0a4d5e0a.0c0b.ffffdcbb@mx.google.com> References: <4b9dccc0.c6c1f10a.3671.44ec@mx.google.com> <20100315090401.GA29891@glaive.weftsoar.net> <4b9e6e80.09b6660a.6769.6832@mx.google.com> <20100316110846.GK10323@survex.com> <4b9fa5d2.0a4d5e0a.0c0b.ffffdcbb@mx.google.com> Date: Tue, 16 Mar 2010 22:40:17 +0530 Message-ID: <87mxy8chvq.fsf@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: notmuch@notmuchmail.org Subject: Re: Notmuch performance (literally, in my case) X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 17:10:28 -0000 On Tue, 16 Mar 2010 08:37:54 -0700 (PDT), Ben Gamari wrote: > On Tue, 16 Mar 2010 11:08:47 +0000, Olly Betts wrote: > > For the issue of a background task interfering with interactive use, the feel > > arguably matters more than the throughput. > > > > I'll probably put that patch in 1.0.19, and look at moving all the fdatasync() > > calls together. This is http://trac.xapian.org/ticket/426 BTW. > > > > The kernel should be able to handle this workload better though, so I would > > say it was worthwhile to bring up on LKML if you have the energy. It certainly > > isn't just you, as apt-xapian-index seems to trigger it for some Ubuntu users, > > and madduck mentioned it on #notmuch a week or so ago. > > Alright. This issue has been bothering me for a very long time and it's frankly > pretty pathetic how badly the kernel falls apart under this sort of workload. > I just wrote up a message (4b9fa440.12135e0a.7fc8.ffffe745@mx.google.com), so > we'll see what happens. In the past kernel developers have been very eager to > write this issue off as not reproducible enough (perhaps wisely), so if anyone > has anything to say, please contribute it to the thread. > Ext3 fsync related issue is a know problem due to the way journalling is handled in ext3. The solution for that would be data=writeback ( with its loss of data integrity ) or not yet upstreamed data=guarded. Another option would be to try ext4 which should not be impacted that badly by the data=ordered journalled mode -aneesh