From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id AE001431FBD for ; Sat, 12 Apr 2014 08:41:27 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a0YmBTZ+V4d5 for ; Sat, 12 Apr 2014 08:41:21 -0700 (PDT) Received: from yantan.tethera.net (yantan.tethera.net [199.188.72.155]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 43C76431FBC for ; Sat, 12 Apr 2014 08:41:21 -0700 (PDT) Received: from remotemail by yantan.tethera.net with local (Exim 4.80) (envelope-from ) id 1WZ03P-00044j-Tg; Sat, 12 Apr 2014 12:41:15 -0300 Received: (nullmailer pid 12090 invoked by uid 1000); Sat, 12 Apr 2014 15:41:11 -0000 From: David Bremner To: Nils Dagsson Moskopp , notmuch@notmuchmail.org Subject: Re: notmuch-hello buffer slow doe to slow query In-Reply-To: <874n20go84.fsf@dieweltistgarnichtso.net> References: <874n20go84.fsf@dieweltistgarnichtso.net> User-Agent: Notmuch/0.17+180~g8977b1a (http://notmuchmail.org) Emacs/24.3.1 (x86_64-pc-linux-gnu) Date: Sat, 12 Apr 2014 12:41:11 -0300 Message-ID: <87ppkm4ktk.fsf@zancas.localnet> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 15:41:27 -0000 --=-=-= Content-Type: text/plain Nils Dagsson Moskopp writes: > > mjw1009 can reproduce if the from-query contains an "@" and thinks the > problem may be "something deeper down in notmuch (actually probably in > xapian)". My own experiments contradict that (see attached patch for a test to run). The exact address is about twice as fast to count on both of my machines, which matches the amount of I/O. FWIW, my SSD based laptop is about 10x faster (compared to a similarly specced desktop with HD) for both of them, which kindof reinforces how I/O bound this is. Replacing 'count' with 'search' gives pretty similar relative behaviour, although much slower since it seems to do about 10x I/O. Finally after an initial "notmuch tag +fromlinus from:torvalds@linux-foundation.org", "notmuch count tag:fromlinus" seems to be about 5-10x faster than "notmuch count from:torvalds@linux-foundation.org". So that might be a workaround for some people. To sum up, I think the underlying question is whether the amount of I/O that count does can be reduced when using non-boolean query terms. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-WIP-new-performance-test-for-notmuch-count.patch >From d3fbbe7ac0a1fb5b5691674226a3a3d000e15a1c Mon Sep 17 00:00:00 2001 From: David Bremner Date: Sat, 12 Apr 2014 11:50:22 -0300 Subject: [PATCH] WIP: new performance test for notmuch count Probably not suitable for production because of sudo --- performance-test/T03-count.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 performance-test/T03-count.sh diff --git a/performance-test/T03-count.sh b/performance-test/T03-count.sh new file mode 100755 index 0000000..e74437e --- /dev/null +++ b/performance-test/T03-count.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +test_description='counting' + +. ./perf-test-lib.sh + +time_start + +for i in $(seq 1 10); do + echo 3 | sudo tee /proc/sys/vm/drop_caches + notmuch --version > /dev/null + time_run "from:torvalds@linux-foundation.org " \ + "notmuch count from:torvalds@linux-foundation.org" + + echo 3 | sudo tee /proc/sys/vm/drop_caches + notmuch --version > /dev/null + + time_run "from:linux-foundation.org " \ + "notmuch count from:linux-foundation.org" +done + + +time_done -- 1.9.1 --=-=-=--