From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 5D4F26DE02B5 for ; Thu, 7 Apr 2016 17:57:35 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -2.377 X-Spam-Level: X-Spam-Status: No, score=-2.377 tagged_above=-999 required=5 tests=[AWL=-0.076, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7JG4OhwXFuzJ for ; Thu, 7 Apr 2016 17:57:27 -0700 (PDT) Received: from atreus.tartarus.org (atreus.tartarus.org [80.252.125.10]) by arlo.cworth.org (Postfix) with ESMTPS id 8387A6DE0134 for ; Thu, 7 Apr 2016 17:57:27 -0700 (PDT) Received: from olly by atreus.tartarus.org with local (Exim 4.69) (envelope-from ) id 1aoKjl-0001nA-D6; Fri, 08 Apr 2016 01:57:25 +0100 Date: Fri, 8 Apr 2016 01:57:25 +0100 From: Olly Betts To: David Bremner Cc: notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org Subject: Re: slowdown in notmuch perf suite with xapian 1.3.5 Message-ID: <20160408005725.GA3037@survex.com> Reply-To: Xapian Discussion Mail-Followup-To: David Bremner , notmuch@notmuchmail.org, xapian-discuss@lists.xapian.org References: <87twjd639d.fsf@zancas.localnet> <20160407232537.GB29434@survex.com> <87h9fd53vo.fsf@zancas.localnet> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87h9fd53vo.fsf@zancas.localnet> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2016 00:57:35 -0000 On Thu, Apr 07, 2016 at 09:40:59PM -0300, David Bremner wrote: > Olly Betts writes: > > > > > So the T00-new.sh numbers make sense - there's more work to do, and > > we need to read existing positional data more to insert the new stuff, > > so the increased reads and writes make sense. > > > > But guessing at what the other two tests do, I wouldn't expect them to > > be affected by this. > > The non-optimized-away cases of T02-tag just adding and deleting terms > to each document with term Tmail That should short-cut to just only changing the data for Tmail. Perhaps that's not working correctly - I'll take a look at this, but probably after 1.4.0 is out. > > I'm also a bit puzzled by how glass can manage not to read any data > > for "dump *", and several tests seem to not read or write anything > > for either backend. What exactly are the "In/Out" numbers? > > that's just the output from /usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' > > The manual describes them as "number of file system > inputs/outputs". From looking at the source, they correspond to > ru_inblock and ru_oublock fields from the getrusage call. AFAIU, that > means the number of non-cached read/writes. Non-cached reads/writes are arguably the most useful sort to measure, but the reads at least will be sensitive to OS caching, which means a repeat run will generally show lower numbers of reads, e.g.: $ /usr/bin/time -f '%I/%O' wc randomfile 240 2908 96780 randomfile 192/0 $ /usr/bin/time -f '%I/%O' wc randomfile 240 2908 96780 randomfile 0/0 So those numbers may not be entirely comparable, depending what order your tests were done in, and whether you'd run the tests (or cloned the repo or some other operation which read or wrote the files used) recently enough that their data might still be cached. Cheers, Olly