From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 842CB431FAF for ; Sun, 25 Nov 2012 19:29:12 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fKGrvirsBoZr for ; Sun, 25 Nov 2012 19:29:11 -0800 (PST) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by olra.theworths.org (Postfix) with ESMTP id C952C431FAE for ; Sun, 25 Nov 2012 19:29:11 -0800 (PST) X-AuditID: 12074424-b7fbe6d000003b02-b3-50b2e207325b Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id 77.C9.15106.702E2B05; Sun, 25 Nov 2012 22:29:11 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id qAQ3T99E011765; Sun, 25 Nov 2012 22:29:10 -0500 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id qAQ3T7B2016560 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Sun, 25 Nov 2012 22:29:08 -0500 (EST) Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) (envelope-from ) id 1TcpNb-0005wO-0a; Sun, 25 Nov 2012 22:29:07 -0500 Date: Sun, 25 Nov 2012 22:29:06 -0500 From: Austin Clements To: David Bremner Subject: Re: [Patch v4 2/2] test: initial performance testing infrastructure Message-ID: <20121126032906.GP4562@mit.edu> References: <1353855745-11697-1-git-send-email-david@tethera.net> <1353855745-11697-3-git-send-email-david@tethera.net> <20121125214023.GO4562@mit.edu> <87obilut2h.fsf@zancas.localnet> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87obilut2h.fsf@zancas.localnet> User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpileLIzCtJLcpLzFFi42IRYrdT0WV/tCnAYNU7YYsbrd2MFtdvzmR2 YPJ4tuoWs8eWQ++ZA5iiuGxSUnMyy1KL9O0SuDLa/zYxFjzgr+hbu5+pgXEVTxcjJ4eEgInE 9Uvz2SFsMYkL99azgdhCAvsYJba2hXUxcgHZGxgltk89xALhXGSSmNlxhwmiagmjxKdjIV2M HBwsAqoSux/pg4TZBDQktu1fzghiiwCFr26bDDaUWUBa4tvvZrBWYQEfiXO7ZoHZvALaErcm fmOEmA+0eMOl42wQCUGJkzOfsEA0a0nc+PeSCWQXyKDl/zhAwpwCuhLbTi0FKxEVUJGYcnIb 2wRGoVlIumch6Z6F0L2AkXkVo2xKbpVubmJmTnFqsm5xcmJeXmqRrrlebmaJXmpK6SZGcFC7 qOxgbD6kdIhRgINRiYdX4timACHWxLLiytxDjJIcTEqivJfuA4X4kvJTKjMSizPii0pzUosP MUpwMCuJ8E5kAsrxpiRWVqUW5cOkpDlYlMR5r6fc9BcSSE8sSc1OTS1ILYLJynBwKEnwsj4E ahQsSk1PrUjLzClBSDNxcIIM5wEaLglSw1tckJhbnJkOkT/FqCglzqsEkhAASWSU5sH1wpLO K0ZxoFeEeTVBqniACQuu+xXQYCagwcnXN4IMLklESEk1MEbkq9vcebBv0pycr0+yzkkmWPe5 b/3yoWdyLK9730uFZr//LaqTLu2++PX9pFOuOxl2dTw/pXeHt6bqYPB/jc1n3i+UmbjZLTrS v3rNrPbT+cXTr00LMNy23vvCUfdV4U57kxwWvq902vVDwPgS09cn8+67bfv9nlOi6lfrtl85 6tJhB6539C9TYinOSDTUYi4qTgQAaPnSUxUDAAA= Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2012 03:29:12 -0000 Quoth David Bremner on Nov 25 at 8:05 pm: > Austin Clements writes: > >> +add_email_corpus takes arguments "--small" and "--medium" for when you > >> +want smaller corpuses to check. > > > > "corpora"? > > reworded to say > > ,---- > | add_email_corpus takes arguments "--small" and "--medium" for when you > | want smaller subsets of the corpus to check. > `---- That's clearer. > > > > I'm a bit confused by this. What happens if you don't specify --small > > or --medium? Is the "large"/default corpus just the combined small > > and medium corpora? Would be worth a comment, at least. > > Hopefully the README makes this clear(er) now? The README definitely helps. Might still be worth a comment in the code since it took me some thinking to realize it would do something reasonable when given no argument. Perhaps above the initial assignment of arg, # With no argument, use the entire (combined) corpus to acknowledge that this is a legitimate and intentional code path? > > This probably doesn't matter now, but I wonder if we want to unpack on > > first use to somewhere not test-specific and then cp -rl the corpus > > into the test directory. I haven't tried unpacking the corpus yet, > > but if you're running tests repeatedly to compare results, or running > > more than one performance test, it seems like a full decompress and > > unpack could get onerous. > > Hmm. On my machine it is 10s for the copy versus 45s for a full > unpack. For some reason I tested with "cp -a" which is incredibly slow, > so I thought there was no loss. For comparison the basic test takes > about 10 minutes on the same machine. > > In any case this can wait until we have a second test file and a second > call to add_mail_corpus, adding caching now would not help. It would help (a little) if you run basic multiple times. I think it's completely reasonable to leave it as is for now and see if caching would help down the road.