From: David Bremner <david@tethera.net>
To: Austin Clements <amdragon@MIT.EDU>
Cc: notmuch@notmuchmail.org
Subject: Re: [Patch v4 2/2] test: initial performance testing infrastructure
Date: Sun, 25 Nov 2012 20:05:42 -0400 [thread overview]
Message-ID: <87obilut2h.fsf@zancas.localnet> (raw)
In-Reply-To: <20121125214023.GO4562@mit.edu>
Austin Clements <amdragon@MIT.EDU> writes:
>> +subdirs := compat completion emacs lib man parse-time-string
>> +subdirs := $(subdirs) performance-test util test
> += ?
>
Sure.
>> +CORPUS_NAME := notmuch-email-corpus-$(PERFTEST_VERSION).tar.xz
>
> Would it make sense to split out the different size corpora so a user
> could, say, only download the small one?
Currently the choice of test is local to given test file; one doing
something particularly intense (or just lots of repetitions) might want
to only use a subset. So I'm not sure if separate downloading of smaller
corpora makes sense. This is all hypothetical at the moment, since the
one test file uses the full corpus.
> "\nPlease download ${TXZFILE} using\n\n"?
OK
>> +add_email_corpus takes arguments "--small" and "--medium" for when you
>> +want smaller corpuses to check.
>
> "corpora"?
reworded to say
,----
| add_email_corpus takes arguments "--small" and "--medium" for when you
| want smaller subsets of the corpus to check.
`----
>
> I'm a bit confused by this. What happens if you don't specify --small
> or --medium? Is the "large"/default corpus just the combined small
> and medium corpora? Would be worth a comment, at least.
Hopefully the README makes this clear(er) now?
> This probably doesn't matter now, but I wonder if we want to unpack on
> first use to somewhere not test-specific and then cp -rl the corpus
> into the test directory. I haven't tried unpacking the corpus yet,
> but if you're running tests repeatedly to compare results, or running
> more than one performance test, it seems like a full decompress and
> unpack could get onerous.
Hmm. On my machine it is 10s for the copy versus 45s for a full
unpack. For some reason I tested with "cp -a" which is incredibly slow,
so I thought there was no loss. For comparison the basic test takes
about 10 minutes on the same machine.
In any case this can wait until we have a second test file and a second
call to add_mail_corpus, adding caching now would not help.
next prev parent reply other threads:[~2012-11-26 0:05 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-25 15:02 v4 of Performance tests david
2012-11-25 15:02 ` [Patch v4 1/2] test: factor out part of test-lib.sh into test-lib-common.sh david
2012-11-25 15:02 ` [Patch v4 2/2] test: initial performance testing infrastructure david
2012-11-25 21:40 ` Austin Clements
2012-11-26 0:05 ` David Bremner [this message]
2012-11-26 3:29 ` Austin Clements
2012-11-26 12:44 ` David Bremner
2012-11-25 21:19 ` v4 of Performance tests Tomi Ollila
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87obilut2h.fsf@zancas.localnet \
--to=david@tethera.net \
--cc=amdragon@MIT.EDU \
--cc=notmuch@notmuchmail.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).