From: Don Zickus <dzickus@redhat.com>
To: David Bremner <david@tethera.net>
Cc: notmuch@notmuchmail.org
Subject: Re: performance problems with notmuch new
Date: Mon, 20 Apr 2020 11:25:19 -0400 [thread overview]
Message-ID: <20200420152519.gt73upo6mowhzsca@redhat.com> (raw)
In-Reply-To: <87y2qwsdba.fsf@tethera.net>
On Wed, Apr 15, 2020 at 05:29:13PM -0300, David Bremner wrote:
> Don Zickus <dzickus@redhat.com> writes:
>
> >> runs in about 30s here (i7 4770 / SSD). Replacing --small with --medium
> >> takes about 10M (so a superlinear slowdown in wall clock time, since
> >> that represents a 10x scale-up in the corpus size.).
> >
> > Hmm, for me --small was 35s and --medium was 32 minutes. This is on a
> > i7-9750H / nvme. I would expect numbers similar to yours.
>
> I did another few tests test for --medium and they all take 7-9 minutes,
> depending what else is going on on the machine.
> Here's my breakdown of times (unfortunately a bit of hand editing is
> needed to clean up the warnings)
Sorry for the delay. I re-ran the test today and see similar numbers to
yours now. Roughly 6m30s per run (3 runs).
T00-new.sh: Testing notmuch new [0.4 medium]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
Initial notmuch new 61.86 59.13 2.53 228124 2024/1087672
notmuch new #2 0.01 0.00 0.00 9136 0/160
notmuch new #3 0.00 0.00 0.00 9068 0/8
notmuch new #4 0.00 0.00 0.00 8940 0/8
notmuch new #5 0.00 0.00 0.00 9060 0/8
notmuch new #6 0.00 0.00 0.00 8932 0/8
new (7500 mv) 44.76 32.67 12.02 186156 0/541024
new (7500 mv back) 47.82 34.81 12.91 187240 0/500000
new (7500 cp) 26.41 20.06 6.28 183448 0/378192
T01-dump-restore.sh: Testing dump and restore [0.4 medium]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
3.35 1.82 1.31 11992 624/31976
dump * 0.60 0.57 0.02 11356 0/4344
restore * 0.69 0.62 0.06 9064 0/0
T02-tag.sh: Testing tagging [0.4 medium]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
tag * +new_tag 39.96 24.99 14.74 80048 16/282944
tag * +existing_tag 0.00 0.00 0.00 8968 0/0
tag * -existing_tag 29.98 18.18 11.70 20040 0/208264
tag * -missing_tag 0.00 0.00 0.00 9036 0/0
T03-reindex.sh: Testing reindexing [0.4 medium]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
reindex * 40.56 33.46 7.02 228908 0/459240
reindex * 39.91 33.10 6.75 223768 112/378824
reindex * 40.04 33.10 6.89 223724 0/334824
T04-thread-subquery.sh: Testing thread subqueries [0.4 medium]
Wall(s) Usr(s) Sys(s) Res(K) In/Out(512B)
search thread:{} ... 0.29 0.26 0.02 23256 0/24
search thread:{} ... 0.27 0.25 0.02 23296 0/24
search thread:{} ... 0.29 0.27 0.02 23192 0/24
The only thing I can think of is my fstrim service runs 1x / week on Monday
at midnight and maybe that helped clean things up?? Perhaps I should
increase that frequency or run it manually when things go bad.
Thanks for the feedback!
Cheers,
Don
next prev parent reply other threads:[~2020-04-20 15:25 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-15 15:08 performance problems with notmuch new Don Zickus
2020-04-15 16:01 ` David Bremner
2020-04-15 17:31 ` Don Zickus
2020-04-15 20:29 ` David Bremner
2020-04-20 15:25 ` Don Zickus [this message]
2020-04-24 10:36 ` David Bremner
2020-04-24 19:05 ` Don Zickus
2020-04-24 23:07 ` David Bremner
2020-04-27 12:48 ` Don Zickus
2020-04-27 12:56 ` David Bremner
2020-04-20 14:31 ` David Bremner
2020-04-20 16:26 ` Don Zickus
2020-04-18 11:59 ` Franz Fellner
2020-04-20 14:36 ` David Bremner
2020-04-22 10:46 ` Franz Fellner
2020-04-22 23:21 ` Olly Betts
2020-04-24 18:35 ` Franz Fellner
2020-04-24 23:13 ` David Bremner
2020-04-29 9:30 ` Eric Wong
2020-04-20 15:04 ` Kim Minh Kaplan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200420152519.gt73upo6mowhzsca@redhat.com \
--to=dzickus@redhat.com \
--cc=david@tethera.net \
--cc=notmuch@notmuchmail.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).