Re: performance problems with notmuch new

[Franz Fellner <alpine.art.de@gmail.com>]

On Thu Apr 23 00:21:30 2020, Olly Betts <olly@survex.com> wrote:
> First question: what version of Xapian are you using?

On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux)

> And second thing to check, are you committing each message separately?

No, I sync with mbsync which dosnloads a bunch of mails,
then I run notmuch new which indexes all in one go.

> After reboot the disk cache won't have any of the database in, so the
> first operation will typically be slower, especially with a spinning
> drive where seeks are relatively slow.

Yes, I know that, I just wanted to mention the number, which IMO is insane.
I want to setup notmuch for my dad on the desktop PC.
5 minutes to wait for his mail in the morning would have made notmuch a no-go.
 
> It sounds like you're seek-limited in this "cold cache" phase.  That is
> not necessarily related to the slow indexing, but it could be.
> 
> I'd check the SMART diagnostics for the drive first (e.g. with
> smartctl).  It's not the most likely cause, but it's quick to check and
> if the drive is starting to fail it's better to find out sooner rather
> than later.

HDDs are healthy. I actually checked quite recently when converting 
the laptop from Gentoo to arch.

> 
> Then I'd try compacting the database (I think there's a "notmuch
> compact" subcommand to do this).

And there we go. Cured the issues.
Dropped the very first indexing from several minutes to 1.5 seconds on the desktop.
?!?!
This is a really new setup and I suffered from bad performance from the
very first notmuch new after the initial indexing.
Is it really needed to run notmch compact directly after the initial notmuch new?
Desktop currently has 38502 messages indexed, in case that matters.

Regards
Franz