Performance-issues with large threads

Performance-issues with large threads

[Franz Fellner]

Hi,

"Large" in the context of this issue is >20 replies, I have to wait ~1 sec until the thread shows up. With larger conversations it get's worse (a bug report on bugs.kde.org with ~400 replies takes ~13 secs. to load).
This is due to the fact, that notmuch-{vim,emacs} load the complete mail including the body.
ner does a great job here, it loads all headers but only the body of the first unread mail, resulting in instant loading even of gigantic conversations. It would be terrific to see something like that, at least in my favored vim-client ;)

Thx
Franz

Re: Performance-issues with large threads

[Mark Walters]

Hi

Sorry this will be a big list of questions but they might help narrow
down the problem. 

How long does 

notmuch show --format=json thread:thread-id  > /dev/null

take? Is it much faster after the first run? What about the same with
the added cli option --body=false?

(you probably want something like  echo 3 > /proc/sys/vm/drop_caches
between batches of runs)

Do the messages have html parts?

Also what disk are you using (and does it have any other significant
activity happening)? And what version of notmuch are you using?

Best wishes

Mark






Franz Fellner <alpine.art.de@gmail.com> writes:

> Hi,
>
> "Large" in the context of this issue is >20 replies, I have to wait ~1 sec until the thread shows up. With larger conversations it get's worse (a bug report on bugs.kde.org with ~400 replies takes ~13 secs. to load).
> This is due to the fact, that notmuch-{vim,emacs} load the complete mail including the body.
> ner does a great job here, it loads all headers but only the body of the first unread mail, resulting in instant loading even of gigantic conversations. It would be terrific to see something like that, at least in my favored vim-client ;)
>
> Thx
> Franz
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: Performance-issues with large threads

[Franz Fellner]

Hej,

> How long does 
>
> notmuch show --format=json thread:thread-id  > /dev/null
>
> take? Is it much faster after the first run? What about the same with
> the added cli option --body=false?

[7:23] $ time notmuch show --format=json thread:00000000000002d1 >|/dev/null
notmuch show --format=json thread:00000000000002d1 >| /dev/null  0,33s user 0,09s system 8% cpu 4,571 total
[7:25] $ time notmuch show --format=json thread:00000000000002d1 >|/dev/null
notmuch show --format=json thread:00000000000002d1 >| /dev/null  0,12s user 0,02s system 99% cpu 0,139 total
[7:25] $ time notmuch show --format=json --body=false thread:00000000000002d1 >|/dev/null
notmuch show --format=json --body=false thread:00000000000002d1 >| /dev/null  0,30s user 0,08s system 8% cpu 4,446 total
[7:25] $ time notmuch show --format=json --body=false thread:00000000000002d1 >|/dev/null
notmuch show --format=json --body=false thread:00000000000002d1 >| /dev/null  0,10s user 0,01s system 99% cpu 0,119 total
[7:26] $

So fetching without cache takes 4.5 secs, with only 0.1 -> The issue could be in parsing the result inside notmuch-vim/alot/...

> Do the messages have html parts?
Yes, most of them do (unless you reply by mail and only use plaintext).

> Also what disk are you using (and does it have any other significant
> activity happening)? And what version of notmuch are you using?
I use a HDD with 5400 RPM. The most interesting activity is ~80-100% CPU usage during loading, all the 13 secs - quite a battery killer ;)
I regularly build from latest git master (due to the vim-client).

>
> Best wishes
>
> Mark
>
>
>
>
>
>
> Franz Fellner <alpine.art.de@gmail.com> writes:
>
>> Hi,
>> 
>> "Large" in the context of this issue is >20 replies, I have to 
>> wait ~1 sec until the thread shows up. With larger 
>> conversations it get's worse (a bug report on bugs.kde.org with 
>> ~400 replies takes ~13 secs. to load).
>> This is due to the fact, that notmuch-{vim,emacs} load the 
>> complete mail including the body.
>> ner does a great job here, it loads all headers but only the 
>> body of the first unread mail, resulting in instant loading 
>> even of gigantic conversations. It would be terrific to see 
>> something like that, at least in my favored vim-client ;)
>> 
>> Thx ...
>
>