emacs slow with large threads

emacs slow with large threads

[Landry, Walter]

Hello Everyone,

I am using the emacs frontend to notmuch.  It has mostly been a pleasant
experience, but I am having a problem with large threads.  Essentially,
when I try to view a large thread, the machine locks up for many
minutes.  The problem seems very similar to these posts.

  https://notmuchmail.org/pipermail/notmuch/2013/014811.html
  https://notmuchmail.org/pipermail/notmuch/2015/020379.html

I tried turning off html rendering by setting mm-text-html-renderer to
nil.  That helped, but it is still taking at least 10 minutes to render
a thread.  I killed it when I ran out of patience.

The thread has 231 messages, and running

  notmuch show thread:000000000000d637 | wc -l

shows that it is 46918 lines long.  Running that on the command line is
fast, taking 0.123 seconds.  As a comparison, I tried opening the thread
with astroid.  It was not instantaneous, but it only took about 3
seconds.

I am guessing that the emacs mode is trying to process the result.  I
can work around this a bit by opening individual messages with "C-u RET"
instead of "RET".  But then I lose context.

Is there anything else I can do to make this work?

Thanks,
Walter Landry

Re: emacs slow with large threads

[Dan Čermák]

Hi Landry,

I am afraid this is a common limitation of emacs: if you start opening
large files (especially with long lines) it becomes very slow.

A viable workaround is to open the offending thread in the tree view
(bound to 'z' by default) and then only view individual messages.


Cheers,

Dan

"Landry, Walter" <wlandry@caltech.edu> writes:

> Hello Everyone,
>
> I am using the emacs frontend to notmuch.  It has mostly been a pleasant
> experience, but I am having a problem with large threads.  Essentially,
> when I try to view a large thread, the machine locks up for many
> minutes.  The problem seems very similar to these posts.
>
>   https://notmuchmail.org/pipermail/notmuch/2013/014811.html
>   https://notmuchmail.org/pipermail/notmuch/2015/020379.html
>
> I tried turning off html rendering by setting mm-text-html-renderer to
> nil.  That helped, but it is still taking at least 10 minutes to render
> a thread.  I killed it when I ran out of patience.
>
> The thread has 231 messages, and running
>
>   notmuch show thread:000000000000d637 | wc -l
>
> shows that it is 46918 lines long.  Running that on the command line is
> fast, taking 0.123 seconds.  As a comparison, I tried opening the thread
> with astroid.  It was not instantaneous, but it only took about 3
> seconds.
>
> I am guessing that the emacs mode is trying to process the result.  I
> can work around this a bit by opening individual messages with "C-u RET"
> instead of "RET".  But then I lose context.
>
> Is there anything else I can do to make this work?
>
> Thanks,
> Walter Landry
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: emacs slow with large threads

[Brian Sniffen]

This is much worse than normal problems with large files or long lines. There’s something worse-than-quadratic in the nesting and indenting elisp. I’ve been meaning to get around to figuring out what, but no luck yet. 

-- 
Brian Sniffen

> On Jan 18, 2019, at 7:03 PM, Landry, Walter <wlandry@caltech.edu> wrote:
> 
> Hello Everyone,
> 
> I am using the emacs frontend to notmuch.  It has mostly been a pleasant
> experience, but I am having a problem with large threads.  Essentially,
> when I try to view a large thread, the machine locks up for many
> minutes.  The problem seems very similar to these posts.
> 
>  https://notmuchmail.org/pipermail/notmuch/2013/014811.html
>  https://notmuchmail.org/pipermail/notmuch/2015/020379.html
> 
> I tried turning off html rendering by setting mm-text-html-renderer to
> nil.  That helped, but it is still taking at least 10 minutes to render
> a thread.  I killed it when I ran out of patience.
> 
> The thread has 231 messages, and running
> 
>  notmuch show thread:000000000000d637 | wc -l
> 
> shows that it is 46918 lines long.  Running that on the command line is
> fast, taking 0.123 seconds.  As a comparison, I tried opening the thread
> with astroid.  It was not instantaneous, but it only took about 3
> seconds.
> 
> I am guessing that the emacs mode is trying to process the result.  I
> can work around this a bit by opening individual messages with "C-u RET"
> instead of "RET".  But then I lose context.
> 
> Is there anything else I can do to make this work?
> 
> Thanks,
> Walter Landry
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: emacs slow with large threads

[David Edmondson]

On Saturday, 2019-01-19 at 00:03:18 +00, Landry, Walter wrote:

> Hello Everyone,
>
> I am using the emacs frontend to notmuch.  It has mostly been a pleasant
> experience, but I am having a problem with large threads.  Essentially,
> when I try to view a large thread, the machine locks up for many
> minutes.  The problem seems very similar to these posts.
>
>   https://notmuchmail.org/pipermail/notmuch/2013/014811.html
>   https://notmuchmail.org/pipermail/notmuch/2015/020379.html
>
> I tried turning off html rendering by setting mm-text-html-renderer to
> nil.  That helped, but it is still taking at least 10 minutes to render
> a thread.  I killed it when I ran out of patience.
>
> The thread has 231 messages, and running
>
>   notmuch show thread:000000000000d637 | wc -l
>
> shows that it is 46918 lines long.  Running that on the command line is
> fast, taking 0.123 seconds.  As a comparison, I tried opening the thread
> with astroid.  It was not instantaneous, but it only took about 3
> seconds.
>
> I am guessing that the emacs mode is trying to process the result.  I
> can work around this a bit by opening individual messages with "C-u RET"
> instead of "RET".  But then I lose context.
>
> Is there anything else I can do to make this work?

Do things improve if you set notmuch-show-insert-text/plain-hook to nil?

Can you share the messages in the thread?

dme.
-- 
Sorry I'm not home right now, I'm walking into spiderwebs.

Re: emacs slow with large threads

[Landry, Walter]

David Edmondson <dme@dme.org> writes:
> Do things improve if you set notmuch-show-insert-text/plain-hook to nil?

That helps a great deal!  The key option to turn off is

     notmuch-wash-wrap-long-lines

That makes it take only 10 seconds.  That is livable, but not ideal.

> Can you share the messages in the thread?

Unfortunately, no.

Thanks,
Walter Landry

Re: emacs slow with large threads

[Landry, Walter]

Dan Čermák <dan.cermak@posteo.net> writes:

> Hi Landry,
>
> I am afraid this is a common limitation of emacs: if you start opening
> large files (especially with long lines) it becomes very slow.
>
> A viable workaround is to open the offending thread in the tree view
> (bound to 'z' by default) and then only view individual messages.

Tree view works well [1].  Everything is fast, and I kind of prefer that
view anyway.

Thanks,
Walter Landry

[1] I have been using notmuch for almost a year, and I did not realize
    that there is a separate tree view.

Re: emacs slow with large threads

[Dan Čermák]

"Landry, Walter" <wlandry@caltech.edu> writes:

> Dan Čermák <dan.cermak@posteo.net> writes:
>
>> Hi Landry,
>>
>> I am afraid this is a common limitation of emacs: if you start opening
>> large files (especially with long lines) it becomes very slow.
>>
>> A viable workaround is to open the offending thread in the tree view
>> (bound to 'z' by default) and then only view individual messages.
>
> Tree view works well [1].  Everything is fast, and I kind of prefer that
> view anyway.
>
> Thanks,
> Walter Landry
>
> [1] I have been using notmuch for almost a year, and I did not realize
>     that there is a separate tree view.

I have been using notmuch for about two years and iirc didn't realize
tree view existed until about half a year ago. Should really have read
the manual and all the key bindings at first.

> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: emacs slow with large threads

[Emilio Francesquini]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

Hello there,

I recently ran into the same (slowness) problem when viewing long threads.
After some profiling I found out that disabling indentation did the trick
for me. In my case it went from unusable (>5 minutes) to ~3 seconds for
large threads.

(setq notmuch-show-indent-content nil)

Hope it helps.

On Mon, Jan 21, 2019 at 9:07 AM Dan Čermák <dan.cermak@posteo.net> wrote:

> "Landry, Walter" <wlandry@caltech.edu> writes:
>
> > Dan Čermák <dan.cermak@posteo.net> writes:
> >
> >> Hi Landry,
> >>
> >> I am afraid this is a common limitation of emacs: if you start opening
> >> large files (especially with long lines) it becomes very slow.
> >>
> >> A viable workaround is to open the offending thread in the tree view
> >> (bound to 'z' by default) and then only view individual messages.
> >
> > Tree view works well [1].  Everything is fast, and I kind of prefer that
> > view anyway.
> >
> > Thanks,
> > Walter Landry
> >
> > [1] I have been using notmuch for almost a year, and I did not realize
> >     that there is a separate tree view.
>
> I have been using notmuch for about two years and iirc didn't realize
> tree view existed until about half a year ago. Should really have read
> the manual and all the key bindings at first.
>
> > _______________________________________________
> > notmuch mailing list
> > notmuch@notmuchmail.org
> > https://notmuchmail.org/mailman/listinfo/notmuch
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
>

[-- Attachment #2: Type: text/html, Size: 2592 bytes --]

Re: emacs slow with large threads

[Brian Sniffen]

[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]

I think that becomes buffer local when set. Are you sure it me a difference?

-- 
Brian Sniffen

> On Jan 21, 2019, at 7:39 AM, Emilio Francesquini <francesquini@gmail.com> wrote:
> 
> Hello there,
> 
> I recently ran into the same (slowness) problem when viewing long threads.
> After some profiling I found out that disabling indentation did the trick for me. In my case it went from unusable (>5 minutes) to ~3 seconds for large threads.
> 
> (setq notmuch-show-indent-content nil)
> 
> Hope it helps.
> 
>> On Mon, Jan 21, 2019 at 9:07 AM Dan Čermák <dan.cermak@posteo.net> wrote:
>> "Landry, Walter" <wlandry@caltech.edu> writes:
>> 
>> > Dan Čermák <dan.cermak@posteo.net> writes:
>> >
>> >> Hi Landry,
>> >>
>> >> I am afraid this is a common limitation of emacs: if you start opening
>> >> large files (especially with long lines) it becomes very slow.
>> >>
>> >> A viable workaround is to open the offending thread in the tree view
>> >> (bound to 'z' by default) and then only view individual messages.
>> >
>> > Tree view works well [1].  Everything is fast, and I kind of prefer that
>> > view anyway.
>> >
>> > Thanks,
>> > Walter Landry
>> >
>> > [1] I have been using notmuch for almost a year, and I did not realize
>> >     that there is a separate tree view.
>> 
>> I have been using notmuch for about two years and iirc didn't realize
>> tree view existed until about half a year ago. Should really have read
>> the manual and all the key bindings at first.
>> 
>> > _______________________________________________
>> > notmuch mailing list
>> > notmuch@notmuchmail.org
>> > https://notmuchmail.org/mailman/listinfo/notmuch
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

[-- Attachment #2: Type: text/html, Size: 3457 bytes --]

Re: emacs slow with large threads

[Emilio Francesquini]

You are absolutely right. It does become buffer local. Yet setting it to
nil changes the behavior of new windows on notmuch-show-mode. I'm at a
loss why... :) Maybe because I set it inside my .emacs? I need to study
a bit more of elisp...

Anyway, my previous message was actually incomplete. This change alone
accounted only for a marginal improvement in performance. I restored my
old .emacs from git and realized that the real culprit was fci-mode (
fill-column-indicator package) *even if it was not enabled on the
message buffer*. Removing the package did the trick. For a thread with
~70 messages it took ~20 seconds against less than 1 without the package.


On Tue, Jan 22 2019, Brian Sniffen wrote:

> I think that becomes buffer local when set. Are you sure it me a difference?
>
> -- 
> Brian Sniffen
>
>> On Jan 21, 2019, at 7:39 AM, Emilio Francesquini <francesquini@gmail.com> wrote:
>> 
>> Hello there,
>> 
>> I recently ran into the same (slowness) problem when viewing long threads.
>> After some profiling I found out that disabling indentation did the trick for me. In my case it went from unusable (>5 minutes) to ~3 seconds for large threads.
>> 
>> (setq notmuch-show-indent-content nil)
>> 
>> Hope it helps.
>> 
>>> On Mon, Jan 21, 2019 at 9:07 AM Dan Čermák <dan.cermak@posteo.net> wrote:
>>> "Landry, Walter" <wlandry@caltech.edu> writes:
>>> 
>>> > Dan Čermák <dan.cermak@posteo.net> writes:
>>> >
>>> >> Hi Landry,
>>> >>
>>> >> I am afraid this is a common limitation of emacs: if you start opening
>>> >> large files (especially with long lines) it becomes very slow.
>>> >>
>>> >> A viable workaround is to open the offending thread in the tree view
>>> >> (bound to 'z' by default) and then only view individual messages.
>>> >
>>> > Tree view works well [1].  Everything is fast, and I kind of prefer that
>>> > view anyway.
>>> >
>>> > Thanks,
>>> > Walter Landry
>>> >
>>> > [1] I have been using notmuch for almost a year, and I did not realize
>>> >     that there is a separate tree view.
>>> 
>>> I have been using notmuch for about two years and iirc didn't realize
>>> tree view existed until about half a year ago. Should really have read
>>> the manual and all the key bindings at first.
>>> 
>>> > _______________________________________________
>>> > notmuch mailing list
>>> > notmuch@notmuchmail.org
>>> > https://notmuchmail.org/mailman/listinfo/notmuch
>>> _______________________________________________
>>> notmuch mailing list
>>> notmuch@notmuchmail.org
>>> https://notmuchmail.org/mailman/listinfo/notmuch
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch

Re: emacs slow with large threads

[Emilio Francesquini]

On Tue, Jan 22 2019, Emilio Francesquini wrote:

> You are absolutely right. It does become buffer local. Yet setting it to
> nil changes the behavior of new windows on notmuch-show-mode. I'm at a
> loss why... :) Maybe because I set it inside my .emacs? I need to study
> a bit more of elisp...

About that, forget it. It was something else changing the
behavior. Indeed I need to study more elisp. :p Sorry for the noise.

The real culprit was indeed fci-mode.