Notmuch emacs client is slow when opening inbox.

Notmuch emacs client is slow when opening inbox.

[sam.hedin]

Hello!
While the notmuch emacs client is in general very impressive,
opening my inbox makes Emacs lag terribly.
I believe the reason is that there seems to be
no way to limit the number of results, so
when I search for the inbox tag, there are too many mails
for emacs to display comfortably.
I would like a way to limit the search results when using the emacs client,
but have not found a way to do so.

I found this from 2011, which seems to discuss the feature I want.
However, it was never merged as far as I can see: https://notmuchmail.org/pipermail/notmuch/2011/006297.html

Am I the only one with this issue?
If so, I will have to look through my
setup. If not, how do other people get around it?

Re: Notmuch emacs client is slow when opening inbox.

[Teemu Likonen]

[-- Attachment #1.1: Type: text/plain, Size: 630 bytes --]

* 2022-09-17 12:19:44+0200, sam hedin wrote:

> While the notmuch emacs client is in general very impressive, opening
> my inbox makes Emacs lag terribly. I believe the reason is that there
> seems to be no way to limit the number of results, so when I search
> for the inbox tag, there are too many mails for emacs to display
> comfortably.

I think it's normal to set saved searches so that they have a time
limit, like this:

    to:something  AND date:90days..
    tag:something AND date:90days..

-- 
/// Teemu Likonen - .-.. https://www.iki.fi/tlikonen/
// OpenPGP: 6965F03973F0D4CA22B9410F0F2CAE0E07608462

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Notmuch emacs client is slow when opening inbox.

[Jose A Ortega Ruiz]

On Sat, Sep 17 2022, sam.hedin@gmail.com wrote:

> I found this from 2011, which seems to discuss the feature I want.
> However, it was never merged as far as I can see:
> https://notmuchmail.org/pipermail/notmuch/2011/006297.html

interesting!  if we were to have this, i think i'd prefer pagination
rather than increasing the limit and keeping the old ones, and make it
work for tree searches too.  and perhaps the limit (page size) could be
a query parameter, so that one could have different page sizes for
different queries.  if that makes sense (and i'm not missing a better
way already in place), i might try to implement it when i have a bit of
time.

cheers,
jao
-- 
Keep away from people who try to belittle your ambitions. Small people
always do that, but the really great make you feel that you, too, can
become great. - Mark Twain

Re: Notmuch emacs client is slow when opening inbox.

[Sam Hedin]

[-- Attachment #1.1: Type: text/plain, Size: 516 bytes --]

On Sat, Sep 17, 2022 at 12:46 PM Teemu Likonen wrote:
>    to:something  AND date:90days..
>    tag:something AND date:90days..

That's perfect, thank you. I hadn't even considered a date range.

On Sat, Sep 17, 2022 at 8:01 PM Jose A Ortega Ruiz <mail@jao.io> wrote:

> interesting!  if we were to have this, i think i'd prefer pagination
> rather than increasing the limit and keeping the old ones

Agreed that pagination seems preferrable, though the solution posted above
is everything I wanted from this PR. :)

[-- Attachment #1.2: Type: text/html, Size: 789 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Notmuch emacs client is slow when opening inbox.

[David Bremner]

Jose A Ortega Ruiz <mail@jao.io> writes:
>
> interesting!  if we were to have this, i think i'd prefer pagination
> rather than increasing the limit and keeping the old ones, and make it
> work for tree searches too.  and perhaps the limit (page size) could be
> a query parameter, so that one could have different page sizes for
> different queries.  if that makes sense (and i'm not missing a better
> way already in place), i might try to implement it when i have a bit of
> time.

Before going to far with pagination, maybe get Jani to recap the
difficulties he saw at the time that led him to propose the simpler
approach.

I can think of a few issues, but I don't really know if they are
blockers.
- the result set is going to vary dynamically as messages are added and
  removed from the database

- As far as I can tell, there isn't really an easy way to jump to page n of the results.

Re: Notmuch emacs client is slow when opening inbox.

[Jose A Ortega Ruiz]

On Sun, Sep 18 2022, David Bremner wrote:

> Jose A Ortega Ruiz <mail@jao.io> writes:
>>
>> interesting!  if we were to have this, i think i'd prefer pagination
>> rather than increasing the limit and keeping the old ones, and make it
>> work for tree searches too.  and perhaps the limit (page size) could be
>> a query parameter, so that one could have different page sizes for
>> different queries.  if that makes sense (and i'm not missing a better
>> way already in place), i might try to implement it when i have a bit of
>> time.
>
> Before going to far with pagination, maybe get Jani to recap the
> difficulties he saw at the time that led him to propose the simpler
> approach.
>
> I can think of a few issues, but I don't really know if they are
> blockers.
> - the result set is going to vary dynamically as messages are added and
>   removed from the database
>
> - As far as I can tell, there isn't really an easy way to jump to page
> n of the results.

fwiw, my idea here was to always run the full, non-paginated query, and
populate its result buffer just as now, but show, instead of that
buffer, an indirect buffer pointing to it, narrowed to the page at hand.

in that scenario, one can always compute easily how many pages of output
are available so far, and know whether there are more to come (and ask
the user to wait a bit).  my feeling is that, in most cases, by the time
the user has perused one or two pages, all the results will already be
available and it'll be just a matter of changing narrow regions.

this also avoids the complication that changing tags as one reads
messages would also, in general, change the result set.

not as efficient as a limit/offset query per page, but i think the
bottleneck here is in displaying results rather than running the query
in most cases, isn't it?

jao
-- 
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -Georges W. Bush

Re: Notmuch emacs client is slow when opening inbox.

[David Bremner]

Jose A Ortega Ruiz <jao@gnu.org> writes:

> On Sun, Sep 18 2022, David Bremner wrote:
>
>> Jose A Ortega Ruiz <mail@jao.io> writes:
>>>
>>> interesting!  if we were to have this, i think i'd prefer pagination
>>> rather than increasing the limit and keeping the old ones, and make it
>>> work for tree searches too.  and perhaps the limit (page size) could be
>>> a query parameter, so that one could have different page sizes for
>>> different queries.  if that makes sense (and i'm not missing a better
>>> way already in place), i might try to implement it when i have a bit of
>>> time.
>>
>> Before going to far with pagination, maybe get Jani to recap the
>> difficulties he saw at the time that led him to propose the simpler
>> approach.
>>
>> I can think of a few issues, but I don't really know if they are
>> blockers.
>> - the result set is going to vary dynamically as messages are added and
>>   removed from the database
>>
>> - As far as I can tell, there isn't really an easy way to jump to page
>> n of the results.
>
> fwiw, my idea here was to always run the full, non-paginated query, and
> populate its result buffer just as now, but show, instead of that
> buffer, an indirect buffer pointing to it, narrowed to the page at hand.
[snip]
> not as efficient as a limit/offset query per page, but i think the
> bottleneck here is in displaying results rather than running the query
> in most cases, isn't it?

Ah, indeed that sounds like a different approach than was previously
mooted. I guess it might be a simple-ish way of making the display more
lazy, which I agree is most likely the bottleneck most people are seeing.