bug#63842: 30.0.50; Slow 'gnus-summary-refer-thread'

[Manuel Giraud via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Andrew Cohen <cohen@bu.edu> writes:

> Sorry, I have gotten busy with other things at the moment.

Hi Andrew and you don't need to be sorry for this :-)

>>>>>> "MG" == Manuel Giraud <manuel@ledu-giraud.fr> writes:
>
>     MG> Hi, So here is the crux of this issue.  When using
>     MG> 'gnus-summary-refer-thread' in a nnml group, Emacs ends up
>     MG> calling 'gnus-get-newsgroup-headers-xover' (via
>     MG> 'gnus-fetch-headers').  AFAIU in this function when
>     MG> 'gnus-read-all-available-headers' is t, Emacs will parse *all*
>     MG> of the " *nntp*" buffer content.  In my case, this buffer is
>     MG> quite big (about 50k lines and 23MiB) hence the slowness.
>
> Thanks for continuing to debug this. I am confused---why is the nntp
> buffer so full?

I think in a nnml group the nntp buffer is populated with the content of
the ".overview" file.  In this particular group, I have thousands of
messages and I think that explains the size of this file.

> The search routine should populate the buffer only with the headers of
> the articles found in the search (I am assuming that this list of
> found articles is not 50K lines long).  Maybe the search is not
> working properly?

As we are talking about 'gnus-summary-refer-thread', I guess that it is
expected that the nntp buffer is filled with this content.  A regular
query (with 'G G') is still fast, so I think my search engine is set up
properly.

> Can you step through gnus-summary-refer-thread and in the conditional
> that retrieves the new headers can you tell me which branch of the
> conditional is chosen (there are three possibilities:
> 'gnus-request-thread, 'gnus-search-thread, and the clause with the
> comment "Otherwise just retrieve some headers").

In my case, Emacs is using the 'gnus-search-thread' branch and ends up
calling 'gnus-get-newsgroup-headers-xover' which is the function that
parses all the nntp buffer content.

>     MG>BTW, I also have examples where 'gnus-summary-refer-thread' gives me
>     MG>some false positives (eg., not the same thread but part of the subject
>     MG>matching)
>
> This is probably by design: in the olden days many mailers were broken
> and didn't handle the references header properly (I don't know if this
> is still the case). So by default gnus tries to use information from the
> subject header to help gather loose threads, which can result in
> articles not actually part of the thread being included. You can check
> if this is the reason for what you are seeing by setting
>
> (setq gnus-summary-thread-gathering-function
>                       'gnus-gather-threads-by-references)
>
> and seeing if this makes a difference.

Ok, thanks for the explanation and FWIW, my
'gnus-summary-thread-gathering-function' is already set to
'gnus-gather-threads-by-references.

Best regards,
-- 
Manuel Giraud