bug#59935: 29.0.60; project-list-buffers is slow

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

X-Debbugs-Cc: juri@linkov.net

When you invoke 'C-x p C-b', there's a noticeable pause before the
buffer list is displayed. It's ~400ms over here when there are 46
non-hidden buffers in the running session.

What I think is happening, the predicate is called 46, which in turn 
calls (project-buffers pr) 46 times, and that ends up being 46 times 
slower than one might have anticipated.

Not sure what's the best fix here (especially in time for the release),
but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
solution.

Another would be to only leave the legacy codepath: it's not affected,
project-buffers is only called once there.

Curiously, though, it shows a different list of buffers. It also
includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
into that either way.

And the patch is this:

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 016dfdd5b4..ad49df0423 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1342,16 +1342,13 @@ project-list-buffers
    (interactive "P")
    (let ((pr (project-current t)))
      (display-buffer
-     (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
-           (with-current-buffer buf
-             (setq-local revert-buffer-function
-                         (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
-                           (tabulated-list-print t))))
-           buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+     (let ((buf (list-buffers-noselect arg (project-buffers pr))))
+       (with-current-buffer buf
+         (setq-local revert-buffer-function
+                     (lambda (&rest _ignored)
+                       (list-buffers--refresh (project-buffers pr))
+                       (tabulated-list-print t))))
+       buf))))
   (defcustom project-kill-buffer-conditions
    '(buffer-file-name    ; All file-visiting buffers are included.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 03:49, Dmitry Gutov wrote:
> Curiously, though, it shows a different list of buffers. It also
> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
> into that either way.

The combined fix for both can be this:

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 016dfdd5b4..835ab07e50 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1340,18 +1340,21 @@ project-list-buffers
  start with a space (which are for internal use).  With prefix argument
  ARG, show only buffers that are visiting files."
    (interactive "P")
-  (let ((pr (project-current t)))
+  (let* ((pr (project-current t))
+         (fetcher (lambda ()
+                    (cl-delete-if-not
+                     (lambda (b)
+                       (or (buffer-file-name b)
+                           (string-match-p "\\`[^ ]" (buffer-name b))))
+                     (project-buffers pr)))))
      (display-buffer
-     (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
-           (with-current-buffer buf
-             (setq-local revert-buffer-function
-                         (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
-                           (tabulated-list-print t))))
-           buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+     (let ((buf (list-buffers-noselect arg (funcall fetcher))))
+       (with-current-buffer buf
+         (setq-local revert-buffer-function
+                     (lambda (&rest _ignored)
+                       (list-buffers--refresh (funcall fetcher))
+                       (tabulated-list-print t))))
+       buf))))

  (defcustom project-kill-buffer-conditions
    '(buffer-file-name    ; All file-visiting buffers are included.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Cc: juri@linkov.net
> Date: Sat, 10 Dec 2022 03:49:47 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> When you invoke 'C-x p C-b', there's a noticeable pause before the
> buffer list is displayed. It's ~400ms over here when there are 46
> non-hidden buffers in the running session.
> 
> What I think is happening, the predicate is called 46, which in turn 
> calls (project-buffers pr) 46 times, and that ends up being 46 times 
> slower than one might have anticipated.
> 
> Not sure what's the best fix here (especially in time for the release),
> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
> solution.
> 
> Another would be to only leave the legacy codepath: it's not affected,
> project-buffers is only called once there.
> 
> Curiously, though, it shows a different list of buffers. It also
> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
> into that either way.

I'm not following.  Are you saying that the problem is that the
FILTER-PREDICATE argument to list-buffers-noselect calls
project-buffers for each buffer?  If so, I don't understand why a
trivial change to project-list-buffers could not call project-buffers
just once, and then reuse the value in the call to
list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
in the FILTER-PREDICATE argument, using memq.

If this is not the problem, then what is?

> And the patch is this:

Consequently, I don't understand the rationale for this patch, nor for
the combined one you posted later.  Please elaborate.

P.S. Btw, if you are not sure the multiple calls to project-buffers is
the reason for the slow operation, how about profiling it to be sure?
We may be barking up the wrong tree.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 10:11, Eli Zaretskii wrote:
> I'm not following.  Are you saying that the problem is that the
> FILTER-PREDICATE argument to list-buffers-noselect calls
> project-buffers for each buffer?

Yes.

> If so, I don't understand why a
> trivial change to project-list-buffers could not call project-buffers
> just once, and then reuse the value in the call to
> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> in the FILTER-PREDICATE argument, using memq.

If we just do that, pressing 'g' will only show the buffers that 
belonged to the project when project-list-buffers was called. And not 
necessarily the *current* list of project buffers.

The previously attached patches solved that problem.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sat, 10 Dec 2022 12:47:38 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > If so, I don't understand why a
> > trivial change to project-list-buffers could not call project-buffers
> > just once, and then reuse the value in the call to
> > list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> > in the FILTER-PREDICATE argument, using memq.
> 
> If we just do that, pressing 'g' will only show the buffers that 
> belonged to the project when project-list-buffers was called. And not 
> necessarily the *current* list of project buffers.
> 
> The previously attached patches solved that problem.

I don't understand: how is 'g' relevant?  You were talking about
'C-x p C-b' in your original message.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 13:12, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 12:47:38 +0200
>> Cc:59935@debbugs.gnu.org,juri@linkov.net
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>>> If so, I don't understand why a
>>> trivial change to project-list-buffers could not call project-buffers
>>> just once, and then reuse the value in the call to
>>> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
>>> in the FILTER-PREDICATE argument, using memq.
>> If we just do that, pressing 'g' will only show the buffers that
>> belonged to the project when project-list-buffers was called. And not
>> necessarily the*current*  list of project buffers.
>>
>> The previously attached patches solved that problem.
> I don't understand: how is 'g' relevant?  You were talking about
> 'C-x p C-b' in your original message.

'C-x p C-b' creates a buffer which lists buffers. It also sets up 
revert-buffer-function in that buffer.

And pressing 'g' in that new buffer should supposedly refresh that list 
of buffers appropriately.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sat, 10 Dec 2022 13:37:37 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 10/12/2022 13:12, Eli Zaretskii wrote:
> >> Date: Sat, 10 Dec 2022 12:47:38 +0200
> >> Cc:59935@debbugs.gnu.org,juri@linkov.net
> >> From: Dmitry Gutov<dgutov@yandex.ru>
> >>
> >>> If so, I don't understand why a
> >>> trivial change to project-list-buffers could not call project-buffers
> >>> just once, and then reuse the value in the call to
> >>> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> >>> in the FILTER-PREDICATE argument, using memq.
> >> If we just do that, pressing 'g' will only show the buffers that
> >> belonged to the project when project-list-buffers was called. And not
> >> necessarily the*current*  list of project buffers.
> >>
> >> The previously attached patches solved that problem.
> > I don't understand: how is 'g' relevant?  You were talking about
> > 'C-x p C-b' in your original message.
> 
> 'C-x p C-b' creates a buffer which lists buffers. It also sets up 
> revert-buffer-function in that buffer.
> 
> And pressing 'g' in that new buffer should supposedly refresh that list 
> of buffers appropriately.

OK, but I didn't suggest making any changes to revert-buffer-function
set up by "C-x p C-b".  You can still use process-buffers there, and
it shouldn't slow down "C-x p C-b" itself, since the call to
process-buffers that is part of revert-buffer-function is not supposed
to be evaluated when you set revert-buffer-function.  Right?  Or what
else am I missing?

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 16:33, Eli Zaretskii wrote:
> OK, but I didn't suggest making any changes to revert-buffer-function
> set up by "C-x p C-b".  You can still use process-buffers there, and
> it shouldn't slow down "C-x p C-b" itself, since the call to
> process-buffers that is part of revert-buffer-function is not supposed
> to be evaluated when you set revert-buffer-function.  Right?  Or what
> else am I missing?

Then only the 'g' binding will remain slow. An improvement, of course.

But if we go this route, then we don't use the "new" 
list-buffers-noselect convention either, so we might as well do the full 
fix.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sat, 10 Dec 2022 21:19:30 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 10/12/2022 16:33, Eli Zaretskii wrote:
> > OK, but I didn't suggest making any changes to revert-buffer-function
> > set up by "C-x p C-b".  You can still use process-buffers there, and
> > it shouldn't slow down "C-x p C-b" itself, since the call to
> > process-buffers that is part of revert-buffer-function is not supposed
> > to be evaluated when you set revert-buffer-function.  Right?  Or what
> > else am I missing?
> 
> Then only the 'g' binding will remain slow. An improvement, of course.

Why slow? it calls project-buffers only once.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 21:42, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 21:19:30 +0200
>> Cc:59935@debbugs.gnu.org,juri@linkov.net
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>> On 10/12/2022 16:33, Eli Zaretskii wrote:
>>> OK, but I didn't suggest making any changes to revert-buffer-function
>>> set up by "C-x p C-b".  You can still use process-buffers there, and
>>> it shouldn't slow down "C-x p C-b" itself, since the call to
>>> process-buffers that is part of revert-buffer-function is not supposed
>>> to be evaluated when you set revert-buffer-function.  Right?  Or what
>>> else am I missing?
>> Then only the 'g' binding will remain slow. An improvement, of course.
> Why slow? it calls project-buffers only once.

You're looking at the "legacy" code path. Where emacs-version is < 29.

In the "new" one revert-buffer-function uses the filter-predicate 
argument we pass to list-buffers-noselect. Which is

   (lambda (buf) (memq buf (project-buffers pr)))

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sat, 10 Dec 2022 21:53:28 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 10/12/2022 21:42, Eli Zaretskii wrote:
> >> Date: Sat, 10 Dec 2022 21:19:30 +0200
> >> Cc:59935@debbugs.gnu.org,juri@linkov.net
> >> From: Dmitry Gutov<dgutov@yandex.ru>
> >>
> >> On 10/12/2022 16:33, Eli Zaretskii wrote:
> >>> OK, but I didn't suggest making any changes to revert-buffer-function
> >>> set up by "C-x p C-b".  You can still use process-buffers there, and
> >>> it shouldn't slow down "C-x p C-b" itself, since the call to
> >>> process-buffers that is part of revert-buffer-function is not supposed
> >>> to be evaluated when you set revert-buffer-function.  Right?  Or what
> >>> else am I missing?
> >> Then only the 'g' binding will remain slow. An improvement, of course.
> > Why slow? it calls project-buffers only once.
> 
> You're looking at the "legacy" code path. Where emacs-version is < 29.

I'm looking at the code you posted as the original one, which is
identical to what I see on the current emacs-29 branch.  That means
Emacs 29, unless I'm seriously confused.

> In the "new" one revert-buffer-function uses the filter-predicate 
> argument we pass to list-buffers-noselect. Which is
> 
>    (lambda (buf) (memq buf (project-buffers pr)))

I see this in emacs-29:

             (setq-local revert-buffer-function
                         (lambda (&rest _ignored)
                           (list-buffers--refresh (project-buffers pr))
                           (tabulated-list-print t))))

So I still don't understand your fears.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 22:11, Eli Zaretskii wrote:
> I see this in emacs-29:
> 
>               (setq-local revert-buffer-function
>                           (lambda (&rest _ignored)
>                             (list-buffers--refresh (project-buffers pr))
>                             (tabulated-list-print t))))
> 
> So I still don't understand your fears.

That's the (version< emacs-version "29.0.50") branch.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sat, 10 Dec 2022 22:23:01 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 10/12/2022 22:11, Eli Zaretskii wrote:
> > I see this in emacs-29:
> > 
> >               (setq-local revert-buffer-function
> >                           (lambda (&rest _ignored)
> >                             (list-buffers--refresh (project-buffers pr))
> >                             (tabulated-list-print t))))
> > 
> > So I still don't understand your fears.
> 
> That's the (version< emacs-version "29.0.50") branch.

Still unclear.  It almost looks like you don't _want_ me to
understand.  A detailed explanation with code fragments would be much
more effective.

Anyway, I only entered this discussion because you asked about making
some non-trivial changes on the release branch.  If that is no longer
an issue, I will gladly bow out of this.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 08:19, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 22:23:01 +0200
>> Cc: 59935@debbugs.gnu.org, juri@linkov.net
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> On 10/12/2022 22:11, Eli Zaretskii wrote:
>>> I see this in emacs-29:
>>>
>>>                (setq-local revert-buffer-function
>>>                            (lambda (&rest _ignored)
>>>                              (list-buffers--refresh (project-buffers pr))
>>>                              (tabulated-list-print t))))
>>>
>>> So I still don't understand your fears.
>>
>> That's the (version< emacs-version "29.0.50") branch.
> 
> Still unclear.  It almost looks like you don't _want_ me to
> understand.  A detailed explanation with code fragments would be much
> more effective.

I think I've explained everything, and we are now going in circles. Why 
not look at the definition yourself? It's 12 lines.

The function has two execution branches: one for the latest Emacs (at 
the moment), and one (the largest one) for older Emacsen.

The main problematic behavior (low performance) is exhibited by the 
"latest" branch, which looks like this:

   (let ((pr (project-current t)))
       (display-buffer
        (if (version< emacs-version "29.0.50")
            ;; ...
          (list-buffers-noselect
           arg nil (lambda (buf) (memq buf (project-buffers pr)))))))

> Anyway, I only entered this discussion because you asked about making
> some non-trivial changes on the release branch.  If that is no longer
> an issue, I will gladly bow out of this.

I expect Juri is going to argue for that later. For non-trivial changes 
in buff-menu.el, no less.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sun, 11 Dec 2022 12:23:13 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> The main problematic behavior (low performance) is exhibited by the 
> "latest" branch, which looks like this:
> 
>    (let ((pr (project-current t)))
>        (display-buffer
>         (if (version< emacs-version "29.0.50")
>             ;; ...
>           (list-buffers-noselect
>            arg nil (lambda (buf) (memq buf (project-buffers pr)))))))

Yes, I see that.  I still don't understand: this can easily be changed
to call project-buffers just once.

> > Anyway, I only entered this discussion because you asked about making
> > some non-trivial changes on the release branch.  If that is no longer
> > an issue, I will gladly bow out of this.
> 
> I expect Juri is going to argue for that later. For non-trivial changes 
> in buff-menu.el, no less.

I will wait for his comments, then.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 12:54, Eli Zaretskii wrote:
>> Date: Sun, 11 Dec 2022 12:23:13 +0200
>> Cc: 59935@debbugs.gnu.org, juri@linkov.net
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> The main problematic behavior (low performance) is exhibited by the
>> "latest" branch, which looks like this:
>>
>>     (let ((pr (project-current t)))
>>         (display-buffer
>>          (if (version< emacs-version "29.0.50")
>>              ;; ...
>>            (list-buffers-noselect
>>             arg nil (lambda (buf) (memq buf (project-buffers pr)))))))
> 
> Yes, I see that.  I still don't understand: this can easily be changed
> to call project-buffers just once.

Cue my very first response:

   If we just do that, pressing 'g' will only show the buffers that
   belonged to the project when project-list-buffers was called. And not
   necessarily the *current* list of project buffers.

Though I guess there might be ways to avoid this, like creating a cache 
which will be invalidated in post-command-hook, something like that.

>>> Anyway, I only entered this discussion because you asked about making
>>> some non-trivial changes on the release branch.  If that is no longer
>>> an issue, I will gladly bow out of this.
>>
>> I expect Juri is going to argue for that later. For non-trivial changes
>> in buff-menu.el, no less.
> 
> I will wait for his comments, then.

Let's.

bug#59935: 29.0.60; project-list-buffers is slow

[Jean Louis]

If I may add to this, that project-list-buffers also incidentally
invokes re-connection of any Tramp buffers.

--
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

> If I may add to this, that project-list-buffers also incidentally
> invokes re-connection of any Tramp buffers.

Does list-buffers invoke Tramp re-connection as well?

bug#59935: 29.0.60; project-list-buffers is slow

[Jean Louis]

* Juri Linkov <juri@linkov.net> [2022-12-12 20:20]:
> > If I may add to this, that project-list-buffers also incidentally
> > invokes re-connection of any Tramp buffers.
> 
> Does list-buffers invoke Tramp re-connection as well?

I have never observed it.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 12/12/2022 12:36, Jean Louis wrote:
> If I may add to this, that project-list-buffers also incidentally
> invokes re-connection of any Tramp buffers.

Which part of it does? The project-buffers call, or some other step?

bug#59935: 29.0.60; project-list-buffers is slow

[Jean Louis]

* Dmitry Gutov <dgutov@yandex.ru> [2022-12-12 22:59]:
> On 12/12/2022 12:36, Jean Louis wrote:
> > If I may add to this, that project-list-buffers also incidentally
> > invokes re-connection of any Tramp buffers.
> 
> Which part of it does? The project-buffers call, or some other step?

I don't know which part. All I can see is message below:

Tramp: Opening connection nil for root@my.localdomain.localhost using sudo...done

Because I was in sudo buffer (my.localdomain.localhost has been
changed)


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 13/12/2022 05:10, Jean Louis wrote:
> * Dmitry Gutov<dgutov@yandex.ru>  [2022-12-12 22:59]:
>> On 12/12/2022 12:36, Jean Louis wrote:
>>> If I may add to this, that project-list-buffers also incidentally
>>> invokes re-connection of any Tramp buffers.
>> Which part of it does? The project-buffers call, or some other step?
> I don't know which part. All I can see is message below:

You can find out by stepping through project-list-buffers with edebug.

bug#59935: 29.0.60; project-list-buffers is slow

[Jean Louis]

* Dmitry Gutov <dgutov@yandex.ru> [2022-12-13 18:30]:
> On 13/12/2022 05:10, Jean Louis wrote:
> > * Dmitry Gutov<dgutov@yandex.ru>  [2022-12-12 22:59]:
> > > On 12/12/2022 12:36, Jean Louis wrote:
> > > > If I may add to this, that project-list-buffers also incidentally
> > > > invokes re-connection of any Tramp buffers.
> > > Which part of it does? The project-buffers call, or some other step?
> > I don't know which part. All I can see is message below:
> 
> You can find out by stepping through project-list-buffers with edebug.

It iterates over results of list-buffers-noselect and among many
buffers finds those Tramp buffers. But why those Tramp buffers start
re-connecting I do not know.

For me this description below is not what that function does, as that
function seem not to select properly. What are project buffers? Are
they not files which are in directories specified as project? It seems
that function is iterating over buffers not necessary to iterate.

project-list-buffers is an autoloaded interactive Lisp closure in
‘project.el’.

It is bound to C-x p C-b.

(project-list-buffers &optional ARG)

Display a list of project buffers.
The list is displayed in a buffer named "*Buffer List*".

By default, all project buffers are listed except those whose names
start with a space (which are for internal use).  With prefix argument
ARG, show only buffers that are visiting files.



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 13/12/2022 21:30, Jean Louis wrote:
> * Dmitry Gutov <dgutov@yandex.ru> [2022-12-13 18:30]:
>> On 13/12/2022 05:10, Jean Louis wrote:
>>> * Dmitry Gutov<dgutov@yandex.ru>  [2022-12-12 22:59]:
>>>> On 12/12/2022 12:36, Jean Louis wrote:
>>>>> If I may add to this, that project-list-buffers also incidentally
>>>>> invokes re-connection of any Tramp buffers.
>>>> Which part of it does? The project-buffers call, or some other step?
>>> I don't know which part. All I can see is message below:
>>
>> You can find out by stepping through project-list-buffers with edebug.
> 
> It iterates over results of list-buffers-noselect and among many
> buffers finds those Tramp buffers. But why those Tramp buffers start
> re-connecting I do not know.

So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not 
specific to project-list-buffers?

> For me this description below is not what that function does, as that
> function seem not to select properly. What are project buffers? Are
> they not files which are in directories specified as project? It seems
> that function is iterating over buffers not necessary to iterate.

The answer to what buffers are project buffers is encoded in each 
project backend's implementation of 'project-buffers'.

The default considers non-file buffers as well.

bug#59935: 29.0.60; project-list-buffers is slow

[Jean Louis]

* Dmitry Gutov <dgutov@yandex.ru> [2022-12-13 23:32]:
> > It iterates over results of list-buffers-noselect and among many
> > buffers finds those Tramp buffers. But why those Tramp buffers start
> > re-connecting I do not know.
> 
> So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not
> specific to project-list-buffers?

Not, it is specific to project-list-buffers and not to list-buffers.

> > For me this description below is not what that function does, as that
> > function seem not to select properly. What are project buffers? Are
> > they not files which are in directories specified as project? It seems
> > that function is iterating over buffers not necessary to iterate.
> 
> The answer to what buffers are project buffers is encoded in each project
> backend's implementation of 'project-buffers'.
> 
> The default considers non-file buffers as well.

How do I define non-default?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 15/12/2022 16:58, Jean Louis wrote:
> * Dmitry Gutov <dgutov@yandex.ru> [2022-12-13 23:32]:
>>> It iterates over results of list-buffers-noselect and among many
>>> buffers finds those Tramp buffers. But why those Tramp buffers start
>>> re-connecting I do not know.
>>
>> So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not
>> specific to project-list-buffers?
> 
> Not, it is specific to project-list-buffers and not to list-buffers.

Then could you explain it in more detail? When you say "It iterates over 
results of list-buffers-noselect", what is "it"? Surely not 
project-list-buffers itself: list-buffers-noselect returns a buffer, not 
a list anyway.

>>> For me this description below is not what that function does, as that
>>> function seem not to select properly. What are project buffers? Are
>>> they not files which are in directories specified as project? It seems
>>> that function is iterating over buffers not necessary to iterate.
>>
>> The answer to what buffers are project buffers is encoded in each project
>> backend's implementation of 'project-buffers'.
>>
>> The default considers non-file buffers as well.
> 
> How do I define non-default?

By either redefinining project-vc's project-buffers method, or writing 
your own entire backend.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

> Not sure what's the best fix here (especially in time for the release),
> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
> solution.

I see only one solution: to replace the argument FILTER-PREDICATE
with BUFFER-LIST-FUNCTION.

>> Curiously, though, it shows a different list of buffers. It also
>> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
>> into that either way.
> +                    (cl-delete-if-not
> +                     (lambda (b)
> +                       (or (buffer-file-name b)
> +                           (string-match-p "\\`[^ ]" (buffer-name b))))
> +                     (project-buffers pr)))))

Please try to copy the exact logic from list-buffers--refresh:

			 (and (or (not (string= (substring name 0 1) " "))
                                  buffer-file-name)
			      (not (eq buffer (current-buffer)))
			      (or file (not Buffer-menu-files-only))

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 10/12/2022 19:45, Juri Linkov wrote:
>> Not sure what's the best fix here (especially in time for the release),
>> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
>> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
>> solution.
> 
> I see only one solution: to replace the argument FILTER-PREDICATE
> with BUFFER-LIST-FUNCTION.

Ah yeah, that should also work.

>>> Curiously, though, it shows a different list of buffers. It also
>>> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
>>> into that either way.
>> +                    (cl-delete-if-not
>> +                     (lambda (b)
>> +                       (or (buffer-file-name b)
>> +                           (string-match-p "\\`[^ ]" (buffer-name b))))
>> +                     (project-buffers pr)))))
> 
> Please try to copy the exact logic from list-buffers--refresh:
> 
> 			 (and (or (not (string= (substring name 0 1) " "))
>                                    buffer-file-name)
> 			      (not (eq buffer (current-buffer)))
> 			      (or file (not Buffer-menu-files-only))

Do you have the time to finish the fix yourself? It was your feature, 
after all.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

> Do you have the time to finish the fix yourself? It was your feature, after
> all.

Here is a complete patch tested for all possible cases:
- emacs 28/29
- with/without file arg
- with/without reverting


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Buffer-menu-buffer-list-function.patch --]
[-- Type: text/x-diff, Size: 5560 bytes --]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 38d4fdad5fc..64af86558e4 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1321,18 +1321,31 @@ project-list-buffers
 start with a space (which are for internal use).  With prefix argument
 ARG, show only buffers that are visiting files."
   (interactive "P")
-  (let ((pr (project-current t)))
+  (let* ((pr (project-current t))
+         (buffer-list-function
+          (lambda ()
+            (seq-filter
+             (lambda (buffer)
+               (let ((name (buffer-name buffer))
+                     (file (buffer-file-name buffer)))
+                 (and (or (not (string= (substring name 0 1) " "))
+                          file)
+                      (not (eq buffer (current-buffer)))
+                      (or file (not Buffer-menu-files-only)))))
+             (project-buffers pr)))))
     (display-buffer
      (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
+         (let ((buf (list-buffers-noselect
+                     arg (let ((Buffer-menu-files-only arg))
+                           (funcall buffer-list-function)))))
            (with-current-buffer buf
              (setq-local revert-buffer-function
                          (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
+                           (list-buffers--refresh
+                            (funcall buffer-list-function))
                            (tabulated-list-print t))))
            buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+       (list-buffers-noselect arg buffer-list-function)))))
 
 (defcustom project-kill-buffer-conditions
   '(buffer-file-name    ; All file-visiting buffers are included.
diff --git a/lisp/buff-menu.el b/lisp/buff-menu.el
index 588fe599a46..cabf7cb8fc4 100644
--- a/lisp/buff-menu.el
+++ b/lisp/buff-menu.el
@@ -100,12 +100,8 @@ Buffer-menu-files-only
 This is set by the prefix argument to `buffer-menu' and related
 commands.")
 
-(defvar-local Buffer-menu-filter-predicate nil
-  "Function to filter out buffers in the buffer list.
-Buffers that don't satisfy the predicate will be skipped.
-The value should be a function of one argument; it will be
-called with the buffer.  If this function returns non-nil,
-then the buffer will be displayed in the buffer list.")
+(defvar-local Buffer-menu-buffer-list-function nil
+  "Function to return buffers for the buffer list.")
 
 (defvar-keymap Buffer-menu-mode-map
   :doc "Local keymap for `Buffer-menu-mode' buffers."
@@ -623,23 +619,23 @@ Buffer-menu-view-other-window
 ;;; Functions for populating the Buffer Menu.
 
 ;;;###autoload
-(defun list-buffers-noselect (&optional files-only buffer-list filter-predicate)
+(defun list-buffers-noselect (&optional files-only buffer-list)
   "Create and return a Buffer Menu buffer.
 This is called by `buffer-menu' and others as a subroutine.
 
 If FILES-ONLY is non-nil, show only file-visiting buffers.
-If BUFFER-LIST is non-nil, it should be a list of buffers; it
-means list those buffers and no others.
-If FILTER-PREDICATE is non-nil, it should be a function
-that filters out buffers from the list of buffers.
-See more at `Buffer-menu-filter-predicate'."
+If BUFFER-LIST is non-nil, it should be either a list of buffers
+or a function that returns a list of buffers; it means
+list those buffers and no others.
+See more at `Buffer-menu-buffer-list-function'."
   (let ((old-buffer (current-buffer))
 	(buffer (get-buffer-create "*Buffer List*")))
     (with-current-buffer buffer
       (Buffer-menu-mode)
       (setq Buffer-menu-files-only
 	    (and files-only (>= (prefix-numeric-value files-only) 0)))
-      (setq Buffer-menu-filter-predicate filter-predicate)
+      (when (functionp buffer-list)
+        (setq Buffer-menu-buffer-list-function buffer-list))
       (list-buffers--refresh buffer-list old-buffer)
       (tabulated-list-print))
     buffer))
@@ -661,13 +657,17 @@ list-buffers--refresh
         (marked-buffers (Buffer-menu-marked-buffers))
         (buffer-menu-buffer (current-buffer))
 	(show-non-file (not Buffer-menu-files-only))
-	(filter-predicate (and (functionp Buffer-menu-filter-predicate)
-			       Buffer-menu-filter-predicate))
 	entries name-width)
     ;; Collect info for each buffer we're interested in.
-    (dolist (buffer (or buffer-list
-			(buffer-list (if Buffer-menu-use-frame-buffer-list
-					 (selected-frame)))))
+    (dolist (buffer (cond
+                     ((functionp buffer-list)
+                      (funcall buffer-list))
+                     (buffer-list)
+                     (Buffer-menu-buffer-list-function
+                      (funcall Buffer-menu-buffer-list-function))
+                     (t (buffer-list
+                         (if Buffer-menu-use-frame-buffer-list
+                             (selected-frame))))))
       (with-current-buffer buffer
 	(let* ((name (buffer-name))
 	       (file buffer-file-name))
@@ -676,9 +676,7 @@ list-buffers--refresh
 			 (and (or (not (string= (substring name 0 1) " "))
                                   file)
 			      (not (eq buffer buffer-menu-buffer))
-			      (or file show-non-file)
-			      (or (not filter-predicate)
-				  (funcall filter-predicate buffer)))))
+			      (or file show-non-file))))
 	    (push (list buffer
 			(vector (cond
                                  ((eq buffer old-buffer) ".")

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Cc: 59935@debbugs.gnu.org
> From: Juri Linkov <juri@linkov.net>
> Date: Sun, 11 Dec 2022 19:07:27 +0200
> 
> > Do you have the time to finish the fix yourself? It was your feature, after
> > all.
> 
> Here is a complete patch tested for all possible cases:
> - emacs 28/29
> - with/without file arg
> - with/without reverting

Thanks, but I hope you don't intend to ask to install this on the
release branch.  And changing back the signature of
list-buffers-noselect is extremely problematic, even though we changed
it only for Emacs 29.

Please try to find a safer, more compatible solution.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

>> > Do you have the time to finish the fix yourself? It was your feature, after
>> > all.
>> 
>> Here is a complete patch tested for all possible cases:
>> - emacs 28/29
>> - with/without file arg
>> - with/without reverting
>
> Thanks, but I hope you don't intend to ask to install this on the
> release branch.  And changing back the signature of
> list-buffers-noselect is extremely problematic, even though we changed
> it only for Emacs 29.
>
> Please try to find a safer, more compatible solution.

This is the safest solution for the emacs-29 branch.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: dgutov@yandex.ru,  59935@debbugs.gnu.org
> Date: Sun, 11 Dec 2022 19:56:24 +0200
> 
> >> > Do you have the time to finish the fix yourself? It was your feature, after
> >> > all.
> >> 
> >> Here is a complete patch tested for all possible cases:
> >> - emacs 28/29
> >> - with/without file arg
> >> - with/without reverting
> >
> > Thanks, but I hope you don't intend to ask to install this on the
> > release branch.  And changing back the signature of
> > list-buffers-noselect is extremely problematic, even though we changed
> > it only for Emacs 29.
> >
> > Please try to find a safer, more compatible solution.
> 
> This is the safest solution for the emacs-29 branch.

I don't see how it could be the safest.  For example, an almost
identical changeset that doesn't change the signature of
list-buffers-noselect is definitely safer.  And likewise with the
change of list-buffers--refresh -- why do we have to do that?

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 20:08, Eli Zaretskii wrote:
> I don't see how it could be the safest.  For example, an almost
> identical changeset that doesn't change the signature of
> list-buffers-noselect is definitely safer.  And likewise with the
> change of list-buffers--refresh -- why do we have to do that?

How do we avoid changing the signature?

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 19:49, Eli Zaretskii wrote:
> Thanks, but I hope you don't intend to ask to install this on the
> release branch.  And changing back the signature of
> list-buffers-noselect is extremely problematic, even though we changed
> it only for Emacs 29.

The problem with *not* changing it, is we don't seem to be able to use 
the new calling convention to implement project-list-buffers working 
without this performance problem.

We could just use the old (private) interface, but then the new calling 
convention will be left without any users at all. We extended it in 
order to implement this feature anyway.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sun, 11 Dec 2022 20:12:25 +0200
> Cc: 59935@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 11/12/2022 19:49, Eli Zaretskii wrote:
> > Thanks, but I hope you don't intend to ask to install this on the
> > release branch.  And changing back the signature of
> > list-buffers-noselect is extremely problematic, even though we changed
> > it only for Emacs 29.
> 
> The problem with *not* changing it, is we don't seem to be able to use 
> the new calling convention to implement project-list-buffers working 
> without this performance problem.
> 
> We could just use the old (private) interface, but then the new calling 
> convention will be left without any users at all. We extended it in 
> order to implement this feature anyway.

I don't think I follow.  Why not leave alone that additional PREDICATE
argument of list-buffers-noselect, and add a feature where the
BUFFER-LIST argument could be a function?  That's be a compatible
change.  Just keep the PREDICATE argument and the code which supports
it, and document that if PREDICATE is a function, BUFFER-LIST cannot
be a function.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 20:17, Eli Zaretskii wrote:
> Why not leave alone that additional PREDICATE
> argument of list-buffers-noselect, and add a feature where the
> BUFFER-LIST argument could be a function?  That's be a compatible
> change.  Just keep the PREDICATE argument and the code which supports
> it, and document that if PREDICATE is a function, BUFFER-LIST cannot
> be a function.

While that should work, that change would *also* change the signature. 
Though in a safer way, yes.

Are you worried about some out-of-tree clients that started using the 
new feature in the meantime?

Otherwise, this change could be split into two steps:

- Rollback that signature to pre-commit 125b5684c (i.e. revert its part 
in buff-menu.el). This is something we usually considered to be safe.

- Add new change that allows BUFFER-LIST to be a function.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sun, 11 Dec 2022 20:35:24 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 11/12/2022 20:17, Eli Zaretskii wrote:
> > Why not leave alone that additional PREDICATE
> > argument of list-buffers-noselect, and add a feature where the
> > BUFFER-LIST argument could be a function?  That's be a compatible
> > change.  Just keep the PREDICATE argument and the code which supports
> > it, and document that if PREDICATE is a function, BUFFER-LIST cannot
> > be a function.
> 
> While that should work, that change would *also* change the signature. 
> Though in a safer way, yes.

Indeed.

> Are you worried about some out-of-tree clients that started using the 
> new feature in the meantime?

Yes.  In particular, we never change signatures after the branch point
of a release branch -- the branch point is a de-facto code-freeze
point for the release branch.

> Otherwise, this change could be split into two steps:
> 
> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part 
> in buff-menu.el). This is something we usually considered to be safe.
> 
> - Add new change that allows BUFFER-LIST to be a function.

We can do it safer, so I'd prefer that.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 21:00, Eli Zaretskii wrote:
>> Otherwise, this change could be split into two steps:
>>
>> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part
>> in buff-menu.el). This is something we usually considered to be safe.
>>
>> - Add new change that allows BUFFER-LIST to be a function.
> We can do it safer, so I'd prefer that.

I don't like that idea because we'll leave an unproven new feature in 
the codebase, without any known callers or requestors. That's just extra 
complexity, with all associated long-term costs.

But you're the boss. It's better to fix this than not, obviously.

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> Date: Sun, 11 Dec 2022 21:41:40 +0200
> Cc: 59935@debbugs.gnu.org, juri@linkov.net
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 11/12/2022 21:00, Eli Zaretskii wrote:
> >> Otherwise, this change could be split into two steps:
> >>
> >> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part
> >> in buff-menu.el). This is something we usually considered to be safe.
> >>
> >> - Add new change that allows BUFFER-LIST to be a function.
> > We can do it safer, so I'd prefer that.
> 
> I don't like that idea because we'll leave an unproven new feature in 
> the codebase, without any known callers or requestors.

Neither do I.  But this is a price to pay for coming this late with
changes in interfaces that are needed to make commands faster, and we
cannot wait.  If this came up half a year ago, we wouldn't be having
this conversation.

The schedule for releasing Emacs 29.1 is very ambitious, from where I
stand, and I don't want any risks we can avoid.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

>> I don't like that idea because we'll leave an unproven new feature in
>> the codebase, without any known callers or requestors.
>
> Neither do I.  But this is a price to pay for coming this late with
> changes in interfaces that are needed to make commands faster, and we
> cannot wait.  If this came up half a year ago, we wouldn't be having
> this conversation.

Ok, then here is the patch for emacs-29:

diff --git a/etc/NEWS b/etc/NEWS
index 233ef3f5729..2e5bb40c972 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -3437,6 +3437,10 @@ The following generalized variables have been made obsolete:
 \f
 * Lisp Changes in Emacs 29.1
 
+---
+** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
+It will be removed in next versions.
+
 +++
 ** Interpreted closures are "safe for space".
 As was already the case for byte-compiled closures, instead of capturing

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  59935@debbugs.gnu.org
> Date: Mon, 12 Dec 2022 19:16:22 +0200
> 
> >> I don't like that idea because we'll leave an unproven new feature in
> >> the codebase, without any known callers or requestors.
> >
> > Neither do I.  But this is a price to pay for coming this late with
> > changes in interfaces that are needed to make commands faster, and we
> > cannot wait.  If this came up half a year ago, we wouldn't be having
> > this conversation.
> 
> Ok, then here is the patch for emacs-29:
> 
> diff --git a/etc/NEWS b/etc/NEWS
> index 233ef3f5729..2e5bb40c972 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -3437,6 +3437,10 @@ The following generalized variables have been made obsolete:
>  \f
>  * Lisp Changes in Emacs 29.1
>  
> +---
> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
> +It will be removed in next versions.
> +
>  +++
>  ** Interpreted closures are "safe for space".
>  As was already the case for byte-compiled closures, instead of capturing

I thought we wanted to make the command faster in Emacs 29.  I very
much doubt that deprecation of an argument will have that effect.

I'm also not sure I agree with obsoleting and removing that argument.
We don't have enough justification for that, not yet.  That it was
introduced for a particular use case, and that use case no longer uses
it, doesn't mean there won't be others.  After all, we don't introduce
additional arguments unless we think it will be useful in more than
one case.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

>> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
>> +It will be removed in next versions.
>
> I thought we wanted to make the command faster in Emacs 29.  I very
> much doubt that deprecation of an argument will have that effect.
>
> I'm also not sure I agree with obsoleting and removing that argument.
> We don't have enough justification for that, not yet.  That it was
> introduced for a particular use case, and that use case no longer uses
> it, doesn't mean there won't be others.  After all, we don't introduce
> additional arguments unless we think it will be useful in more than
> one case.

Having both FILTER-PREDICATE and BUFFER-LIST-FUNCTION is too problematic.
Which one should take priority over another when both specified?

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: dgutov@yandex.ru,  59935@debbugs.gnu.org
> Date: Mon, 12 Dec 2022 19:51:48 +0200
> 
> >> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
> >> +It will be removed in next versions.
> >
> > I thought we wanted to make the command faster in Emacs 29.  I very
> > much doubt that deprecation of an argument will have that effect.
> >
> > I'm also not sure I agree with obsoleting and removing that argument.
> > We don't have enough justification for that, not yet.  That it was
> > introduced for a particular use case, and that use case no longer uses
> > it, doesn't mean there won't be others.  After all, we don't introduce
> > additional arguments unless we think it will be useful in more than
> > one case.
> 
> Having both FILTER-PREDICATE and BUFFER-LIST-FUNCTION is too problematic.
> Which one should take priority over another when both specified?

I thought I answered that in my original proposal:

> Why not leave alone that additional PREDICATE argument of
> list-buffers-noselect, and add a feature where the BUFFER-LIST
> argument could be a function?  That's be a compatible change.  Just
> keep the PREDICATE argument and the code which supports it, and
> document that if PREDICATE is a function, BUFFER-LIST cannot be a
> function.

Note the last sentence.  Doesn't it resolve the difficulty in a
reasonable way?

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

>> Why not leave alone that additional PREDICATE argument of
>> list-buffers-noselect, and add a feature where the BUFFER-LIST
>> argument could be a function?  That's be a compatible change.  Just
>> keep the PREDICATE argument and the code which supports it, and
>> document that if PREDICATE is a function, BUFFER-LIST cannot be a
>> function.
>
> Note the last sentence.  Doesn't it resolve the difficulty in a
> reasonable way?

Then support for a function in BUFFER-LIST should be implemented
in emacs-29?  (while keeping the recently added PREDICATE argument)

bug#59935: 29.0.60; project-list-buffers is slow

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: dgutov@yandex.ru,  59935@debbugs.gnu.org
> Date: Mon, 12 Dec 2022 20:14:46 +0200
> 
> >> Why not leave alone that additional PREDICATE argument of
> >> list-buffers-noselect, and add a feature where the BUFFER-LIST
> >> argument could be a function?  That's be a compatible change.  Just
> >> keep the PREDICATE argument and the code which supports it, and
> >> document that if PREDICATE is a function, BUFFER-LIST cannot be a
> >> function.
> >
> > Note the last sentence.  Doesn't it resolve the difficulty in a
> > reasonable way?
> 
> Then support for a function in BUFFER-LIST should be implemented
> in emacs-29?  (while keeping the recently added PREDICATE argument)

Yes.  Why not?  It cannot possibly hurt any existing callers, because
they won't use a function.  Right?

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

close 59935 29.0.60
thanks

>> Then support for a function in BUFFER-LIST should be implemented
>> in emacs-29?  (while keeping the recently added PREDICATE argument)
>
> Yes.  Why not?  It cannot possibly hurt any existing callers, because
> they won't use a function.  Right?

So now pushed to emacs-29 and closed.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 11/12/2022 19:07, Juri Linkov wrote:
> +(defvar-local Buffer-menu-buffer-list-function nil
> +  "Function to return buffers for the buffer list.")
> 
>         (setq Buffer-menu-files-only
>   	    (and files-only (>= (prefix-numeric-value files-only) 0)))
> -      (setq Buffer-menu-filter-predicate filter-predicate)
> +      (when (functionp buffer-list)
> +        (setq Buffer-menu-buffer-list-function buffer-list))

Here's an idea: when 'list-buffers-noselect' received a plain list of 
buffers in its BUFFER-LIST argument, it doesn't save that anywhere.

That seems like a bug, doesn't it? That hitting 'g' in such a 
buffer-list buffer resets its contents to all buffer (except hidden, etc).

So it probably makes sense to save it as well.

Long story short, I suggest to name the new variable 
Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument 
to it no matter what. And, likewise, use it. But when the value is a 
function, call it to obtain the actual list.

One side-effect of this, though, is that the BUFFER-LIST argument to 
list-buffers--refresh will have no purpose anymore.

bug#59935: 29.0.60; project-list-buffers is slow

[Juri Linkov]

> Here's an idea: when 'list-buffers-noselect' received a plain list of
> buffers in its BUFFER-LIST argument, it doesn't save that anywhere.
>
> That seems like a bug, doesn't it? That hitting 'g' in such a buffer-list
> buffer resets its contents to all buffer (except hidden, etc).
>
> So it probably makes sense to save it as well.
>
> Long story short, I suggest to name the new variable
> Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument to
> it no matter what. And, likewise, use it. But when the value is a function,
> call it to obtain the actual list.

Thanks for the idea.  Implemented in the commit a99d0e7e6c9.

> One side-effect of this, though, is that the BUFFER-LIST argument to
> list-buffers--refresh will have no purpose anymore.

I'm not sure about removing the old argument BUFFER-LIST from
list-buffers--refresh, even though it's an internal function.

It also has another argument OLD-BUFFER that could be turned later
into buffer-local, to keep the current buffer marker after revert.

bug#59935: 29.0.60; project-list-buffers is slow

[Dmitry Gutov]

On 13/12/2022 19:49, Juri Linkov wrote:
>> Here's an idea: when 'list-buffers-noselect' received a plain list of
>> buffers in its BUFFER-LIST argument, it doesn't save that anywhere.
>>
>> That seems like a bug, doesn't it? That hitting 'g' in such a buffer-list
>> buffer resets its contents to all buffer (except hidden, etc).
>>
>> So it probably makes sense to save it as well.
>>
>> Long story short, I suggest to name the new variable
>> Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument to
>> it no matter what. And, likewise, use it. But when the value is a function,
>> call it to obtain the actual list.
> 
> Thanks for the idea.  Implemented in the commit a99d0e7e6c9.

Thank you.

>> One side-effect of this, though, is that the BUFFER-LIST argument to
>> list-buffers--refresh will have no purpose anymore.
> 
> I'm not sure about removing the old argument BUFFER-LIST from
> list-buffers--refresh, even though it's an internal function.

Maybe sometime later on master.

> It also has another argument OLD-BUFFER that could be turned later
> into buffer-local, to keep the current buffer marker after revert.

Why not. Although at that point which of the buffers was current 
previously doesn't seem too important.