bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers

[Eli Zaretskii <eliz@gnu.org>]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: dmitry@gutov.dev, 66117@debbugs.gnu.org
> Date: Sat, 07 Oct 2023 08:25:06 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> >> If one uses `get-file-buffer' instead of `find-buffer-visiting', the
> >> >> total runtime becomes 5.1 sec - almost 4x faster.
> >> >
> >> > This is also not very interesting, since find-file-noselect calls
> >> > get-file-buffer as well.
> >> 
> >> No. `find-file-noselect' calls `find-buffer-visiting'.
> >
> > Unless we use different Emacsen, find-file-noselect calls both
> > get-file-buffer and find-buffer-visiting:
> >
> >       (let* ((buf (get-file-buffer filename))  <<<<<<<<<<<<<<<<<<<<<<<<<<<<
> 
> We are probably mis-communicating. My point is that `get-file-buffer' is
> very fast (for my purposes). So, it does not matter as much if it is
> called somewhere else and how many times.

It matters to me.  Timing code must be very accurate and must not
modify the code in question, certainly not by invoking the same
primitives that are already called by the code being times.

I must say that this discussion is very frustrating from my POV.  Lots
of information, a large portion of it irrelevant, and very little
systematical analysis of the involved code, its actual performance,
and the conclusions with numbers to back them up.  On top of that,
gaps for a week or more between a message and a response, something
that makes it hard to follows the discussion.  We should be able to do
better.

> >> I still think that my previous conclusions are true. And I agree that
> >> rewriting these expensive loops in C makes sense. Maybe two new
> >> subroutines to find buffer by `buffer-file-truename' and by
> >> `buffer-file-number'?
> >
> > Yes, that's what I had in mind.
> 
> I looked closer, and there is already `get_truename_buffer', which can
> simply be exposed to Lisp.
> 
> `buffer-file-number' is a bit more tricky - it is not defined in C, but
> in files.el. However, I am wondering if this variable should be moved to
> C or maybe into the buffer object. `make-indirect-buffer' (defined in C)
> has
> 
>       Fset (intern ("buffer-save-without-query"), Qnil);
>       Fset (intern ("buffer-file-number"), Qnil);
> 
> WDYT?

TTTT, I don't know what to think.

From my POV, there are two alternatives here:

  . expose several new primitives to Lisp to make find-buffer-visiting
    faster without changing the way we store the file-to-buffer
    association information
  . introduce caches or change the way file-to-buffer associations are
    stored to speed up find-buffer-visiting

What I'd like to see is that someone implements the first idea, and
times find-buffer-visiting after that to see if it becomes fast
enough.  Then we can discuss whether anything else is needed.

> >> Aside: this reminds me about obsoletion of generalized buffer-local
> >> variable. AFAIU, there is currently no way to set buffer-local value in
> >> buffer without setting that buffer to current. It would be nice if such
> >> setting were possible, especially in performance-critical code.
> >
> > Maybe, but is there any performance-critical code which needs that?
> 
> For example, org-element.el needs to set buffer-local values in base
> buffer of an indirect buffer every time buffer text is being edited.

And that is performance-critical? in what way and in which situations?