bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening large number of buffers

[Dmitry Gutov <dmitry@gutov.dev>]

On 22/09/2023 15:41, Ihor Radchenko wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>>> Moreover, `find-buffer-visiting' is called by `find-file-noselect', and
>>> I simply cannot make use of cache there without modifying the code
>>> upstream; or writing my own version of `find-file-noselect' - bad idea
>>> maintenance-wise.
>>
>> ...if most of said calls are done through find-file-noselect, I suppose
>> that solution is a no-go.
> 
> Not a no-go, but not a complete solution either.
> I asked the user who provided the profiler report I have attached to
> replace `find-buffer-visiting' with `get-file-buffer' in the relevant Org
> sources and it did lead to ~30% runtime reduction.
> 
> However, upon recording the profiler report with that change,
> `find-buffer-visiting' still took a significant fraction of CPU time.
> So, I decided to reach out upstream.

Because the rest of the calls are made from Org?

I see this in the profile, though:

         6247  20%        - editorconfig--advice-find-file-noselect

There seems to be a lot of timers in that profile, and this is in one of 
them (under "Automatic GC" which takes up 53% of the time).

So one was to improve this in the meantime would be to bring it up with 
"editorconfig" authors. I would even question the wisdom of advising 
find-file-noselect.

> Will it be acceptable to implement the cache using variable watchers?

I think variable watchers are mostly for debugging? I was thinking of a 
straight approach where the base primitives that visit a file or kill a 
buffer update said cache.

>>> I think that the best way that will benefit more than Org mode is
>>> arranging internal cache that will link buffer-file-name,
>>> buffer-file-truename, and buffer-file-number with buffers; and maintain
>>> the correctness of the cache if buffer-file-name changes for any reason.
>>
>> I think that is doable.
>>
>> It probably won't regress the performance of any particular scenario
>> either, but we should benchmark opening a bunch of files this way anyway
>> (might actually get faster, due to find-buffer-visiting calls).
> 
> The regression might happen when the number of buffers is small -
> when hash tables become slower compared to simple list lookup.
> But in such scenario, we will be talking about very small absolute
> runtimes anyway, so it should probably not matter in practice.

I suppose.