bug#49264: 28.0.50; project.el+tramp performance issue

[Eli Zaretskii <eliz@gnu.org>]

> Date: Tue, 29 Jun 2021 00:11:00 +0200
> From:  Ergus via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Using tramp I tried to use project.el with a command like
> project-switch-to-buffer and it took like 10 minutes to complete.
> 
> I ran a profiler and I found that most of the time was taken by an
> external function: global-tags-try-project-root

That doesn't follow from the profile you show.  According to the
profile, global-tags-try-project-root takes just 6% of the CPU time.

> project-current is called in a loop for all the opened buffers it calls
> project--find-in-directory that calls project-find-functions and there
> is going all the time.

I don't see project-find-functions in the profile.  Where is it and
how does it come into this picture?

> After some optimization in an external package; now the time is half
> than before but still very slow to use the command (around 3-5 minutes
> to complete) and running again the profiler I get this:
> 
>          5637  89% - command-execute
>          5549  88%  - byte-code
>          5549  88%   - project--read-project-buffer
>          5549  88%    - let*
>          5336  85%     - read-buffer
>          5323  84%      - ivy-completing-read
>          5323  84%       - ivy-read
>          4941  78%        - ivy--reset-state
>          4941  78%         - ivy--buffer-list
>          4941  78%          - internal-complete-buffer
>          4941  78%           - #<lambda -0x1a357caf01243d61>
>          4941  78%            - and
>          4941  78%             - equal
>          4941  78%              - save-current-buffer
>          4941  78%               - project-current
>          4941  78%                - project--find-in-directory
>          4548  72%                 - project-try-vc
>          4537  72%                  - vc-responsible-backend
>          4478  71%                   - #<compiled 0xd3f2e32af0966f7>
>          4478  71%                    - vc-call-backend
>          4478  71%                     - apply
>          1470  23%                      + vc-svn-responsible-p
>          1142  18%                      + vc-bzr-responsible-p
>           970  15%                      + vc-hg-responsible-p
>           390   6%                      + vc-git-responsible-p
>           156   2%                      + vc-cvs-responsible-p
>           126   2%                      + vc-rcs-responsible-p
>           108   1%                      + vc-sccs-responsible-p
>            98   1%                      + vc-src-responsible-p
>            57   0%                   + tramp-file-name-handler
>            11   0%                  + vc-file-getprop
>           393   6%                 + global-tags-try-project-root
>           375   5%        + read-from-minibuffer
>            13   0%      + if
>           213   3%     + project-current
>            88   1%  + funcall-interactively
>           572   9% + ...
>            51   0% + timer-event-handler
>             8   0% + redisplay_internal (C function)
> 
> 
> As you can see most of the time is still taken by project-current and I
> can't really understand why:

AFAICT, most of the time is taken by 'apply', but the profile doesn't
show which function is called by 'apply'.  Can you tell which function
is that?

> 1) Are so many samples 4548 seems a very high number for only 25 opened
> buffers.

These two numbers are unrelated.  4548 is the number of time the
profiler found the program counter inside project-try-vc and the
functions it calls.  This number has no relation to the number of
buffers you have, it just means that code runs slowly.

> 2) why project-try-vc still takes so much...? Specially for unfrequent
> vc systems in our days like svn or bzr that I am not using.

That was explained on emacs-devel.  However, ...

> As a workaround I removed all the uninteresting handlers from
> vc-handled-backends and I get better times now, but IMHO it is still
> very inefficient (almost a minute for project-switch-to-buffer is
> excessive). And make it practically unusable.

... after removing the "unused" VC back-ends, you say that the code
still runs very slowly.  So is the issue with VC back-ends still
relevant, and if so, how?

More importantly, what is the profile after you remove the extra VC
calls?

> VCS changing is not something that happens very often to require a check
> of all the backends everytime, several times for every buffer in many
> project.el functions right? Specially when using tramp.

Once again, given what you say above, this doesn't sound important,
does it?  The slow processing is elsewhere, and without seeing a
profile with VC calls removed, it's hard to make progress in this
matter, or give you some advice regarding potential reason(s).

> vc has vc-file-prop-obarray; maybe vc-responsible-backend should cache
> it's result there to avoid repeating time consuming computations?

Again: is this issue relevant, given that without the VC calls the
code is still very slow?