all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Ergus via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
To: Eli Zaretskii <eliz@gnu.org>
Cc: 49264@debbugs.gnu.org
Subject: bug#49264: 28.0.50; project.el+tramp performance issue
Date: Wed, 30 Jun 2021 17:10:28 +0200	[thread overview]
Message-ID: <20210630151028.hi6eb4v4qvrjihh5@Ergus> (raw)
In-Reply-To: <83sg0zmsf1.fsf@gnu.org>

On Wed, Jun 30, 2021 at 03:46:26PM +0300, Eli Zaretskii wrote:
>> Date: Wed, 30 Jun 2021 00:21:42 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: 49264@debbugs.gnu.org
>>
>> >AFAICT, most of the time is taken by 'apply', but the profile doesn't
>> >show which function is called by 'apply'.  Can you tell which function
>> >is that?
>> >
>> apply is called in vc-responsible-backend and looking at the percentages
>> it is obvious that the times are going in
>>
>>    23%                      + vc-svn-responsible-p
>>    18%                      + vc-bzr-responsible-p
>>    15%                      + vc-hg-responsible-p
>>     6%                      + vc-git-responsible-p
>>     2%                      + vc-cvs-responsible-p
>>     2%                      + vc-rcs-responsible-p
>>     1%                      + vc-sccs-responsible-p
>>     1%                      + vc-src-responsible-p
>> ------
>>    68%
>>
>> These are the backends' functions that are passed to vc-call-backend in
>> the mapcar in vc-responsible-backend and the output of
>> vc-find-backend-function.
>
>But if you still need to wait for dozens of seconds with just 2
>backends, which take only 20% of the time according to the above, then
>how do you want to speed this up in your case?  How about not using
>ivy (which AFAICT is the root cause of this slowness)?
>
In the test Dimitry requested I sent the numbers of eval:

(benchmark 1 '(project-current))

Only with git and mercurial it takes:

Elapsed time: 3.018998s (0.109912s in 1 GCs)

With the entire list in vc-handled-backends

Elapsed time: 8.197923s (0.507396s in 6 GCs)

So, ivy is not the problem.

>> >... after removing the "unused" VC back-ends, you say that the code
>> >still runs very slowly.  So is the issue with VC back-ends still
>> >relevant, and if so, how?
>> >
>> Yes, it is. It is still slow, lets say around 20-40 seconds to complete
>> the command. But that's that's much faster than before (3-5 minutes);
>> but still too slow to make the command usable.
>
>It's still unacceptably slow, so that couldn't be a solution,
>certainly not a general one, IMO.
>
Agree, but it is another symptom about where the problem is.

>> As a note here, when N files are in the same directory the normal thing
>> is that all of them share the VCS. So calling a check function for all
>> of them is redundant and slow.
>
>AFAIR, that's not really true, and ISTR project.el aims to support the
>use cases with several different VC backends.
>
I can't speak for all the possible configurations; but IMO the most
common configurations around are projects with single vcs OR projects
with subprojects (git submodules) in different subdirs I am not aware of
any project where different vcs are in the same root directory... For
sure I have never seen that...

For example something like:

project0
|- file0-0
|- dir0
|   |- file0-1
|   |- file0-1
|
|- subproject1
|  |- file1-1
|  |- file1-2
|  |- dir1
|
|- subproject2
    |- file2-1
    |- file2-2
    |- dir2
    |  |- file2-3
    |
    |- subproject3
       |- file3-1
       |- dir2
          |- file3-2


Again: IMHO if we support this configuration we support 99% of the
projects around. And I already proposed an option (using the default-dir
as cache index) that already cover this with no functionality penalty.

>And again, waiting for 30 seconds when you have just 10 buffers is
>unacceptable.  E.g., in the session where I'm writing this, I have 80
>buffers that visit files in a single branch of the Emacs Git
>repository, almost an order of magnitude more than your 10 buffers.
>
With my solution we only impact performance:
1) The first time we call the command
AND
2) All the 80 files are all in different directories.

Otherwise there will be some benefit either for files located in the
same directory and/or the next time you call the command (or open a file
in one of the directories emacs already knows about.).

>So we must find some better way of getting reasonable performance in
>this use case.  If the round-trip of the VC backend to a remote
>filesystem is the bottleneck, let's try to speed that up in some way.

In case you are interested about the issue with the external package:

https://github.com/FelipeLema/emacs-counsel-gtags/issues/29





  parent reply	other threads:[~2021-06-30 15:10 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <87fsx13aiz.fsf.ref@aol.com>
2021-06-28 22:11 ` bug#49264: 28.0.50; project.el+tramp performance issue Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-06-29 12:05   ` Eli Zaretskii
2021-06-29 22:21     ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-06-30 12:46       ` Eli Zaretskii
2021-06-30 13:25         ` Phil Sainty
2021-06-30 13:35           ` Eli Zaretskii
2021-06-30 15:10         ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors [this message]
2021-06-29 13:00   ` Dmitry Gutov
2021-06-30  0:01     ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-07-26 16:56   ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-07-26 23:04     ` Dmitry Gutov
2021-08-09  0:59       ` Dmitry Gutov
2021-08-17  0:45         ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-08-19  1:19           ` Dmitry Gutov
2021-08-19  3:08             ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-08-21  2:23               ` Dmitry Gutov
2021-08-21  5:43                 ` Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors
2021-08-21 10:59                   ` Dmitry Gutov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210630151028.hi6eb4v4qvrjihh5@Ergus \
    --to=bug-gnu-emacs@gnu.org \
    --cc=49264@debbugs.gnu.org \
    --cc=eliz@gnu.org \
    --cc=spacibba@aol.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.