From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ergus via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#49264: 28.0.50; project.el+tramp performance issue Date: Wed, 30 Jun 2021 17:10:28 +0200 Message-ID: <20210630151028.hi6eb4v4qvrjihh5@Ergus> References: <87fsx13aiz.fsf.ref@aol.com> <87fsx13aiz.fsf@aol.com> <83mtr8ooz4.fsf@gnu.org> <20210629222142.jbyegdo72wkp6vv2@Ergus> <83sg0zmsf1.fsf@gnu.org> Reply-To: Ergus Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="339"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 49264@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Jun 30 17:11:18 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lybry-000ANX-Ae for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 30 Jun 2021 17:11:18 +0200 Original-Received: from localhost ([::1]:57522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lybrx-00050J-Ay for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 30 Jun 2021 11:11:17 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60234) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lybrj-0004qm-Cb for bug-gnu-emacs@gnu.org; Wed, 30 Jun 2021 11:11:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:46923) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lybrj-0007Mb-29 for bug-gnu-emacs@gnu.org; Wed, 30 Jun 2021 11:11:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lybri-00052R-Tw for bug-gnu-emacs@gnu.org; Wed, 30 Jun 2021 11:11:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ergus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 30 Jun 2021 15:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49264 X-GNU-PR-Package: emacs Original-Received: via spool by 49264-submit@debbugs.gnu.org id=B49264.162506584719330 (code B ref 49264); Wed, 30 Jun 2021 15:11:02 +0000 Original-Received: (at 49264) by debbugs.gnu.org; 30 Jun 2021 15:10:47 +0000 Original-Received: from localhost ([127.0.0.1]:58467 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lybrT-00051h-1y for submit@debbugs.gnu.org; Wed, 30 Jun 2021 11:10:47 -0400 Original-Received: from sonic313-13.consmr.mail.bf2.yahoo.com ([74.6.133.123]:41455) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lybrR-00051V-PZ for 49264@debbugs.gnu.org; Wed, 30 Jun 2021 11:10:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aol.com; s=a2048; t=1625065839; bh=NPSobimR8mROdz8XoORH96x/vSWfecJyZjOdZTJmSGI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject:Reply-To; b=LdfhvTahGoPeassEMdlUCStS4m0TSx44gFfaQaq4s7pSbxMpSNLXJ/UQFwonvXYrVPmsd+7Y6h1JY/SM3bPgSVZMtqWQAgVqg4jQipZtbD/e70t4DKf5O0+R8H979QYpoogFMbAChF/1jikqP5imtxXW2Vcff7hDtbctouIsKszfKT0KcCRy4qzQjy47TcnUiTEgtDWUxpgz47h+Qeji5Hdn2TE4rQDx8GzjB9IyaYgmus8rYazYigJDZMwWP919Nl4oFPsrH3Pc12uaBJcbnySHcImVDWxdBQi/6YPf9PXISUZHEEud5x22OgXZx0puB3WV2WAnZCLsG8CH6nFyOg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1625065839; bh=WQ9OMYlpb2MFEKfttefuFe9pXnXq9TUprnlOpPfvv8l=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=K7pTvMbG3SDYymwbLl8i5jRDzaDGLLW3vSy2L2ASEcbsljA+Mrj3Wj3iKLcN+hGnVLsPZSsL/uE0h+8EvnpVtO3/gnmK7EAY5Er6mDEe5m8ogvixQvdL3mcBXDNIiRssI343KSkRKlavzIhx9DcTKjZcsCvoVBzMogktEwnZ4EKCBQoG/K4ToKZYsz9/rnfv3cgvrrW/aFyjDRaBHa68xPEEspDBKisEU77PjVxFAdVxPeurU93LD7Ni1I3ru/tqR/kEEN8dMzf11OX4PvrHtOemPPXvFsEkaCOs+m3N4u4MPQ7NVHpbTgfMJMN46EkxVcwwNi1tNfHE0zF0iP9S3w== X-YMail-OSG: fNjsgmAVM1lPNS_3cHIuceXRR_W9cnPBVoCMCbikvuWDIxQaL_li2ZXLXf9RyJs NV6Gmn3PbSl9E84.202DcRR45Lrbv8iXA4L6l7je5owJx8dfIPFemFzcM4EvhJH6IANsSmc2WmM_ TvO7y0B1HuyLs25eYBUbpHUkZuFC2zyfTA4wi3U8cVT9JjQgrw3WmwC.ZKyujDBJ5YhANgB4lMZG ND9lLbP1XZYq8KU2TkmHHEYyxwgo8IsrvMc8Xjs7cKZXaY_og76vqcr17_rNKwm.2o9VGBSLTH7x 4CGJAIKHNs578Ij1afsomuHNQpXU4iXyvaPmXvO71OxkBm8c_EndxYCueh8xh8RjHREcnRLqMhXw NJ6HWv3W7X2E7Sgrv3fIVUWM54RBHHjzLVSMTKvYKX66fkELG.B6HM041NO1KmdpwHENH7wig.ze uS49Vw5wpLUoZhSGFXa306.q3Mls7N.medMITVCuMS5V5z9yBKEDdlayZbcDPgIhs90cGScUZMRO J42y3sOvBAZAdS.fhHUxuqXDaJc0v2zNhTVKIeKgevupLZlqoeSzSIU6GNuNp.ERtCmXCOnf7ak0 u6PQF1t4cPERr.xCpXfVi7iW6mIEVjPcLRlsERdMAOoMO1Q0VJTCEkHP97wzXh8NgGJWt1akXYwh DhiR0x78Wt5EkSaqDcBCxWm6V25GSewFJ2QbmD11tmWsF3UZq7tAK1pcINKjwdHYybHZ5p0Hw0o8 XZxUAmnL_kbDTGh2xNQy3t7S9wAlUeYnvjWq6xMPhKs.vQh5_J8QtFcgj789VKjQAZHZhQgC1_rw LU3ebuPu.OkZtkFJNeJJ0q7c_.BkZaONgfB9jd0XZu X-Sonic-MF: Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.bf2.yahoo.com with HTTP; Wed, 30 Jun 2021 15:10:39 +0000 Original-Received: by kubenode518.mail-prod1.omega.ir2.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID f4b54442e6dfa5939f3d42a2a6a48fdb; Wed, 30 Jun 2021 15:10:37 +0000 (UTC) Content-Disposition: inline In-Reply-To: <83sg0zmsf1.fsf@gnu.org> X-Mailer: WebService/1.1.18469 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.aol X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:209205 Archived-At: On Wed, Jun 30, 2021 at 03:46:26PM +0300, Eli Zaretskii wrote: >> Date: Wed, 30 Jun 2021 00:21:42 +0200 >> From: Ergus >> Cc: 49264@debbugs.gnu.org >> >> >AFAICT, most of the time is taken by 'apply', but the profile doesn't >> >show which function is called by 'apply'. Can you tell which function >> >is that? >> > >> apply is called in vc-responsible-backend and looking at the percentages >> it is obvious that the times are going in >> >> 23% + vc-svn-responsible-p >> 18% + vc-bzr-responsible-p >> 15% + vc-hg-responsible-p >> 6% + vc-git-responsible-p >> 2% + vc-cvs-responsible-p >> 2% + vc-rcs-responsible-p >> 1% + vc-sccs-responsible-p >> 1% + vc-src-responsible-p >> ------ >> 68% >> >> These are the backends' functions that are passed to vc-call-backend in >> the mapcar in vc-responsible-backend and the output of >> vc-find-backend-function. > >But if you still need to wait for dozens of seconds with just 2 >backends, which take only 20% of the time according to the above, then >how do you want to speed this up in your case? How about not using >ivy (which AFAICT is the root cause of this slowness)? > In the test Dimitry requested I sent the numbers of eval: (benchmark 1 '(project-current)) Only with git and mercurial it takes: Elapsed time: 3.018998s (0.109912s in 1 GCs) With the entire list in vc-handled-backends Elapsed time: 8.197923s (0.507396s in 6 GCs) So, ivy is not the problem. >> >... after removing the "unused" VC back-ends, you say that the code >> >still runs very slowly. So is the issue with VC back-ends still >> >relevant, and if so, how? >> > >> Yes, it is. It is still slow, lets say around 20-40 seconds to complete >> the command. But that's that's much faster than before (3-5 minutes); >> but still too slow to make the command usable. > >It's still unacceptably slow, so that couldn't be a solution, >certainly not a general one, IMO. > Agree, but it is another symptom about where the problem is. >> As a note here, when N files are in the same directory the normal thing >> is that all of them share the VCS. So calling a check function for all >> of them is redundant and slow. > >AFAIR, that's not really true, and ISTR project.el aims to support the >use cases with several different VC backends. > I can't speak for all the possible configurations; but IMO the most common configurations around are projects with single vcs OR projects with subprojects (git submodules) in different subdirs I am not aware of any project where different vcs are in the same root directory... For sure I have never seen that... For example something like: project0 |- file0-0 |- dir0 | |- file0-1 | |- file0-1 | |- subproject1 | |- file1-1 | |- file1-2 | |- dir1 | |- subproject2 |- file2-1 |- file2-2 |- dir2 | |- file2-3 | |- subproject3 |- file3-1 |- dir2 |- file3-2 Again: IMHO if we support this configuration we support 99% of the projects around. And I already proposed an option (using the default-dir as cache index) that already cover this with no functionality penalty. >And again, waiting for 30 seconds when you have just 10 buffers is >unacceptable. E.g., in the session where I'm writing this, I have 80 >buffers that visit files in a single branch of the Emacs Git >repository, almost an order of magnitude more than your 10 buffers. > With my solution we only impact performance: 1) The first time we call the command AND 2) All the 80 files are all in different directories. Otherwise there will be some benefit either for files located in the same directory and/or the next time you call the command (or open a file in one of the directories emacs already knows about.). >So we must find some better way of getting reasonable performance in >this use case. If the round-trip of the VC backend to a remote >filesystem is the bottleneck, let's try to speed that up in some way. In case you are interested about the issue with the external package: https://github.com/FelipeLema/emacs-counsel-gtags/issues/29