From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ergus via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#49264: 28.0.50; project.el+tramp performance issue Date: Tue, 29 Jun 2021 00:11:00 +0200 Message-ID: <87fsx13aiz.fsf@aol.com> References: <87fsx13aiz.fsf.ref@aol.com> Reply-To: Ergus Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2160"; mail-complaints-to="usenet@ciao.gmane.io" To: 49264@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jun 29 00:12:14 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lxzUD-0000Of-Is for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 29 Jun 2021 00:12:13 +0200 Original-Received: from localhost ([::1]:55682 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lxzUC-0001TS-EN for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 28 Jun 2021 18:12:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lxzU2-0001Lr-Q0 for bug-gnu-emacs@gnu.org; Mon, 28 Jun 2021 18:12:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:41681) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lxzU2-0000CR-Hb for bug-gnu-emacs@gnu.org; Mon, 28 Jun 2021 18:12:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lxzU2-00025e-9P for bug-gnu-emacs@gnu.org; Mon, 28 Jun 2021 18:12:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ergus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 28 Jun 2021 22:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 49264 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.16249182927997 (code B ref -1); Mon, 28 Jun 2021 22:12:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 28 Jun 2021 22:11:32 +0000 Original-Received: from localhost ([127.0.0.1]:53227 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxzTX-00024u-8G for submit@debbugs.gnu.org; Mon, 28 Jun 2021 18:11:31 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:40232) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxzTT-00024m-Op for submit@debbugs.gnu.org; Mon, 28 Jun 2021 18:11:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lxzTS-0007w9-1p for bug-gnu-emacs@gnu.org; Mon, 28 Jun 2021 18:11:27 -0400 Original-Received: from sonic302-3.consmr.mail.bf2.yahoo.com ([74.6.135.42]:41670) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lxzTN-000887-HB for bug-gnu-emacs@gnu.org; Mon, 28 Jun 2021 18:11:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aol.com; s=a2048; t=1624918279; bh=yXMBLpUkKtciNPAdYKF0xinWWVzc328j8BDTdQPc5Fg=; h=From:To:Cc:Subject:Date:References:From:Subject:Reply-To; b=fyj53isZtmkLBs/SBIJds6xb/TTHdfQlSJohVPmzIpxO1X+4jdKM29WXLpgRHC7y2t8ywvKXcyZ1O02vgJlkoTQVIhsQlmtsGC+KCg+Vg+3hkN/e1egPgJtGgStdmQuNnReSYq+Zvmh0yRVnrk8gj0HCnvrXMKunzumkGHyBq3n7yckfnbVFP3XuzyswpC0PBywSLSifYMGXbTkH5ILhvdCibwusZItX5bSrWj9NC83p9dh1Ogi8Ht+Z7zh6rAiQPTYf1sDZqaNrD3mqLtkD4HuMbdSawXVA/quG1Neb4c34kHCXXvNBIrcZHe4o4/6iNNTd6o0TFiAZed+eDQBEJA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1624918279; bh=wH2frLZevEKxC6JMKP/jJVQ6D2KA+qNU9BTZIJDvhRw=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=IT2PycGsTWQvWFs4adtBuIonpY0nwKUkVWh9PFy9IC821DKe+iFb4f34ZsSFmTW6JNYT2xicLq+Xdi4kbZaw+hfAYGEZCexAvTFGfUEQcQma7wLwSP08D9AMhqcH+BEAbzRTEkNBKp6FhPUKMXUey/9aE5TPEeH2I80F9DXyZpBDJmnDDs6j7EtTDHmENXGAmlM5V+dZBzt4lqYR6NO9I+ViCn6/EJw435sUSJFXF0DvyJpgyhsm72Ex4VmtKTuNYbBKeeBB0R1zRsVHyIN7mqHck/43WAlnrW+KEeqaQq71AkkwFoJeIcvdZ9jj0tGCAKE9Mu0x44be3d+5h3VqbA== X-YMail-OSG: WN7SN1wVM1nzAbeJV7E9LtPCBrHIeQf1a_NwYHycAAr9lT1ZAUR.9E1zO8PmMY3 Jh.hpjMzdiJfWr7bQjkGK.3EC90wTcaknCiARy.3qwdVFW0GywARC9g3R6MjAw.w98evw_lu8aLU Ay9vLk6AdphvGMCx33dMl0c7XSKQ68nqIs3wqczQtLpxWFLSFJpwtM1lPZ_TL3Xn5sqKysdjkTJv rHSjG4rpTqlHjGhW5fnr7QfJN5H5NX_kPK9flWp16vucEHTQ2xEKn9xVdTtIZjzGl37UlzKzlJEF xelCSN.kfcwHUlADEi.e_6WiXZAOFDVi91fZE4vqSTJL1Iw7xVn3ubjGbXMxLRRzRqGlsZxh40Wa 3wudgkttiTF..w0ijj0Ymtx.n7XzfQVHeH2SbcoEvqyW9FkM2wiqiZyTtN_4n7O9EMU7IFBbvBtm ERpayLBrhESKraTf_Ag_5IW46gv6URVB5IKsfwYNxa2swoDsPa2hIcDX9h_Tpea4lhCL_Lae0TER 3kc.OU.bgb1mFmgDyAUc34Dp09P1B1i0ZD0l0fKJEfVZXq5S.PextjLfZowffriQNrW_NGWcE.mS KIFjzU15QbSSqKVL653T.oXidTtbUlnTB37H1w9jqZAz8tnxw7QNgR_NHvgg0JlvaCNAOsATBgjy 2PDQlmyLwU_8PB6EGozmihaHz3udwViNMNv9rXGj7SKyYM9X9OfJw0SM96LsvRpI7Y1hOOozX5UO EXH6A8xA8ip0oTLbetege0YM9e6YqugsniGoOnQz2fr2DUTeboyr3eyCECOyWqXgEQpRzDE18i84 PgmPnAbYSf37ZmpM4UgBfXKa.W8Q02Vh.ZSGYrJgw5 X-Sonic-MF: Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic302.consmr.mail.bf2.yahoo.com with HTTP; Mon, 28 Jun 2021 22:11:19 +0000 Original-Received: by kubenode503.mail-prod1.omega.ir2.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID e2ff5b4c72c268af2e9aa79d29da4757; Mon, 28 Jun 2021 22:11:17 +0000 (UTC) X-Mailer: WebService/1.1.18469 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.aol Received-SPF: pass client-ip=74.6.135.42; envelope-from=spacibba@aol.com; helo=sonic302-3.consmr.mail.bf2.yahoo.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:209083 Archived-At: Hi: Using tramp I tried to use project.el with a command like project-switch-to-buffer and it took like 10 minutes to complete. I ran a profiler and I found that most of the time was taken by an external function: global-tags-try-project-root project-current is called in a loop for all the opened buffers it calls project--find-in-directory that calls project-find-functions and there is going all the time. After some optimization in an external package; now the time is half than before but still very slow to use the command (around 3-5 minutes to complete) and running again the profiler I get this: 5637 89% - command-execute 5549 88% - byte-code 5549 88% - project--read-project-buffer 5549 88% - let* 5336 85% - read-buffer 5323 84% - ivy-completing-read 5323 84% - ivy-read 4941 78% - ivy--reset-state 4941 78% - ivy--buffer-list 4941 78% - internal-complete-buffer 4941 78% - # 4941 78% - and 4941 78% - equal 4941 78% - save-current-buffer 4941 78% - project-current 4941 78% - project--find-in-directory 4548 72% - project-try-vc 4537 72% - vc-responsible-backend 4478 71% - # 4478 71% - vc-call-backend 4478 71% - apply 1470 23% + vc-svn-responsible-p 1142 18% + vc-bzr-responsible-p 970 15% + vc-hg-responsible-p 390 6% + vc-git-responsible-p 156 2% + vc-cvs-responsible-p 126 2% + vc-rcs-responsible-p 108 1% + vc-sccs-responsible-p 98 1% + vc-src-responsible-p 57 0% + tramp-file-name-handler 11 0% + vc-file-getprop 393 6% + global-tags-try-project-root 375 5% + read-from-minibuffer 13 0% + if 213 3% + project-current 88 1% + funcall-interactively 572 9% + ... 51 0% + timer-event-handler 8 0% + redisplay_internal (C function) As you can see most of the time is still taken by project-current and I can't really understand why: 1) Are so many samples 4548 seems a very high number for only 25 opened buffers. 2) why project-try-vc still takes so much...? Specially for unfrequent vc systems in our days like svn or bzr that I am not using. As a workaround I removed all the uninteresting handlers from vc-handled-backends and I get better times now, but IMHO it is still very inefficient (almost a minute for project-switch-to-buffer is excessive). And make it practically unusable. In any case: Maybe (I think I mentioned this before) `project.el` needs a sort of cache to speedup some functions like `project-current` that are called very frequently inside the project.el code. Related with https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42966 VCS changing is not something that happens very often to require a check of all the backends everytime, several times for every buffer in many project.el functions right? Specially when using tramp. vc has vc-file-prop-obarray; maybe vc-responsible-backend should cache it's result there to avoid repeating time consuming computations? Either in the local system the performance penalty seems to be significant to me. Best, Ergus In GNU Emacs 28.0.50 (build 50, x86_64-pc-linux-gnu, GTK+ Version 3.24.29, cairo version 1.17.4) of 2021-06-26 built on Ergus Repository revision: b8f9e58ef72402e69a1f0960816184d52e5d2d29 Repository branch: master System Description: Arch Linux Configured using: 'configure --prefix=/home/ergo/.local/ --with-mailutils --with-json --with-x-toolkit=gtk3 --with-xft --with-wide-int --with-modules --with-cairo --with-harfbuzz --with-native-compilation' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: C++//law Minor modes in effect: global-git-commit-mode: t magit-auto-revert-mode: t diff-hl-margin-mode: t windmove-mode: t subword-mode: t hide-ifdef-mode: t preproc-font-lock-mode: t shell-dirtrack-mode: t show-paren-mode: t global-auto-revert-mode: t xclip-mode: t yas-global-mode: t yas-minor-mode: t electric-pair-mode: t flyspell-mode: t company-mode: t flycheck-mode: t counsel-mode: t ivy-mode: t composable-mark-mode: t composable-mode: t repeat-mode: t xterm-mouse-mode: t winner-mode: t save-place-mode: t which-key-mode: t override-global-mode: t delete-selection-mode: t savehist-mode: t global-display-fill-column-indicator-mode: t display-fill-column-indicator-mode: t global-display-line-numbers-mode: t display-line-numbers-mode: t mouse-wheel-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t size-indication-mode: t column-number-mode: t line-number-mode: t transient-mark-mode: t abbrev-mode: t Load-path shadows: /usr/share/emacs/site-lisp/cmake-mode hides /home/ergo/.emacs.d/elpa/cmake-mode-20210104.1831/cmake-mode /usr/share/emacs/site-lisp/notmuch-crypto hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-crypto /usr/share/emacs/site-lisp/notmuch-compat hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-compat /usr/share/emacs/site-lisp/notmuch-hello hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-hello /usr/share/emacs/site-lisp/notmuch hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch /usr/share/emacs/site-lisp/notmuch-show hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-show /usr/share/emacs/site-lisp/notmuch-maildir-fcc hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-maildir-fcc /usr/share/emacs/site-lisp/coolj hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/coolj /usr/share/emacs/site-lisp/notmuch-draft hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-draft /usr/share/emacs/site-lisp/notmuch-tree hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-tree /usr/share/emacs/site-lisp/notmuch-parser hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-parser /usr/share/emacs/site-lisp/notmuch-lib hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-lib /usr/share/emacs/site-lisp/notmuch-mua hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-mua /usr/share/emacs/site-lisp/notmuch-message hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-message /usr/share/emacs/site-lisp/notmuch-address hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-address /usr/share/emacs/site-lisp/notmuch-wash hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-wash /usr/share/emacs/site-lisp/notmuch-tag hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-tag /usr/share/emacs/site-lisp/notmuch-print hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-print /usr/share/emacs/site-lisp/notmuch-query hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-query /usr/share/emacs/site-lisp/notmuch-jump hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-jump /usr/share/emacs/site-lisp/notmuch-company hides /home/ergo/.emacs.d/elpa/notmuch-20210627.1741/notmuch-company /home/ergo/.emacs.d/elpa/transient-20210619.1100/transient hides /home/ergo/.local/share/emacs/28.0.50/lisp/transient Features: (shadow sort notmuch-company notmuch-lib notmuch-version notmuch-compat mm-view mml-smime smime dig mail-extr emacsbug sendmail magit-extras hi-lock magit-bookmark magit-submodule magit-obsolete magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit magit-sequence magit-notes magit-worktree magit-tag magit-merge magit-branch magit-reset magit-files magit-refs magit-status magit package browse-url url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util mailcap url-handlers magit-repos magit-apply magit-wip magit-log which-func imenu magit-diff smerge-mode diff git-commit log-edit message rmc puny rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader add-log magit-core magit-autorevert magit-margin magit-transient magit-process with-editor server magit-mode transient magit-git magit-section magit-utils crm tramp-cmds global-tags ht generator async counsel-gtags pulse mc-separate-operations mc-edit-lines mc-hide-unmatched-lines-mode mc-mark-more mc-cycle-cursors multiple-cursors-core rect move-dup diff-hl-margin eieio-opt speedbar ezimage dframe shortdoc help-fns radix-tree vc-annotate amx s windmove misearch multi-isearch ffap url-parse url-vars face-remap vc-hg macrostep-c cmacexp macrostep cap-words superword subword hideif preproc-font-lock cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs dired-aux diff-hl-dired diff-hl log-view pcvs-util vc-dir ewoc vc tramp-cache tramp-sh tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat shell pcomplete parse-time iso8601 time-date ls-lisp format-spec auth-source password-cache thingatpt vc-git diff-mode vc-dispatcher bookmark pp paren autorevert filenotify xclip yasnippet-snippets yasnippet elec-pair flyspell-correct-ivy flyspell-correct flyspell ispell company-keywords company-gtags company-dabbrev-code company-dabbrev company-files company-semantic company-template company-capf company flycheck json map find-func dash counsel xdg xref project dired-x dired dired-loaddefs compile text-property-search comint ansi-color swiper ivy-avy avy ivy flx ivy-faces ivy-overlay colir pcase term/tmux term/xterm xterm jka-compr init composable composable-mark powerline comp comp-cstr warnings subr-x powerline-separators color powerline-themes repeat xt-mouse simple-16-theme winner ring saveplace diminish edmacro kmacro which-key advice configmail cl-extra help-mode use-package use-package-ensure use-package-delight use-package-diminish use-package-bind-key bind-key use-package-core disp-table delsel savehist easy-mmode display-fill-column-indicator display-line-numbers info ede/auto eieio-base cl-seq seq eieio byte-opt bytecomp byte-compile cconv eieio-core cl-macs gv eieio-loaddefs cl-loaddefs cl-lib tex-site rx slime-autoloads early-init iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 811583 75249) (symbols 48 30651 0) (strings 32 107218 13868) (string-bytes 1 4408538) (vectors 16 57480) (vector-slots 8 1324399 45245) (floats 8 342 1198) (intervals 56 53473 1428) (buffers 992 37))