bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi Dmitry and Michael:

I enabled the project-mode-line recently and now when I try to open a
remote file emacs seems frozen forever (minutes in a not so bad
connection).

I made a profile and it looks like there is some bad interaction between
project-mode-line and tramp names

I already tried:

(connection-local-set-profile-variables 'my-profile '((project-mode-line . nil)))
(connection-local-set-profiles '(:application tramp :user "myuser") 'my-profile)

But is not disabling the modeline for remote files.

And

vc-ignore-dir-regexp (format "\\(%s\\)\\|\\(%s\\)" ;; disable vc on remotes
					   vc-ignore-dir-regexp
					   tramp-file-name-regexp)

But the project-try-vc is still trying in the remote files.

The profile report:

15458  75% - redisplay_internal (C function)
15442  75%  - eval
15442  75%   - project-mode-line-format
15442  75%    - project-current
15442  75%     - project--find-in-directory
15442  75%      - run-hook-with-args-until-success
15442  75%       - project-try-vc
15442  75%        - project--value-in-dir
15442  75%         - hack-dir-local-variables
15442  75%          - #<byte-code-function 8FF>
15442  75%           - hack-dir-local--get-variables
15442  75%            - dir-locals-find-file
14648  71%             - locate-dominating-file
14644  71%              - abbreviate-file-name
14644  71%               - tramp-file-name-handler
14637  71%                - apply
14637  71%                 - tramp-sh-file-name-handler
14611  71%                  - tramp-handle-insert-file-contents
14611  71%                   - file-local-copy
14611  71%                    - tramp-file-name-handler
14611  71%                     - apply
14611  71%                      - tramp-sh-file-name-handler
14611  71%                       - tramp-sh-handle-file-local-copy
14611  71%                        - tramp-barf-unless-okay
14611  71%                         - tramp-send-command-and-check
14611  71%                          - tramp-send-command
14611  71%                           - tramp-wait-for-output
14611  71%                            - tramp-wait-for-regexp
14611  71%                             - sit-for
14611  71%                              - redisplay_internal (C function)
14574  71%                               - eval
14574  71%                                - project-mode-line-format
14574  71%                                 - project-current
14574  71%                                  - project--find-in-directory
14574  71%                                   - run-hook-with-args-until-success
14574  71%                                    - project-try-vc
14530  70%                                     - project--value-in-dir
14530  70%                                      - hack-dir-local-variables
14530  70%                                       - #<byte-code-function 08F>
14530  70%                                        - hack-dir-local--get-variables
14523  70%                                         - dir-locals-find-file
 9766  47%                                          - locate-dominating-file
 7983  38%                                           - dir-locals--all-files
 7964  38%                                            - tramp-file-name-handler
 7909  38%                                             - apply
 7906  38%                                              - tramp-sh-file-name-handler
 5131  25%                                               - tramp-sh-handle-file-readable-p
 5052  24%                                                - tramp-run-test
 5052  24%                                                 - tramp-send-command-and-check
 5052  24%                                                  - tramp-send-command
 5052  24%                                                   - tramp-wait-for-output
 5052  24%                                                    - tramp-wait-for-regexp
 3188  15%                                                     - tramp-accept-process-output
 2193  10%                                                        accept-process-output
  201   0%                                                      + #<byte-code-function 0F2>
  201   0%                                                      + tramp-get-connection-property
  126   0%                                                      + tramp-set-connection-property
   59   0%                                                        make-closure
   40   0%                                                        tramp-message
   38   0%                                                      + #<byte-code-function FA1>
   35   0%                                                        buffer-string
   13   0%                                                        process-buffer
   12   0%                                                        process-status
   11   0%                                                        with-timeout-suspend
    9   0%                                                        process-get
 1340   6%                                                       sit-for
  481   2%                                                     + tramp-check-for-regexp
   33   0%                                                       process-live-p
   41   0%                                                + tramp-file-name-handler
   22   0%                                                + tramp-get-file-property
   13   0%                                                + tramp-dissect-file-name
 1822   8%                                               + tramp-handle-file-regular-p
  912   4%                                               + tramp-sh-handle-file-directory-p
   38   0%                                               + tramp-sh-handle-expand-file-name
    3   0%                                                 #<byte-code-function 9AA>
    3   0%                                                tramp-file-name-for-operation
   27   0%                                             + tramp-dissect-file-name
   11   0%                                               tramp-tramp-file-p
    4   0%                                             + tramp-replace-environment-variables
    3   0%                                               tramp-file-name-equal-p
    3   0%                                             + #<byte-code-function 152>
 1774   8%                                           + tramp-file-name-handler
 4734  23%                                          - dir-locals--all-files
 4731  23%                                           - tramp-file-name-handler
 4679  22%                                            - apply
 4679  22%                                             - tramp-sh-file-name-handler
 2659  12%                                              - tramp-sh-handle-file-readable-p
 2634  12%                                               - tramp-run-test
 2634  12%                                                - tramp-send-command-and-check
 2634  12%                                                 - tramp-send-command
 2634  12%                                                  - tramp-wait-for-output
 2634  12%                                                   - tramp-wait-for-regexp
 1642   8%                                                    + tramp-accept-process-output
  700   3%                                                      sit-for
  276   1%                                                    + tramp-check-for-regexp
    6   0%                                                      process-live-p
   19   0%                                               + tramp-file-name-handler
    6   0%                                               + tramp-get-file-property
 1356   6%                                              + tramp-handle-file-regular-p
  629   3%                                              + tramp-sh-handle-file-directory-p
   35   0%                                              + tramp-sh-handle-expand-file-name
   16   0%                                            + tramp-dissect-file-name
   14   0%                                              tramp-tramp-file-p
    8   0%                                            + tramp-file-name-equal-p
    4   0%                                            + #<byte-code-function B99>
    4   0%                                              tramp-flush-connection-property
    3   0%                                              make-closure
   23   0%                                          + tramp-file-name-handler
    7   0%                                         + dir-locals-collect-variables
   44   0%                                     + locate-dominating-file
    4   0%                               + file-remote-p
   15   0%                  + tramp-sh-handle-set-visited-file-modtime
   11   0%                  + tramp-handle-abbreviate-file-name
    4   0%                + tramp-replace-environment-variables
    3   0%                  tramp-dissect-file-name
    4   0%                tramp-file-name-handler
  794   3%             + tramp-file-name-handler
   10   0%  + file-remote-p
 4838  23%   Automatic GC
  170   0% + command-execute
    4   0% + ...

At the moment I disabled the project-mode-line. However There may be
a better solutions for this issue.

Best,
Ergus


In GNU Emacs 31.0.50 (build 16, x86_64-pc-linux-gnu, GTK+ Version
 3.24.43, cairo version 1.18.2) of 2024-10-08 built on RTX
Repository revision: 7f70165ad6959193266df1da5b1ce53c34c72d47
Repository branch: project
System Description: Arch Linux

Configured using:
 'configure --prefix=/home/ergo/.local/ --with-mailutils --with-pgtk
 --with-modules --with-cairo --with-harfbuzz
 --with-native-compilation=aot
 '--program-transform-name=s/^ctags$/ctags.emacs/''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  fancy-compilation-mode: t
  global-auto-revert-mode: t
  electric-pair-mode: t
  whitespace-mode: t
  flyspell-mode: t
  completion-preview-mode: t
  diff-hl-margin-local-mode: t
  diff-hl-margin-mode: t
  diff-hl-mode: t
  corfu-terminal-mode: t
  global-corfu-mode: t
  corfu-mode: t
  repeat-mode: t
  xterm-mouse-mode: t
  xclip-mode: t
  override-global-mode: t
  winner-mode: t
  save-place-mode: t
  delete-selection-mode: t
  savehist-mode: t
  global-display-fill-column-indicator-mode: t
  display-fill-column-indicator-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  which-key-mode: t
  eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/mnt/casa/gits/emacs_clones/cuda-mode/cuda-mode hides /home/ergo/.config/emacs/elpa/cuda-mode-20240819.11/cuda-mode
/home/ergo/.config/emacs/elpa/transient-20241006.2150/transient hides /home/ergo/.local/share/emacs/31.0.50/lisp/transient

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny rfc822
mml mml-sec epa derived epg rfc6068 epg-config gnus-util mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
misearch multi-isearch vc-git help-fns radix-tree profiler dired-aux
hl-line tramp-cmds fancy-compilation compile text-property-search
comp-run comp-common tramp-cache time-stamp tramp-sh tramp trampver
tramp-integration files-x tramp-message tramp-compat xdg shell pcomplete
comint ansi-osc parse-time iso8601 time-date format-spec ansi-color
tramp-loaddefs autorevert filenotify ffap thingatpt url-parse
auth-source eieio eieio-core cl-macs icons password-cache json map
url-vars elec-pair whitespace flyspell-correct flyspell ispell
completion-preview diff-hl-margin diff-hl-dired citre-lang-fileref
citre-tags citre-ctags citre-readtags citre-readtags-tables
citre-backend-interface citre-common-tag rx citre-common-util project
dired-x dired dired-loaddefs diff-hl log-view pcvs-util vc-dir ewoc vc
vc-dispatcher diff-mode track-changes corfu-terminal popon corfu
term/xterm xterm init repeat markdown-ts-mode subr-x treesit cape compat
use-package-ensure use-package-diminish xt-mouse xclip edmacro kmacro
byte-opt gv use-package-bind-key bind-key cl-extra help-mode
simple-16-theme winner ring saveplace delsel savehist easy-mmode
display-fill-column-indicator display-line-numbers diminish which-key
cl-seq use-package-core cl-loaddefs cl-lib bytecomp byte-compile
disp-table info 0blayout-autoloads ac-emoji-autoloads
ac-haskell-process-autoloads ac-html-autoloads
arduino-cli-mode-autoloads auctex-autoloads tex-site
auto-complete-autoloads avy-zap-autoloads avy-autoloads
better-shell-autoloads caml-autoloads cape-autoloads citre-autoloads
clang-format-autoloads cobol-mode-autoloads compile-multi-autoloads
corfu-terminal-autoloads corfu-autoloads crdt-autoloads
csv-mode-autoloads cuda-mode-autoloads d-mode-autoloads
deadgrep-autoloads debbugs-autoloads diff-hl-autoloads
diminish-autoloads dired-sidebar-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads dumb-jump-autoloads e2ansi-autoloads
emamux-autoloads esup-autoloads evil-collection-autoloads
annalist-autoloads evil-leader-autoloads evil-autoloads
face-explorer-autoloads fancy-compilation-autoloads flx-autoloads
flycheck-julia-autoloads flycheck-rust-autoloads flycheck-autoloads
flymake-nasm-autoloads flymake-quickdef-autoloads
flyspell-correct-autoloads git-modes-autoloads git-timemachine-autoloads
gnuplot-autoloads google-c-style-autoloads goto-chg-autoloads
groovy-mode-autoloads gtags-mode-autoloads haskell-mode-autoloads
highlight-indent-guides-autoloads i3wm-config-mode-autoloads
ibuffer-sidebar-autoloads iedit-autoloads imenu-list-autoloads
julia-ts-mode-autoloads julia-mode-autoloads languagetool-autoloads
lice-autoloads lorem-ipsum-autoloads lua-mode-autoloads magit-autoloads
magit-section-autoloads markdown-mode-autoloads
markdown-ts-mode-autoloads modern-cpp-font-lock-autoloads
move-dup-autoloads multiple-cursors-autoloads mutt-mode-autoloads
nasm-mode-autoloads neotree-autoloads nftables-mode-autoloads
nginx-mode-autoloads notmuch-autoloads objed-autoloads
opencl-mode-autoloads paradox-autoloads phi-search-autoloads
pkg-info-autoloads epl-autoloads pkgbuild-mode-autoloads
platformio-mode-autoloads async-autoloads popon-autoloads
popup-autoloads projectile-autoloads projection-autoloads
protobuf-mode-autoloads protobuf-ts-mode-autoloads
ptemplate-templates-autoloads ptemplate-autoloads
py-vterm-interaction-autoloads scopeline-autoloads
shell-command+-autoloads slime-autoloads macrostep-autoloads
sphinx-mode-autoloads f-autoloads dash-autoloads s-autoloads
spinner-autoloads ssh-config-mode-autoloads string-inflection-autoloads
sudo-edit-autoloads systemd-autoloads tmux-mode-autoloads
transient-autoloads tsc-autoloads urgrep-autoloads vdiff-autoloads
hydra-autoloads lv-autoloads vterm-toggle-autoloads vterm-autoloads
vundo-autoloads with-editor-autoloads xclip-autoloads
yasnippet-snippets-autoloads yasnippet-autoloads early-init rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
term/common-win touch-screen pgtk-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty move-toolbar make-network-process native-compile emacs)

Memory information:
((conses 16 235994 86287) (symbols 48 16860 0)
 (strings 32 59529 11399) (string-bytes 1 2092584) (vectors 16 30418)
 (vector-slots 8 1023066 183728) (floats 8 113 783)
 (intervals 56 7143 582) (buffers 992 19))

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2320 bytes --]

Ergus <spacibba@aol.com> writes:

> Hi Dmitry and Michael:

Hi Ergus,

> I enabled the project-mode-line recently and now when I try to open a
> remote file emacs seems frozen forever (minutes in a not so bad
> connection).
>
> I made a profile and it looks like there is some bad interaction between
> project-mode-line and tramp names
>
> The profile report:
>
> 15458  75% - redisplay_internal (C function)
> 15442  75%  - eval
> 15442  75%   - project-mode-line-format
> 15442  75%    - project-current
> 15442  75%     - project--find-in-directory
> 15442  75%      - run-hook-with-args-until-success
> 15442  75%       - project-try-vc
> 15442  75%        - project--value-in-dir
> 15442  75%         - hack-dir-local-variables
> 15442  75%          - #<byte-code-function 8FF>
> 15442  75%           - hack-dir-local--get-variables
> 15442  75%            - dir-locals-find-file
> 14648  71%             - locate-dominating-file
> 14644  71%              - abbreviate-file-name
> 14644  71%               - tramp-file-name-handler
> 14637  71%                - apply
> 14637  71%                 - tramp-sh-file-name-handler
> 14611  71%                  - tramp-handle-insert-file-contents
> 14611  71%                   - file-local-copy
> 14611  71%                    - tramp-file-name-handler
> 14611  71%                     - apply
> 14611  71%                      - tramp-sh-file-name-handler
> 14611  71%                       - tramp-sh-handle-file-local-copy
> 14611  71%                        - tramp-barf-unless-okay
> 14611  71%                         - tramp-send-command-and-check
> 14611  71%                          - tramp-send-command
> 14611  71%                           - tramp-wait-for-output
> 14611  71%                            - tramp-wait-for-regexp
> 14611  71%                             - sit-for
> 14611  71%                              - redisplay_internal (C function)
> 14574  71%                               - eval
> 14574  71%                                - project-mode-line-format

This is unfortune. sit-for should not trigger a redisplay. Does the
appended patch in tramp.el helps?

Best regards, Michael.

> At the moment I disabled the project-mode-line. However There may be
> a better solutions for this issue.
>
> Best,
> Ergus

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 411 bytes --]

diff --git a/lisp/tramp.el b/lisp/tramp.el
index 4ce44119..181bba01 100644
--- a/lisp/tramp.el
+++ b/lisp/tramp.el
@@ -6049,7 +6049,7 @@ nil."
     (with-tramp-timeout (timeout)
       (while (not found)
 	;; This is needed to yield the CPU, otherwise we'll see 100% CPU load.
-	(sit-for 0)
+	(sit-for 0 'nodisp)
 	(tramp-accept-process-output proc)
 	(unless (process-live-p proc)
 	  (tramp-error-with-buffer

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Dmitry Gutov]

Hi!

On 09/10/2024 13:43, Ergus via Bug reports for GNU Emacs, the Swiss army 
knife of text editors wrote:
> I made a profile and it looks like there is some bad interaction between
> project-mode-line and tramp names
> 
> I already tried:
> 
> (connection-local-set-profile-variables 'my-profile '((project-mode-line . nil)))
> (connection-local-set-profiles '(:application tramp :user "myuser") 'my-profile)
> 
> But is not disabling the modeline for remote files.

I wonder if we'll need to disable project-mode-line in remote files 
altogether, or if this is a problem that could be solved in Tramp (with 
extra caching?)

To use connect-local value to disable, BTW, it would need to have extra 
code called from the mode line format (in bindings.el). And it's a part 
that was added only recently, for in Emacs 30.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

I've added file-remote-p guards in a variety of places specifically to
avoid these issues.

On Wed, Oct 9, 2024 at 9:39 AM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Hi!
>
> On 09/10/2024 13:43, Ergus via Bug reports for GNU Emacs, the Swiss army
> knife of text editors wrote:
> > I made a profile and it looks like there is some bad interaction between
> > project-mode-line and tramp names
> >
> > I already tried:
> >
> > (connection-local-set-profile-variables 'my-profile '((project-mode-line
> . nil)))
> > (connection-local-set-profiles '(:application tramp :user "myuser")
> 'my-profile)
> >
> > But is not disabling the modeline for remote files.
>
> I wonder if we'll need to disable project-mode-line in remote files
> altogether, or if this is a problem that could be solved in Tramp (with
> extra caching?)
>
> To use connect-local value to disable, BTW, it would need to have extra
> code called from the mode line format (in bindings.el). And it's a part
> that was added only recently, for in Emacs 30.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1516 bytes --]

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 1895 bytes --]

To be clearer, I've guarded project-try-vc with advice. Perhaps a user
option to enable the guard intrinsically would be better as everyone could
benefit. I have my own option variables doing this so I can enable it when
wanted.

Perhaps Ergus can give something similar a try and see if this solves his
immediate hanging concern.

Ergus, you should be able to just drop the below snippet in your init file
(or eval in *scratch*) and see if it works for you.

  (defun my/project-try-vc-advice (orig-fun dir)
    (or (vc-file-getprop dir 'project-vc)
        (if (file-remote-p dir)
            nil
          (funcall orig-fun dir))))
  (advice-add 'project-try-vc :around #'my/project-try-vc-advice))


On Wed, Oct 9, 2024 at 9:45 AM Ship Mints <shipmints@gmail.com> wrote:

> I've added file-remote-p guards in a variety of places specifically to
> avoid these issues.
>
> On Wed, Oct 9, 2024 at 9:39 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
>> Hi!
>>
>> On 09/10/2024 13:43, Ergus via Bug reports for GNU Emacs, the Swiss army
>> knife of text editors wrote:
>> > I made a profile and it looks like there is some bad interaction between
>> > project-mode-line and tramp names
>> >
>> > I already tried:
>> >
>> > (connection-local-set-profile-variables 'my-profile
>> '((project-mode-line . nil)))
>> > (connection-local-set-profiles '(:application tramp :user "myuser")
>> 'my-profile)
>> >
>> > But is not disabling the modeline for remote files.
>>
>> I wonder if we'll need to disable project-mode-line in remote files
>> altogether, or if this is a problem that could be solved in Tramp (with
>> extra caching?)
>>
>> To use connect-local value to disable, BTW, it would need to have extra
>> code called from the mode line format (in bindings.el). And it's a part
>> that was added only recently, for in Emacs 30.
>>
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 3175 bytes --]

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Juri Linkov]

>> I made a profile and it looks like there is some bad interaction between
>> project-mode-line and tramp names
>> I already tried:
>> (connection-local-set-profile-variables 'my-profile '((project-mode-line
>> . nil)))
>> (connection-local-set-profiles '(:application tramp :user "myuser") 'my-profile)
>> But is not disabling the modeline for remote files.
>
> I wonder if we'll need to disable project-mode-line in remote files
> altogether, or if this is a problem that could be solved in Tramp (with
> extra caching?)
>
> To use connect-local value to disable, BTW, it would need to have extra
> code called from the mode line format (in bindings.el). And it's a part
> that was added only recently, for in Emacs 30.

Or maybe better to cache the value of project-name on remove projects.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi!

Hi Dmitry,

> I wonder if we'll need to disable project-mode-line in remote files
> altogether, or if this is a problem that could be solved in Tramp
> (with extra caching?)

What do you think should be cached in Tramp?

Best regards, Michael.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Ergus,

> This is unfortune. sit-for should not trigger a redisplay. Does the
> appended patch in tramp.el helps?

I've pushed the patch to master. It doesn't hurt, at least.

Best regards, Michael.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Dmitry Gutov]

On 10/10/2024 12:57, Michael Albinus via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
>> I wonder if we'll need to disable project-mode-line in remote files
>> altogether, or if this is a problem that could be solved in Tramp
>> (with extra caching?)
> What do you think should be cached in Tramp?

Not sure if more cache should be the answer here, but in the profile I 
see the hack-dir-local-variables call.

So first, it should do nothing unless enable-remote-dir-locals is set to 
t (right?), so my first guess would be Ergus has customized that var. 
Maybe try turning it off, does that lose some features that you use?

Second, I suppose Tramp could do some caching for the result of 
hack-dir-local--get-variables (just time-based), but I'm not sure how 
much it will help, on average. Only every 100th redisplay causing a 
stutter would still be a problem.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Dmitry Gutov]

On 09/10/2024 19:10, Juri Linkov wrote:
> Or maybe better to cache the value of project-name on remove projects.

Just the project->project-name mapping? Why not. I suppose there'd still 
be a pause when switching projects, but it's not as bad.

For general caching, from past threads it seems the most problematic 
case is "no project". Because OT1H it's still costly it terms of remote 
I/O. But on the other, this is exactly when the cache might get invalid 
soon (because the user will initialize a Git repo, or create another 
root marker, etc).

I guess we should come back to this after bug#72300.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

>>> I wonder if we'll need to disable project-mode-line in remote files
>>> altogether, or if this is a problem that could be solved in Tramp
>>> (with extra caching?)
>> What do you think should be cached in Tramp?
>
> Not sure if more cache should be the answer here, but in the profile I
> see the hack-dir-local-variables call.
>
> So first, it should do nothing unless enable-remote-dir-locals is set
> to t (right?), so my first guess would be Ergus has customized that
> var. Maybe try turning it off, does that lose some features that you
> use?

Neither Tramp nor connection-local variables need this. Perhaps, it is a
good idea idea to bind enable-remote-dir-locals to nil in
project-mode-line-format.

> Second, I suppose Tramp could do some caching for the result of
> hack-dir-local--get-variables (just time-based), but I'm not sure how
> much it will help, on average. Only every 100th redisplay causing a
> stutter would still be a problem.

A redisplay shouldn't happen anymore, at least not triggered by
Tramp. I've added the NODISP arg to the respective sit-for call.

Best regards, Michael.

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 2067 bytes --]

In addition to guarding project-try-vc, I also have a customized cache for
non-projects specifically to get around the incessant retesting for what
are effectively invariant conditions.

If of any utility to others:

(use-package project
...
  (defconst my:project--non-project-dir ".") ; nil disables the cache;
NOTE: eglot relies on project-root so this has to be a real path
  (defun my/project-try-non-project-cache (dir)
    (unless (and my:project-vc-inhibit-remote (file-remote-p dir)) ; yet
another optional remote guard
      (let ((proj (cons 'transient (or my:project--non-project-dir
(expand-file-name dir)))))
        (vc-file-setprop dir 'project-vc proj) ; project caches via vc
internal properties
        proj)))
  (add-to-list 'project-find-functions #'my/project-try-non-project-cache
'append))

  ;; customized project current name function that respects
  ;; non-project marker and returns nil if a non-project
  (defun my/project-current-name (&optional buf)
    "Return the current project name for BUF, or nil if a non-project.
If BUF is nil, the current buffer is used."
    (with-current-buffer (or buf (current-buffer))
      (when-let* ((p (project-current))
                  (pn (project-name p)))
        (unless (string= pn my:project--non-project-dir)
          pn))))
...
)

On Thu, Oct 10, 2024 at 8:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 09/10/2024 19:10, Juri Linkov wrote:
> > Or maybe better to cache the value of project-name on remove projects.
>
> Just the project->project-name mapping? Why not. I suppose there'd still
> be a pause when switching projects, but it's not as bad.
>
> For general caching, from past threads it seems the most problematic
> case is "no project". Because OT1H it's still costly it terms of remote
> I/O. But on the other, this is exactly when the cache might get invalid
> soon (because the user will initialize a Git repo, or create another
> root marker, etc).
>
> I guess we should come back to this after bug#72300.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 3145 bytes --]

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ship Mints]

[-- Attachment #1: Type: text/plain, Size: 2696 bytes --]

Forgot to add this counterpart for when you want to use a non-project
semaphore but also want to project-current to tell the truth about being a
non-project:

  (defun my/project-current-non-project-advice (orig-fun &rest args)
    (let ((p (apply orig-fun args)))
      (when (and p (string= (project-name p) my:project--non-project-dir))
        (setq p nil))
      p))
  (advice-add #'project-current :around
#'my/project-current-non-project-advice))

On Fri, Oct 11, 2024 at 10:35 AM Ship Mints <shipmints@gmail.com> wrote:

> In addition to guarding project-try-vc, I also have a customized cache for
> non-projects specifically to get around the incessant retesting for what
> are effectively invariant conditions.
>
> If of any utility to others:
>
> (use-package project
> ...
>   (defconst my:project--non-project-dir ".") ; nil disables the cache;
> NOTE: eglot relies on project-root so this has to be a real path
>   (defun my/project-try-non-project-cache (dir)
>     (unless (and my:project-vc-inhibit-remote (file-remote-p dir)) ; yet
> another optional remote guard
>       (let ((proj (cons 'transient (or my:project--non-project-dir
> (expand-file-name dir)))))
>         (vc-file-setprop dir 'project-vc proj) ; project caches via vc
> internal properties
>         proj)))
>   (add-to-list 'project-find-functions #'my/project-try-non-project-cache
> 'append))
>
>   ;; customized project current name function that respects
>   ;; non-project marker and returns nil if a non-project
>   (defun my/project-current-name (&optional buf)
>     "Return the current project name for BUF, or nil if a non-project.
> If BUF is nil, the current buffer is used."
>     (with-current-buffer (or buf (current-buffer))
>       (when-let* ((p (project-current))
>                   (pn (project-name p)))
>         (unless (string= pn my:project--non-project-dir)
>           pn))))
> ...
> )
>
> On Thu, Oct 10, 2024 at 8:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
>> On 09/10/2024 19:10, Juri Linkov wrote:
>> > Or maybe better to cache the value of project-name on remove projects.
>>
>> Just the project->project-name mapping? Why not. I suppose there'd still
>> be a pause when switching projects, but it's not as bad.
>>
>> For general caching, from past threads it seems the most problematic
>> case is "no project". Because OT1H it's still costly it terms of remote
>> I/O. But on the other, this is exactly when the cache might get invalid
>> soon (because the user will initialize a Git repo, or create another
>> root marker, etc).
>>
>> I guess we should come back to this after bug#72300.
>>
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 4203 bytes --]

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Mi Michael:



On Thu, Oct 10, 2024 at 01:12:25PM +0200, Michael Albinus wrote:
>Michael Albinus <michael.albinus@gmx.de> writes:
>
>Hi Ergus,
>
>> This is unfortune. sit-for should not trigger a redisplay. Does the
>> appended patch in tramp.el helps?
>
>I've pushed the patch to master. It doesn't hurt, at least.
>
This seems to solve the severe part of the issue. Thanks!

>Best regards, Michael.

Best,
Ergus

bug#73718: 31.0.50; Severe performance issue with Tramp and project-mode-line-format

[Michael Albinus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Version: 31.1

Ergus <spacibba@aol.com> writes:

> Mi Michael:

Hi Ergus,

> This seems to solve the severe part of the issue. Thanks!

Thanks for the feedback. I'm closung the bug.

> Best,
> Ergus

Best regards, Michael.