bug#73046: 29.4; Emacs 100% CPU usage for several seconds when opening dired buffer over TRAMP

[Suhail Singh <suhailsingh247@gmail.com>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Suhail Singh" <suhailsingh247@gmail.com>
>> Date: Thu, 05 Sep 2024 10:24:54 -0400
>> 
>> 
>> I observe 100% CPU usage for several seconds when invoking dired for a
>> directory on one of the remote machines I connect to.  This doesn't seem
>> to happen for another host.  The connection to the affected host is
>> slower, and on that host the issue is worse when the directory has
>> several symlinks.
>> 
>> To reproduce via =emacs -Q=:
>> - Invoke `find-file' and connect to problematic host over ssh.
>> - Once connection is established, open dired pointing to a directory on
>>   the affected host.
>> - Observe bug: it takes _several_ seconds for the dired buffer to show.
>>   During that time, Emacs CPU usage is 100%.
>> - Once dired buffer is open, invoke `revert-buffer' and observe issue
>>   again.
>> 
>> Disabling global-font-lock-mode, makes the situation better.
>> 
>> Given that rudimentary font-locking should be possible via, simply, the
>> output of `ls --dired', it is unclear why enabling font-locking makes
>> things so much worse.  If there are some aspects of font-locking that
>> perform much worse over a slower connection, it would help if there were
>> a user configuration to disable them.
>> 
>> It would also help if being over a slow connection didn't result in
>> Emacs consuming 100% of the CPU via functions such as
>> `tramp-wait-for-regexp' (based on profiler-report).  Could some of this
>> be done asynchronously?
>
> You could probably tell which parts take the time by profiling Emacs
> while it collects the Dired data, using profiler.el.  This could give
> clues about the expensive parts.  My guess would be that retrieving
> the attributes of the files Dired needs are the reason, but I could be
> wrong.

Based on =profiler-report=, the following function "chains" consume most of the
CPU:
- `font-lock-fontify-keywords-region'
  - tramp-sh-file-name-handler
    - tramp-sh-handle-file-truename
      - `tramp-wait-for-regexp'
    - tramp-sh-handle-file-exists-p
      - `tramp-wait-for-regexp'
    - tramp-sh-handle-file-directory-p
      - `tramp-wait-for-regexp'
    - tramp-sh-handle-file-attributes
      - `tramp-wait-for-regexp'

As noted previously, disabling global-font-lock-mode helps.

In related news, one thing I've observed on the affected host is that
the version of `ls' doesn't seem to yield expected output for
`ls --dired'.  Specifically, the output of `ls --dired' is the same as
the output of `ls' (i.e., `--dired' gets treated as a no-op).  The version
of `ls' on this host is:

#+begin_quote
  ls (GNU coreutils) 8.32
  Copyright (C) 2020 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

  Written by Richard M. Stallman and David MacKenzie.
#+end_quote

-- 
Suhail