bug#73046: 29.4; Emacs 100% CPU usage for several seconds when opening dired buffer over TRAMP

[Suhail Singh <suhailsingh247@gmail.com>]

Michael Albinus <michael.albinus@gmx.de> writes:

>>> > FWIW, I cannot reproduce this: I tried Dired on a remote host with
>>> > which I have connection that is quite slow, and saw neither high CPU
>>> > usage nor a significant delay in displaying a Dired buffer.
>>>
>>> It seems to be related to font-locking, indeed. See variable
>>> `dired-font-lock-keywords'. It specifies face recognition running basic
>>> file oprtations. For example, ";; Broken Symbolic link" calls
>>> `file-truename' and `file-exists-p', while "Symbolic link to a directory"
>>> and ";; Symbolic link to a non-directory" invoke `file-truename' and
>>> `file-directory-p'.
>>
>> But font-lock is ON by default, so what I saw also includes this,
>> right?
>
> Sure. But we don't know the exact setting of the OP. The connection
> could be extremely slow for him, Tramp's cache might be disabled,
> whatever. Perhaps it's worth to rerun the test with "emacs -Q".

The issue was observed with emacs -Q on a remote directory that had 22
symlinks.  On the affected host, I am able to reliably reproduce the
issue by evaluating the code below and then navigating to
/tmp/test/links in dired via TRAMP.

#+name: perf-fix/tramp/font-lock-on-dired/reproducer
#+begin_src sh
  cd /tmp
  rm -rf /tmp/test
  mkdir /tmp/test
  cd /tmp/test
  for i in `seq -w 0 21`; do
      mkdir -p src/"$i"
  done
  mkdir -p links
  cd links
  for i in `seq -w 0 21`; do
      ln -sf /tmp/test/src/"$i"
  done
#+end_src

Notably, for the affected host, the issue is reproducible using above
even after a new OS install.  However, the above test case doesn't cover
all variables affecting the issue.  Specifically, when I run the above
code on a different host (with faster connection and fewer hops to get
to) and navigate to /tmp/test/links the severity of the issue is much
less.  That being said, the profiler reports (starting from emacs -Q and
after establishing connection with host in question via TRAMP) for both
the affected and not-as-affected host are similar.  I believe the issue
is present on the unaffected host as well, but with much less severity
for some reason.

My guess is that connection speed (in addition to the number of symlinks
in the directory in question) is a determining factor.  I am speculating
here, but perhaps TRAMP is busy-waiting for a response?  That would be
consistent with what I observe, but I am not sure how to verify that.

>>> I believe it would be helpful to suppress these checks via a user
>>> option.
>>
>> What's wrong with "M-x font-lock-mode RET"?
>
> You would deactivate all fontifications, not only the ones related to
> symlinks.
>
> And you must do this in a hook of the dired buffer, before fontification
> happens.

Yes, precisely.  That is my current workaround.  It's far from ideal,
but it at least allows Emacs to be in a semi-workable state.  It would
be much better if there were a way to suppress only the "expensive"
font-locking checks in this case.

> There exist `dired-hide-details-mode', which I don't know. Perhaps
> enabling it in time would do the job?

It, unfortunately, doesn't help.

-- 
Suhail