file-remote-p being called too often?

file-remote-p being called too often?

[T.V Raman]

this is with emacs built from Git@HEAD:
Discovered while profiling code in *scratch* via profiler-report:

It looks like file-remote-p gets called from a timer (not started by e)
and that that function shows up as being one of the ones eating cycles.


-- 

Re: file-remote-p being called too often?

[Michael Albinus]

"T.V Raman" <raman@google.com> writes:

Hi,

> this is with emacs built from Git@HEAD:
> Discovered while profiling code in *scratch* via profiler-report:
>
> It looks like file-remote-p gets called from a timer (not started by e)
> and that that function shows up as being one of the ones eating cycles.

Do you know which timer this is?

Alternatively, do you have a simple recipe, starting with 'emacs -Q',
which shows the problem?

Best regards, Michael.

Re: file-remote-p being called too often?

[Eli Zaretskii]

> From: "T.V Raman" <raman@google.com>
> Date: Tue, 16 Jan 2024 13:17:21 -0800
> 
> this is with emacs built from Git@HEAD:
> Discovered while profiling code in *scratch* via profiler-report:
> 
> It looks like file-remote-p gets called from a timer (not started by e)
> and that that function shows up as being one of the ones eating cycles.

Thanks.

I cannot reproduce this, and you didn't post the actual profile you
see.  In my naïve attempt to reproduce I didn't see file-remote-p in
the profile I collected.

Is this in "emacs -Q"?  If so, please show the recipe for reproducing
this, starting from "emacs -Q".  Also, what version of Emacs is that?

If this is not in "emacs -Q", perhaps the profile you will post will
give a hint as to what optional features cause this.

(And why is this posted here instead of the bug tracker?)

Re: file-remote-p being called too often?

[T.V Raman]

Appeared to be from idle-timer. Haven't been able to repro -- but will
keep an eye out for it. Running with emacs -q ie without emacspeak is
impossible for me; but I do have a lot of packages loaded, and  can try
when I have time to run with just emacspeak loaded. I posted the note
here since there has been ongoing work on the tramp bits in
emacs@master, so posted in the hope of  the tramp folks catching the
issue if real earlier rather than later.

-- 

Re: file-remote-p being called too often?

[Michael Albinus]

"T.V Raman" <raman@google.com> writes:

Hi,

> Appeared to be from idle-timer. Haven't been able to repro -- but will
> keep an eye out for it. Running with emacs -q ie without emacspeak is
> impossible for me; but I do have a lot of packages loaded, and  can try
> when I have time to run with just emacspeak loaded. I posted the note
> here since there has been ongoing work on the tramp bits in
> emacs@master, so posted in the hope of  the tramp folks catching the
> issue if real earlier rather than later.

Thanks. I fear it is rather a problem in calling file-remote-p in a
package, so Tramp (folks) cannot do too much, until we know which
package it is.

Best regards, Michael.

Re: file-remote-p being called too often?

[T.V Raman]

A quick grep in my installed packages shows it being called by eglot and
magit for starters --one more question: does that function memoize its
result, ie it should be called only once  for real in any buffer; since
a file once local will never become remote.

-- 

Re: file-remote-p being called too often?

[David Ponce]

> this is with emacs built from Git@HEAD:
> Discovered while profiling code in *scratch* via profiler-report:
> 
> It looks like file-remote-p gets called from a timer (not started by e)
> and that that function shows up as being one of the ones eating cycles.

I wonder if the problem could be due to recentf if the option
`recentf-auto-cleanup' is enabled to cleanup the recentf list each
time Emacs has been idle during N seconds?

Because in this case, the function `recentf-cleanup' is called every N
seconds, in turn calling `recentf-keep-p' to checks if a file name
should be kept in the recentf list based on the `recentf-keep' option,
whose default value is the predicate `recentf-keep-default-predicate'
that uses `file-remote-p':

(defun recentf-keep-default-predicate (file)
   "Return non-nil if FILE should be kept in the recent list.
It handles the case of remote files as well."
   (cond
    ((file-remote-p file nil t) (recentf-access-file file))
    ((file-remote-p file))
    ((file-readable-p file))))

Regards

Re: file-remote-p being called too often?

[T.V Raman]

good catch, will chase that down.

David Ponce writes:
 > > this is with emacs built from Git@HEAD:
 > > Discovered while profiling code in *scratch* via profiler-report:
 > > 
 > > It looks like file-remote-p gets called from a timer (not started by e)
 > > and that that function shows up as being one of the ones eating cycles.
 > 
 > I wonder if the problem could be due to recentf if the option
 > `recentf-auto-cleanup' is enabled to cleanup the recentf list each
 > time Emacs has been idle during N seconds?
 > 
 > Because in this case, the function `recentf-cleanup' is called every N
 > seconds, in turn calling `recentf-keep-p' to checks if a file name
 > should be kept in the recentf list based on the `recentf-keep' option,
 > whose default value is the predicate `recentf-keep-default-predicate'
 > that uses `file-remote-p':
 > 
 > (defun recentf-keep-default-predicate (file)
 >    "Return non-nil if FILE should be kept in the recent list.
 > It handles the case of remote files as well."
 >    (cond
 >     ((file-remote-p file nil t) (recentf-access-file file))
 >     ((file-remote-p file))
 >     ((file-readable-p file))))
 > 
 > Regards

-- 

Re: file-remote-p being called too often?

[T.V Raman]

I checked, I dont have recentf-mode enabled, but I went ahead and set
auto-cleanup to never.

David Ponce writes:
 > > this is with emacs built from Git@HEAD:
 > > Discovered while profiling code in *scratch* via profiler-report:
 > > 
 > > It looks like file-remote-p gets called from a timer (not started by e)
 > > and that that function shows up as being one of the ones eating cycles.
 > 
 > I wonder if the problem could be due to recentf if the option
 > `recentf-auto-cleanup' is enabled to cleanup the recentf list each
 > time Emacs has been idle during N seconds?
 > 
 > Because in this case, the function `recentf-cleanup' is called every N
 > seconds, in turn calling `recentf-keep-p' to checks if a file name
 > should be kept in the recentf list based on the `recentf-keep' option,
 > whose default value is the predicate `recentf-keep-default-predicate'
 > that uses `file-remote-p':
 > 
 > (defun recentf-keep-default-predicate (file)
 >    "Return non-nil if FILE should be kept in the recent list.
 > It handles the case of remote files as well."
 >    (cond
 >     ((file-remote-p file nil t) (recentf-access-file file))
 >     ((file-remote-p file))
 >     ((file-readable-p file))))
 > 
 > Regards

-- 

Re: file-remote-p being called too often?

[T.V Raman]

As one more means to chasing down the culprit:

mM-x debug-on-entry
file-remote-p

Above gets invoked when emacspeak plays a local audio file as an earcon
using a shell command; there should be no reason to run file-remote-p in
this case.

Debugger entered--entering a function:
* (file-remote-p "/home/raman/emacs/lisp/emacspeak/sounds/" localname)
  (file-local-name "/home/raman/emacs/lisp/emacspeak/sounds/")
  (executable-find "pactl")
  (emacspeak-sounds-get-file process-active)
  (emacspeak-play-auditory-icon process-active)
  (emacspeak-auditory-icon process-active)
  (emacspeak-speak-mode-line nil)
  (#<subr funcall-interactively> emacspeak-speak-mode-line nil)
  (apply #<subr funcall-interactively> emacspeak-speak-mode-line nil)
  (ad-Advice-funcall-interactively #<subr funcall-interactively> emacspeak-speak-mode-line nil)
  (apply ad-Advice-funcall-interactively #<subr funcall-interactively> (emacspeak-speak-mode-line nil))
  (funcall-interactively emacspeak-speak-mode-line nil)
  (#<subr call-interactively> emacspeak-speak-mode-line nil nil)
  (apply #<subr call-interactively> emacspeak-speak-mode-line (nil nil))
  (call-interactively@ido-cr+-record-current-command #<subr call-interactively> emacspeak-speak-mode-line nil nil)
  (apply call-interactively@ido-cr+-record-current-command #<subr call-interactively> (emacspeak-speak-mode-line nil nil))
  (call-interactively emacspeak-speak-mode-line nil nil)
  (command-execute emacspeak-speak-mode-line)

-- 

Re: file-remote-p being called too often?

[T.V Raman]

global-auto-revert-mode also appears to trigger file-remote-p -- that
would be one example where the call could be usefully memoized.

-- 

Re: file-remote-p being called too often?

[Eli Zaretskii]

> From: "T.V Raman" <raman@google.com>
> Cc: Emacs Devel <emacs-devel@gnu.org>,  Eli Zaretskii <eliz@gnu.org>
> Date: Wed, 17 Jan 2024 13:18:09 -0800
> 
> Above gets invoked when emacspeak plays a local audio file as an earcon
> using a shell command; there should be no reason to run file-remote-p in
> this case.
> 
> Debugger entered--entering a function:
> * (file-remote-p "/home/raman/emacs/lisp/emacspeak/sounds/" localname)
>   (file-local-name "/home/raman/emacs/lisp/emacspeak/sounds/")
>   (executable-find "pactl")
>   (emacspeak-sounds-get-file process-active)
>   (emacspeak-play-auditory-icon process-active)
>   (emacspeak-auditory-icon process-active)
>   (emacspeak-speak-mode-line nil)

Why does emacspeak repeatedly call executable-find, and from a
mode-line format on top of that?  Why not call executable-find just
once, and then store the result in some variable, to be reused on
subsequent calls from emacspeak-speak-mode-line?

In any case, for local file names, such as "pactl", the call to
file-remote-p should be very inexpensive.  Do you have a reason to
believe it is somehow expensive in your case?  Or did you just wonder
why file-remote-p is called that often, regardless of whether it's
expensive?  The reason for frequent calls seems to be that you have
this call in your mode-line-format, which is not a good design.  This
design means executable-find will be called every redisplay cycle,
time and time again, for the same program name, which is hardly a good
idea.

Re: file-remote-p being called too often?

[Eli Zaretskii]

> From: "T.V Raman" <raman@google.com>
> Cc: Emacs Devel <emacs-devel@gnu.org>,  Eli Zaretskii <eliz@gnu.org>
> Date: Wed, 17 Jan 2024 13:56:44 -0800
> 
> global-auto-revert-mode also appears to trigger file-remote-p -- that
> would be one example where the call could be usefully memoized.

Michael will tell, but I don't think this can be memoized, since file
handlers can change dynamically behind autorevert's back.

AFAIU, autorevert calls file-remote-p only when there are
notifications about changes in some visited files, so when Emacs is
idle it should not call file-remote-p due to global-auto-revert-mode.

Re: file-remote-p being called too often?

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi,

>> global-auto-revert-mode also appears to trigger file-remote-p -- that
>> would be one example where the call could be usefully memoized.
>
> Michael will tell, but I don't think this can be memoized, since file
> handlers can change dynamically behind autorevert's back.

Indeed. And even if it would be only Tramp's file name handler, this
cannot be memoized, because remote file name syntax can change on the
fly. See command `tramp-change-syntax'.

And I also ask the same question as Eli: How expensive is a single
`file-remote-p' call? In the Tramp case it is designed to avoid going
remote unless a remote connection is already established. And if the
file name in question does not match the remote file name syntax, it
returns nil almost immediately.

Best regards, Michael.

Re: file-remote-p being called too often?

[T.V Raman]

thanks for the executable-find tip, yes I shall update  to cache the
value



Eli Zaretskii writes:
 > > From: "T.V Raman" <raman@google.com>
 > > Cc: Emacs Devel <emacs-devel@gnu.org>,  Eli Zaretskii <eliz@gnu.org>
 > > Date: Wed, 17 Jan 2024 13:18:09 -0800
 > > 
 > > Above gets invoked when emacspeak plays a local audio file as an earcon
 > > using a shell command; there should be no reason to run file-remote-p in
 > > this case.
 > > 
 > > Debugger entered--entering a function:
 > > * (file-remote-p "/home/raman/emacs/lisp/emacspeak/sounds/" localname)
 > >   (file-local-name "/home/raman/emacs/lisp/emacspeak/sounds/")
 > >   (executable-find "pactl")
 > >   (emacspeak-sounds-get-file process-active)
 > >   (emacspeak-play-auditory-icon process-active)
 > >   (emacspeak-auditory-icon process-active)
 > >   (emacspeak-speak-mode-line nil)
 > 
 > Why does emacspeak repeatedly call executable-find, and from a
 > mode-line format on top of that?  Why not call executable-find just
 > once, and then store the result in some variable, to be reused on
 > subsequent calls from emacspeak-speak-mode-line?
 > 
 > In any case, for local file names, such as "pactl", the call to
 > file-remote-p should be very inexpensive.  Do you have a reason to
 > believe it is somehow expensive in your case?  Or did you just wonder
 > why file-remote-p is called that often, regardless of whether it's
 > expensive?  The reason for frequent calls seems to be that you have
 > this call in your mode-line-format, which is not a good design.  This
 > design means executable-find will be called every redisplay cycle,
 > time and time again, for the same program name, which is hardly a good
 > idea.

-- 

Re: file-remote-p being called too often?

[T.V Raman]

how expensive? dont know yet, I was following up on why it was showing
up as significant when profiling.


Michael Albinus writes:
 > Eli Zaretskii <eliz@gnu.org> writes:
 > 
 > Hi,
 > 
 > >> global-auto-revert-mode also appears to trigger file-remote-p -- that
 > >> would be one example where the call could be usefully memoized.
 > >
 > > Michael will tell, but I don't think this can be memoized, since file
 > > handlers can change dynamically behind autorevert's back.
 > 
 > Indeed. And even if it would be only Tramp's file name handler, this
 > cannot be memoized, because remote file name syntax can change on the
 > fly. See command `tramp-change-syntax'.
 > 
 > And I also ask the same question as Eli: How expensive is a single
 > `file-remote-p' call? In the Tramp case it is designed to avoid going
 > remote unless a remote connection is already established. And if the
 > file name in question does not match the remote file name syntax, it
 > returns nil almost immediately.
 > 
 > Best regards, Michael.

-- 

Re: file-remote-p being called too often?

[Eli Zaretskii]

> From: "T.V Raman" <raman@google.com>
> Date: Thu, 18 Jan 2024 06:44:48 -0800
> Cc: eliz@gnu.org,
>     raman@google.com,
>     da_vid@orange.fr,
>     emacs-devel@gnu.org
> 
> how expensive? dont know yet, I was following up on why it was showing
> up as significant when profiling.

Significant in terms of cpu percentage?  How many percents?

Re: file-remote-p being called too often?

[T.V Raman]

about 16% from memory but was sampled over a relatively short time
while i debugged something else


Eli Zaretskii writes:
 > > From: "T.V Raman" <raman@google.com>
 > > Date: Thu, 18 Jan 2024 06:44:48 -0800
 > > Cc: eliz@gnu.org,
 > >     raman@google.com,
 > >     da_vid@orange.fr,
 > >     emacs-devel@gnu.org
 > > 
 > > how expensive? dont know yet, I was following up on why it was showing
 > > up as significant when profiling.
 > 
 > Significant in terms of cpu percentage?  How many percents?

-- 

Re: file-remote-p being called too often?

[Eli Zaretskii]

> From: "T.V Raman" <raman@google.com>
> Date: Thu, 18 Jan 2024 07:57:41 -0800
> Cc: raman@google.com,
>     michael.albinus@gmx.de,
>     da_vid@orange.fr,
>     emacs-devel@gnu.org
> 
> about 16% from memory but was sampled over a relatively short time
> while i debugged something else

16% doesn't sound too bad.