auto-revert-mode and tramp file handlers

auto-revert-mode and tramp file handlers

[Luc Teirlinck]

>From one of my earlier messages:
   
   I do not know whether the following problem is connected to the update
   or not.

   Visit a file using the /ssh:USER@HOST:FILENAME syntax.  When the file
   is displayed, close your connection.  Obviously, now tramp is not
   going to be able to function normally anymore.  But what happens is
   that Emacs now appears to freeze.  It does not even respond to C-g
   anymore, and I had to kill it from the command line.  Is this really
   unavoidable?  Closing one's connection forgetting that one has active
   trap buffers _is_ sometimes going to happen, so the behavior is a
   nuisance.

The above only happens when auto-revert-mode is enabled.  (Again,
sorry for forgetting to double check using emacs -q).

A problem is that tramp seems to handle functions like file-exists-p
and file-readable-p by trying to connect for 60 seconds and then
throwing an error if the connection has been closed.  I believe that
both functions strictly speaking should return nil in the given
situation.  In the case of file-exists-p that is because it is
impossible to figure out the file's attributes.  From the Elisp
manual:

 -- Function: file-exists-p filename
This function returns `t' if a file named FILENAME appears to exist.
This does not mean you can necessarily read the file, only that
you can find out its attributes.

If there is no way to quickly check whether the connection has been
closed, and if so, return nil instead of throwing an error, then I
believe that maybe we should just disable auto-reverting for remote
files.  (This is trivial, using `file-remote-p'.)  I believe that at
the very least there should be an option to disable auto-reverting for
remote files, because it can be very costly for people with slow
connections.

Sincerely,

Luc.

Re: auto-revert-mode and tramp file handlers

[Kai Grossjohann]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> A problem is that tramp seems to handle functions like file-exists-p
> and file-readable-p by trying to connect for 60 seconds and then
> throwing an error if the connection has been closed.  I believe that
> both functions strictly speaking should return nil in the given
> situation.  In the case of file-exists-p that is because it is
> impossible to figure out the file's attributes.  From the Elisp
> manual:
>
>  -- Function: file-exists-p filename
> This function returns `t' if a file named FILENAME appears to exist.
> This does not mean you can necessarily read the file, only that
> you can find out its attributes.
>
> If there is no way to quickly check whether the connection has been
> closed, and if so, return nil instead of throwing an error, then I
> believe that maybe we should just disable auto-reverting for remote
> files.  (This is trivial, using `file-remote-p'.)  I believe that at
> the very least there should be an option to disable auto-reverting for
> remote files, because it can be very costly for people with slow
> connections.

I'm not sure how to change Tramp to make this better: it often happens
that a connection is closed merely because a timeout has expired.  In
that case, just silently reopening the connection appears to be the
right approach.  But I confess that I haven't thought a lot about your
problem.  So it's quite likely that there are good solutions that I'm
not seeing.

But I do realize that there is a problem here.  Perhaps it is best to
make it easy for the user to customize "no auto-revert on remote
files".

What do people think?

Kai

Re: auto-revert-mode and tramp file handlers

[Luc Teirlinck]

Kai Grossjohann wrote:

   I'm not sure how to change Tramp to make this better: it often happens
   that a connection is closed merely because a timeout has expired.  In
   that case, just silently reopening the connection appears to be the
   right approach.

In the described situation I am off line.  Do you mean Tramp is trying
get me back on line?  If so, there must be something wrong with my
setup, because that definitely never happens.  Anyway, getting on line
takes me on average 30-40 seconds, and often takes more than the
allotted 60 seconds.  So it is definitely _not_ something I would want
to have tried on an automated basis, even if I could correct my setup
to do so.

If tramp is not trying to get me back on line, then

(file-exists-p "/ssh:teirllm@raven.dms.auburn.edu:~/myfile")

takes more than a minute to get the same information that a simple
call to ssh gives me in less than one hundred of a second (see below),
and then throws an error in a case where the function's docstring
suggests a return value of nil.

[bash2.05b.0 ~ 3 6] time ssh raven.dms.auburn.edu
ssh: raven.dms.auburn.edu: Temporary failure in name resolution

real 0m0.009s
user 0m0.010s
sys  0m0.000s

I believe related problems have been discussed previously in:

http://mail.gnu.org/archive/html/emacs-devel/2003-09/msg00198.html

Sincerely,

Luc.

Re: auto-revert-mode and tramp file handlers

[Luc Teirlinck]

>From my earlier message:
   
   In the described situation I am off line.  Do you mean Tramp is trying
   get me back on line?  If so, there must be something wrong with my
   setup, because that definitely never happens.  Anyway, getting on line
   takes me on average 30-40 seconds, and often takes more than the
   allotted 60 seconds.  So it is definitely _not_ something I would want
   to have tried on an automated basis, even if I could correct my setup
   to do so.

I sometimes forget the totally obvious.  All of this assumes that my
modem is still plugged in.  Often it is not and then, obviously, Tramp
is not going to be able to get me back on line.  Even in this
situation, Tramp still tries to connect me for 60 seconds, whereas ssh
notices that this is futile in less than one hundred of a second.

Sincerely,

Luc.

Re: auto-revert-mode and tramp file handlers

[Luc Teirlinck]

Kai Grossjohann wrote:

   I'm not sure how to change Tramp to make this better: it often happens
   that a connection is closed merely because a timeout has expired.  In
   that case, just silently reopening the connection appears to be the
   right approach.  But I confess that I haven't thought a lot about your
   problem.  So it's quite likely that there are good solutions that I'm
   not seeing.

   But I do realize that there is a problem here.  Perhaps it is best to
   make it easy for the user to customize "no auto-revert on remote
   files".

   What do people think?

Not just auto-revert, but _very many_ places in the Emacs code call
file-exists-p and its many friends in situations where the file
usually is local but could be remote.  Usually, the purpose seems to
be to prevent a relatively time consuming operation and/or prevent an
error, in case the file does not exist (or is not readable or
whatever).  I do believe that in many of these places, the person who
put in those calls did not realize that that call could take more than
a full minute of real time, regardless of the CPU power of the
machine, and could then throw an error instead of returning nil if the
file "does not exist" (because of being unreachable).

One interpretation is that all such calls are bugs that one way or
the other should be fixed.  (Presumably by calling file-remote-p
before calling file-exists-p and friends, and doing something ad hoc
if the return value is t.)  Another interpretation is that a handler
for these functions should not take a full minute of real time and
should not throw an error when the docstring suggests a return value
of nil.

Is there no way that tramp could detect that the user is off line in a
fraction of a second (as ssh apparently can) and if so, return nil for
file-exists-p and other functions that expect a return value of nil in
this case?

Sincerely,

Luc.

Re: auto-revert-mode and tramp file handlers

[Kai Grossjohann]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> Is there no way that tramp could detect that the user is off line in a
> fraction of a second (as ssh apparently can) and if so, return nil for
> file-exists-p and other functions that expect a return value of nil in
> this case?

I see what you're talking about.  The problem is that Tramp invokes
ssh to connect the remote host, then waits 60 seconds for the shell
prompt.  Additionally, it could check whether the process is still
alive and stop waiting when the process dies.  I completely forgot to
put this in.

Can't promise to do this right away, but I'll try.

Thanks a lot for helping make Tramp better!

Kai

Re: auto-revert-mode and tramp file handlers

[Michael Albinus]

Kai Grossjohann <kai@emptydomain.de> writes:

> Luc Teirlinck <teirllm@dms.auburn.edu> writes:
>
>> Is there no way that tramp could detect that the user is off line in a
>> fraction of a second (as ssh apparently can) and if so, return nil for
>> file-exists-p and other functions that expect a return value of nil in
>> this case?
>
> I see what you're talking about.  The problem is that Tramp invokes
> ssh to connect the remote host, then waits 60 seconds for the shell
> prompt.  Additionally, it could check whether the process is still
> alive and stop waiting when the process dies.  I completely forgot to
> put this in.

The following patch might do the job:

magdalene:~/src> diff -u emacs/lisp/net/tramp.el tramp/lisp/tramp.el
--- emacs/lisp/net/tramp.el     2004-05-08 22:43:20.000000000 +0200
+++ tramp/lisp/tramp.el 2004-05-13 22:27:19.000000000 +0200
@@ -5025,6 +5025,8 @@
     (with-timeout (60 (throw 'tramp-action 'timeout))
       (while (not found)
        (accept-process-output p 1)
+       (unless (memq (process-status p) '(run open))
+         (throw 'tramp-action 'process-died))
        (goto-char (point-min))
        (setq todo actions)
        (while todo
@@ -5062,6 +5064,8 @@
     (with-timeout (60 (throw 'tramp-action 'timeout))
       (while (not found)
        (accept-process-output p 1)
+       (unless (memq (process-status p) '(run open))
+         (throw 'tramp-action 'process-died))
        (setq todo actions)
        (goto-char (point-min))
        (while todo
@@ -5466,12 +5470,16 @@
              (with-timeout (timeout)
                (while (not found)
                  (accept-process-output proc 1)
+                (unless (memq (process-status proc) '(run open))
+                  (error "Process has died"))
                  (goto-char (point-min))
                  (setq found (when (re-search-forward regexp nil t)
                                (tramp-match-string-list)))))))
           (t
            (while (not found)
              (accept-process-output proc 1)
+            (unless (memq (process-status proc) '(run open))
+              (error "Process has died"))
              (goto-char (point-min))
              (setq found (when (re-search-forward regexp nil t)
                            (tramp-match-string-list))))))
@@ -6079,12 +6087,16 @@
                (with-timeout (timeout)
                  (while (not found)
                    (accept-process-output proc 1)
+                  (unless (memq (process-status proc) '(run open))
+                    (error "Process has died"))
                    (goto-char (point-max))
                    (forward-line -1)
                    (setq found (looking-at end-of-output))))))
             (t
              (while (not found)
                (accept-process-output proc 1)
+              (unless (memq (process-status proc) '(run open))
+                (error "Process has died"))
                (goto-char (point-max))
                (forward-line -1)
                (setq found (looking-at end-of-output))))))

It's not "a fraction of a second", and it just returns with an error
in case a connection is broken, but it is better than waiting 60".
Silently returning for `file-exists-p' and friends is outstanding.

It is really a design question, whether Tramp shall report a broken
connection, or whether it should just return. Don't know what's
better.

Also available in Tramp CVS, together with the recent patch provided
by Kai.

Best regards, Michael.