Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kim F. Storm]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sun, 23 May 2004 19:21:09 +0200
> 
> storm@cua.dk (Kim F. Storm) writes:
> 
> > I have experienced various problems with tramp itself while debugging
> > the GC problems -- they certainly need to be fixed before release.
> 
> Could you, please, be a little bit more verbose? I guess you mean
> something else but the autorevert problem.

My observations were somehow related to autorevert, specifically I
found tramp looping around calls to accept-process-output (I suppose
tramp was waiting for output that didn't arrive).

You may say that such problems are caused by auto-revert, but I think
that tramp should never wait indefinitely for output that obviously
will never come.  Since tramp calls accept-process-output in a loop,
relying on the timeout on accept-process-output alone is not enough.

++kfs

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kai Grossjohann]

storm@cua.dk (Kim F. Storm) writes:

> You may say that such problems are caused by auto-revert, but I think
> that tramp should never wait indefinitely for output that obviously
> will never come.  Since tramp calls accept-process-output in a loop,
> relying on the timeout on accept-process-output alone is not enough.

Tramp can't know whether the output will come or not.  Tramp enters
the infloop only in cases where it expects output from the remote
system.

This is to say, if the output from the remote system doesn't arrive,
then something is wrong anyway, and Tramp will be unable to operate
properly.

The auto-revert problems are, of course, caused by my failure to
anticipate that Tramp might be called in a reentrant fashion.  This
surely needs to be fixed.  But just removing the infloop is not going
to help.  Instead, Tramp needs to be made aware of reentrant calls in
one fashion or another.

Kai

PS: Is "reentrant" the right term here?

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Michael Albinus]

Kai Grossjohann <kai@emptydomain.de> writes:

> storm@cua.dk (Kim F. Storm) writes:
>
>> You may say that such problems are caused by auto-revert, but I think
>> that tramp should never wait indefinitely for output that obviously
>> will never come.  Since tramp calls accept-process-output in a loop,
>> relying on the timeout on accept-process-output alone is not enough.
>
> Tramp can't know whether the output will come or not.  Tramp enters
> the infloop only in cases where it expects output from the remote
> system.
>
> This is to say, if the output from the remote system doesn't arrive,
> then something is wrong anyway, and Tramp will be unable to operate
> properly.

I agree with Kai.

Some days ago, Tramp got a patch checking for a dead process during
accept-process-output in order to react faster (in Tramp CVS, not
synced yet with Emacs CVS).

Despite of this and the outstanding solution for the auto-revert
problem: what else should be done? Tramp must wait for the output,
otherwise it cannot decide how to continue. And it doesn't know that
the output "obviously will never come".

Best regards, Michael.

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Luc Teirlinck]

What if we, for the time being, completely disable auto-reverting of
remote files?  Currently it apparently _even_ gives problems for
people with fast connections.  Once Tramp can better deal with
reentrant (or recursive, whatever one wants to call it) calls, we can
introduce a user option to dis/en-able auto-reverting of remote files.

Sincerely,

Luc.

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kim F. Storm]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:

> What if we, for the time being, completely disable auto-reverting of
> remote files?  Currently it apparently _even_ gives problems for
> people with fast connections.  Once Tramp can better deal with
> reentrant (or recursive, whatever one wants to call it) calls, we can
> introduce a user option to dis/en-able auto-reverting of remote files.

Yes, that's the "trivial fix" we should use for 21.4.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Richard Stallman]

    What if we, for the time being, completely disable auto-reverting of
    remote files?

Would someone please do this without further delay?

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> What if we, for the time being, completely disable auto-reverting of
>> remote files?  Currently it apparently _even_ gives problems for
>> people with fast connections.  Once Tramp can better deal with
>> reentrant (or recursive, whatever one wants to call it) calls, we can
>> introduce a user option to dis/en-able auto-reverting of remote files.

> Yes, that's the "trivial fix" we should use for 21.4.

Indeed, it's probably a good thing to do independently from
any other problem.  But I find it important that we fix the bugs
revealed by such a setup before we change the default.


        Stefan

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> What if we, for the time being, completely disable auto-reverting of
> >> remote files?  Currently it apparently _even_ gives problems for
> >> people with fast connections.  Once Tramp can better deal with
> >> reentrant (or recursive, whatever one wants to call it) calls, we can
> >> introduce a user option to dis/en-able auto-reverting of remote files.
> 
> > Yes, that's the "trivial fix" we should use for 21.4.
> 
> Indeed, it's probably a good thing to do independently from
> any other problem.  But I find it important that we fix the bugs
> revealed by such a setup before we change the default.

I think we have a pretty good understanding of what the problems are
in this case: GC problems have been fixed, so it remains to fix that
tramp is not reentrant.

So it is "safe" to change the default (and it has already been done).

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kai Grossjohann]

storm@cua.dk (Kim F. Storm) writes:

> I think we have a pretty good understanding of what the problems are
> in this case: GC problems have been fixed, so it remains to fix that
> tramp is not reentrant.

I would like to discuss the solution with Michael, but I can think of
the following solutions:

(a) Make Tramp barf on reentrant calls.

(b) Make Tramp wait on reentrant calls (by implementing a queue or
    somesuch).

(c) Implement concurrency for reentrant calls (by opening more than
    one connection to the remote end).

Are you okay with all of these, or is one of them a no-no?

Kai

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kim F. Storm]

Kai Grossjohann <kai@emptydomain.de> writes:

> storm@cua.dk (Kim F. Storm) writes:
> 
> > I think we have a pretty good understanding of what the problems are
> > in this case: GC problems have been fixed, so it remains to fix that
> > tramp is not reentrant.
> 
> I would like to discuss the solution with Michael, but I can think of
> the following solutions:
> 
> (a) Make Tramp barf on reentrant calls.

That would probably break some code which does eg. file-exists-p
in a timer hook or some such.  So it is not an option.

> 
> (b) Make Tramp wait on reentrant calls (by implementing a queue or
>     somesuch).

I prefer this (because there are problems with the alternatives).

> 
> (c) Implement concurrency for reentrant calls (by opening more than
>     one connection to the remote end).

How do you solve the problem with having to enter passwords or other
stuff when invoked from a hook ...  and would you start a new thread
for every reentant call, that sounds like a lot of processes?

Also, tramp is slow at opening a connection (but maybe it can skip
some initial checks when it already has another connection to the
same host?).


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Stefan Monnier]

>> (a) Make Tramp barf on reentrant calls.

> That would probably break some code which does eg. file-exists-p
> in a timer hook or some such.  So it is not an option.

Such code currently breaks anyway, so it's still an improvement since it
makes things break more cleanly.


        Stefan

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kai Grossjohann]

storm@cua.dk (Kim F. Storm) writes:

> Kai Grossjohann <kai@emptydomain.de> writes:
>
>> storm@cua.dk (Kim F. Storm) writes:
>> 
>> > I think we have a pretty good understanding of what the problems are
>> > in this case: GC problems have been fixed, so it remains to fix that
>> > tramp is not reentrant.
>> 
>> I would like to discuss the solution with Michael, but I can think of
>> the following solutions:
>> 
>> (a) Make Tramp barf on reentrant calls.
>
> That would probably break some code which does eg. file-exists-p
> in a timer hook or some such.  So it is not an option.

I agree with Stefan: that code breaks now anyway.

>> (b) Make Tramp wait on reentrant calls (by implementing a queue or
>>     somesuch).
>
> I prefer this (because there are problems with the alternatives).

Michael wants to work on this, I think.  I'm not sure whether there
would be problems with this idea: it would tend to cause timers to
take longer to process (because they have to wait on the "main" Emacs
thread), and the "main" Emacs thread might have to wait longer on the
timers.  This effect could be benign or bad, I don't know.

>> (c) Implement concurrency for reentrant calls (by opening more than
>>     one connection to the remote end).
>
> How do you solve the problem with having to enter passwords or other
> stuff when invoked from a hook ... 

The idea is to cache them.  Multiple connections to the same host are
useful anyway, e.g. for async shell commands.  Also, Tramp could then
reopen the connection without pestering the user.

> and would you start a new thread for every reentant call, that
> sounds like a lot of processes?

The idea is to keep the connections open (at least for a while), and
to reuse them.

> Also, tramp is slow at opening a connection (but maybe it can skip
> some initial checks when it already has another connection to the
> same host?).

Remembering the results of the initial checks is also on the todo
list.

Kai

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Kim F. Storm]

Kai Grossjohann <kai@emptydomain.de> writes:

> > and would you start a new thread for every reentant call, that
> > sounds like a lot of processes?
> 
> The idea is to keep the connections open (at least for a while), and
> to reuse them.
> 
> > Also, tramp is slow at opening a connection (but maybe it can skip
> > some initial checks when it already has another connection to the
> > same host?).
> 
> Remembering the results of the initial checks is also on the todo
> list.

Both of these would be really good enhancements; if they are done, the
third option would be preferable...

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: [gmane.emacs.devel] Tramp with global-auto-revert-mode.

[Richard Stallman]

    (a) Make Tramp barf on reentrant calls.

    (b) Make Tramp wait on reentrant calls (by implementing a queue or
	somesuch).

    (c) Implement concurrency for reentrant calls (by opening more than
	one connection to the remote end).

    Are you okay with all of these, or is one of them a no-no?

I have no objection to any of them, but I don't see how (b)
could make sense.  Since Emacs is not multithreaded, a reentrant
invocation is a recursive invocation.  To wait would mean waiting
forever.