BUG in Tramp: After sleep, saving file hangs

BUG in Tramp: After sleep, saving file hangs

[ken]

Here's the scenario:

Both the local and the remote machines are running linux.  The local
machine is a laptop.  The remote is headless and runs sshd.  I often use
tramp to work on files on the remote machine from the local machine.  So
I'll open a file on the remote machine with tramp, work on it for a
time, then some time later put the local machine into sleep mode.  Going
into sleep mode obviously causes all internet connections to be lost,
including all of those from the local machine to the remote machine.

So, as is almost always the case, if I have a terminal window open from
the local to the remote via ssh, it is lost when the local is put to
sleep, and upon waking I must rerun "ssh remote-machine" to get it
back.  That's to be expected (at least until ssh becomes a lot more
refined and sophisticated).

Similarly, when I have a remote file open in tramp/emacs, put the local
machine to sleep, bring it out of sleep, and then try to work on that
remote file some more, then there is the problem.  Specifically, I can
edit the buffer, but when I try to save the buffer, emacs hangs, for
minutes if I wait that long. 

I can always break that hang by doing "C-g".  But then the file will not
be saved.  Sometimes if I try "C-x C-s" again, it will work, and the
buffer will be saved, but not always.  Sometimes it will take third or
fourth attempt to save the remote buffer.  But sometimes even that many
repeated attempts all fail.  Yesterday, after multiple such attempts to
save, all not working, I identified the "ssh -e remote-machine" process
which was the child process to emacs and killed it -- after doing that,
I did "C-x C-s" in emacs and the remote file was successfully saved.

Sometimes with such a hang emacs will display Debugger output, most of
the time it doesn't.  Yesterday, prior to my killing the child ssh
process emacs displayed the following (byte-code removed for brevity,
sorry for any unintended word-wrapping):

Debugger entered--Lisp error: (args-out-of-range 60 198)
  tramp-process-actions(#<process *tramp/scp lunar*> ["scp" nil "lunar"
"/home/userd/dird/osten.html" nil] 198 ((tramp-login-prompt-regexp
tramp-action-login) (tramp-password-prompt-regexp tramp-action-password)
(tramp-wrong-passwd-regexp tramp-action-permission-denied)
(shell-prompt-pattern tramp-action-succeed) (tramp-shell-prompt-pattern
tramp-action-succeed) (tramp-yesno-prompt-regexp tramp-action-yesno)
(tramp-yn-prompt-regexp tramp-action-yn) (tramp-terminal-prompt-regexp
tramp-action-terminal) (tramp-process-alive-regexp
tramp-action-process-alive)) 60)
  byte-code("...." [level tramp-verbose save-match-data-internal
tramp-message-show-message fmt-string args match-data ((byte-code "...."
[save-match-data-internal set-match-data evaporate] 3)) 3 apply message
0 "" 1 2 "Warning: " "Tramp: " 4 processp buffer-name process-buffer
tramp-dissect-file-name vectorp tramp-debug-message format "(%d) # "
vec-or-proc default-directory] 6) ((error)) (byte-code "...." [vec pr tm
subrp make-progress-reporter functionp zerop tramp-file-name-user format
"Opening connection for %s using %s" tramp-file-name-host
tramp-file-name-method "Opening connection for %s@%s using %s"
run-at-time 3 0.1 tramp-progress-reporter-update] 8) ((error))
((byte-code "...." [tm vec args fmt-string level vec-or-proc subrp
cancel-timer functionp 3 "%s...done" zerop tramp-file-name-user format
"Opening connection for %s using %s" tramp-file-name-host
tramp-file-name-method "Opening connection for %s@%s using %s" nil
(byte-code "...." [level tramp-verbose save-match-data-internal
tramp-message-show-message fmt-string args match-data ((byte-code "...."
[save-match-data-internal set-match-data evaporate] 3)) 3 apply message
0 "" 1 2 "Warning: " "Tramp: " 4 processp buffer-name process-buffer
tramp-dissect-file-name vectorp tramp-debug-message format "(%d) # "
vec-or-proc default-directory] 6) ((error))] 9)) delete-process setenv
"TERM" "LC_ALL" "C" "PROMPT_COMMAND" "PS1" tramp-compute-multi-hops
temporary-file-directory eval standard-value fboundp temp-directory
subrp functionp getenv "TEMP" file-directory-p file-name-as-directory
"TMP" "TMPDIR" file-exists-p ...] 10)
  byte-code("...." [vec process-environment pos process-name p
tramp-current-connection tramp-get-connection-process
tramp-get-connection-property "process-name" nil copy-sequence
tramp-get-connection-buffer processp process-status (run open) butlast
append tramp-time-diff current-time 5 throw suppress (byte-code "...."
[p vec args fmt-string signal vec-or-proc tramp-time-diff current-time
tramp-get-connection-property "last-cmd-time" (0 0 0) 60 processp
process-status (run open) tramp-send-command "echo are you awake" t (run
open) tramp-wait-for-output 10 file-error "Awake did fail" nil 1 "%s"
error-message-string error-message apply format (byte-code "...." [level
tramp-verbose save-match-data-internal tramp-message-show-message
fmt-string args match-data ((byte-code "...." [save-match-data-internal
set-match-data evaporate] 3)) 3 apply message 0 "" 1 2 "Warning: "
"Tramp: " 4 processp buffer-name process-buffer tramp-dissect-file-name
vectorp tramp-debug-message format "(%d) # " vec-or-proc
default-directory] 6) ((error)) tramp-message-show-message level] 10)
((file-error (byte-code "...." [vec p tramp-cleanup nil] 3))) err
(byte-code "...." [p vec tm pr args fmt-string processp process-status
(run open) tramp-get-buffer boundp non-essential throw nil 3 "%s..."
zerop tramp-file-name-user format "Opening connection for %s using %s"
tramp-file-name-host tramp-file-name-method "Opening connection for
%s@%s using %s" (byte-code "...." [level tramp-verbose
save-match-data-internal tramp-message-show-message fmt-string args
match-data ((byte-code "...." [save-match-data-internal set-match-data
evaporate] 3)) 3 apply message 0 "" 1 2 "Warning: " "Tramp: " 4 processp
buffer-name process-buffer tramp-dissect-file-name vectorp
tramp-debug-message format "(%d) # " vec-or-proc default-directory] 6)
((error)) (byte-code "...." [vec pr tm subrp make-progress-reporter
functionp zerop tramp-file-name-user format "Opening connection for %s
using %s" tramp-file-name-host tramp-file-name-method "Opening
connection for %s@%s using %s" run-at-time 3 0.1
tramp-progress-reporter-update] 8) ((error)) ((byte-code "...." [tm vec
args fmt-string level vec-or-proc subrp cancel-timer functionp 3
"%s...done" zerop tramp-file-name-user format "Opening connection for %s
using %s" tramp-file-name-host tramp-file-name-method "Opening
connection for %s@%s using %s" nil (byte-code "...." [level
tramp-verbose save-match-data-internal tramp-message-show-message
fmt-string args match-data ... 3 apply message 0 "" 1 2 "Warning: "
"Tramp: " 4 processp buffer-name process-buffer tramp-dissect-file-name
vectorp tramp-debug-message format "(%d) # " vec-or-proc
default-directory] 6) ((error))] 9)) delete-process setenv "TERM"
"LC_ALL" "C" "PROMPT_COMMAND" "PS1" tramp-compute-multi-hops
temporary-file-directory eval standard-value fboundp temp-directory
subrp functionp getenv "TEMP" file-directory-p file-name-as-directory
"TMP" "TMPDIR" file-exists-p ...] 10) ((quit (byte-code
"\302\b!\210\303    @    A\"\207" [vec err tramp-cleanup signal] 3)))] 6)
  tramp-maybe-open-connection(["scp" nil "lunar"
"/home/userd/dird/osten.html" nil])
  tramp-send-command(["scp" nil "lunar" "/home/userd/dird/osten.html"
nil] "( (test -e /home/userd/dird/osten.html || test -h
/home/userd/dird/osten.html) && \\stat -c '((\"%N\") %h %ue0 %ge0 %Xe0
%Ye0 %Ze0 %se0 \"%A\" t %ie0 -1)' /home/userd/dird/osten.html || echo
nil) 2>/dev/null; echo tramp_exit_status $?")
  tramp-send-command-and-check(["scp" nil "lunar"
"/home/userd/dird/osten.html" nil] "( (test -e
/home/userd/dird/osten.html || test -h /home/userd/dird/osten.html) &&
\\stat -c '((\"%N\") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 \"%A\" t %ie0 -1)'
/home/userd/dird/osten.html || echo nil)")
  tramp-barf-unless-okay(["scp" nil "lunar"
"/home/userd/dird/osten.html" nil] "( (test -e
/home/userd/dird/osten.html || test -h /home/userd/dird/osten.html) &&
\\stat -c '((\"%N\") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 \"%A\" t %ie0 -1)'
/home/userd/dird/osten.html || echo nil)" "`%s' returns with error" "(
(test -e /home/userd/dird/osten.html || test -h
/home/userd/dird/osten.html) && \\stat -c '((\"%N\") %h %ue0 %ge0 %Xe0
%Ye0 %Ze0 %se0 \"%A\" t %ie0 -1)' /home/userd/dird/osten.html || echo nil)")
  tramp-send-command-and-read(["scp" nil "lunar"
"/home/userd/dird/osten.html" nil] "( (test -e
/home/userd/dird/osten.html || test -h /home/userd/dird/osten.html) &&
\\stat -c '((\"%N\") %h %ue0 %ge0 %Xe0 %Ye0 %Ze0 %se0 \"%A\" t %ie0 -1)'
/home/userd/dird/osten.html || echo nil)")
  tramp-do-file-attributes-with-stat(["scp" nil "lunar"
"/home/userd/dird/osten.html" nil] "/home/userd/dird/osten.html" integer)
  tramp-sh-handle-file-attributes("/scp:lunar:/home/userd/dird/osten.html")
  apply(tramp-sh-handle-file-attributes
"/scp:lunar:/home/userd/dird/osten.html")
  tramp-sh-file-name-handler(file-attributes
"/scp:lunar:/home/userd/dird/osten.html")
  apply(tramp-sh-file-name-handler file-attributes
"/scp:lunar:/home/userd/dird/osten.html")
  byte-code("...." [suppress (apply foreign operation args)] 2)
  #[nil "...." [foreign result sf file-name-handler-alist d
default-directory nil autoload boundp temporary-file-directory eval
standard-value fboundp temp-directory subrp functionp getenv "TEMP"
file-directory-p file-name-as-directory "TMP" "TMPDIR" file-exists-p
"c:/temp" message "Neither `temporary-file-directory' nor
`temp-directory' is defined -- using /tmp." "/tmp" load noerror
nomessage non-essential (byte-code "...." [suppress (apply foreign
operation args)] 2) 5 "Non-essential received in operation %s" append
(byte-code "...." [level tramp-verbose save-match-data-internal
tramp-message-show-message fmt-string args match-data ((byte-code "...."
[save-match-data-internal set-match-data evaporate] 3)) 3 apply message
0 "" 1 2 "Warning: " "Tramp: " 4 processp buffer-name process-buffer
tramp-dissect-file-name vectorp tramp-debug-message format "(%d) # "
vec-or-proc default-directory] 6) ((error)) tramp-run-real-handler
suppress 1 "Suppress received in operation %s" tramp-cleanup x v
operation args fmt-string level vec-or-proc tramp-message-show-message] 7]()
  tramp-file-name-handler(file-attributes
"/scp:lunar:/home/userd/dird/osten.html")
  file-attributes("/scp:lunar:/home/userd/dird/osten.html")
  tramp-sh-handle-verify-visited-file-modtime(#<buffer osten.html>)
  apply(tramp-sh-handle-verify-visited-file-modtime #<buffer osten.html>)
  tramp-sh-file-name-handler(verify-visited-file-modtime #<buffer
osten.html>)
  apply(tramp-sh-file-name-handler verify-visited-file-modtime #<buffer
osten.html>)
  byte-code("...." [suppress (apply foreign operation args)] 2)
  #[nil "...." [foreign result sf file-name-handler-alist d
default-directory nil autoload boundp temporary-file-directory eval
standard-value fboundp temp-directory subrp functionp getenv "TEMP"
file-directory-p file-name-as-directory "TMP" "TMPDIR" file-exists-p
"c:/temp" message "Neither `temporary-file-directory' nor
`temp-directory' is defined -- using /tmp." "/tmp" load noerror
nomessage non-essential (byte-code "...." [suppress (apply foreign
operation args)] 2) 5 "Non-essential received in operation %s" append
(byte-code "...." [level tramp-verbose save-match-data-internal
tramp-message-show-message fmt-string args match-data ((byte-code "...."
[save-match-data-internal set-match-data evaporate] 3)) 3 apply message
0 "" 1 2 "Warning: " "Tramp: " 4 processp buffer-name process-buffer
tramp-dissect-file-name vectorp tramp-debug-message format "(%d) # "
vec-or-proc default-directory] 6) ((error)) tramp-run-real-handler
suppress 1 "Suppress received in operation %s" tramp-cleanup x v
operation args fmt-string level vec-or-proc tramp-message-show-message] 7]()
  tramp-file-name-handler(verify-visited-file-modtime #<buffer osten.html>)
  verify-visited-file-modtime(#<buffer osten.html>)
  basic-save-buffer()
  save-buffer(1)
  call-interactively(save-buffer nil nil)



--
To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org
To contact the owner, e-mail: opensuse+owner@opensuse.org

Re: BUG in Tramp: After sleep, saving file hangs

[Michael Albinus]

ken <gebser@mousecar.com> writes:

Hi Ken,

just for the records: which Emacs / Tramp version are you running?

And honestly, I believe this problem will be handled better if you
submit it as Emacs bug.

Best regards, Michael.

Re: BUG in Tramp: After sleep, saving file hangs

[Brett Gilio]

Michael Albinus writes:

> And honestly, I believe this problem will be handled better if 
> you
> submit it as Emacs bug.


Not to belabor the point here, because I do agree with Michael; 
but it
would be EXTRA good to put those logs in a pastebin-like location 
or
export it to a file and attach it.

Maybe I am speaking alone, here.

-- 
Brett M. Gilio
Free Software Foundation, Member
https://parabola.nu | https://emacs.org

Re: BUG in Tramp: After sleep, saving file hangs

[ken]

On 07/17/2018 10:47 PM, Brett Gilio wrote:
>
> Michael Albinus writes:
>
>> And honestly, I believe this problem will be handled better if you
>> submit it as Emacs bug.
>
>
> Not to belabor the point here, because I do agree with Michael; but it
> would be EXTRA good to put those logs in a pastebin-like location or
> export it to a file and attach it.
>
> Maybe I am speaking alone, here.
>

It doesn't matter if just one person says it if it's a good idea.

https://pastebin.com/xdSphZLd

Re: BUG in Tramp: After sleep, saving file hangs

[ken]

On 07/17/2018 10:36 AM, Michael Albinus wrote:
> ken <gebser@mousecar.com> writes:
>
> Hi Ken,
>
> just for the records: which Emacs / Tramp version are you running?

Sorry, a couple details which slipped by me with thoughts going to a lot
of other places.

emacs: 24.3.1

tramp: 2.2.624.3


>
> And honestly, I believe this problem will be handled better if you
> submit it as Emacs bug.

True, probably.  I thought I'd post it here first, see if someone could
come up with a quick response.  And I've never had good experience with
submitted bug reports: they seem always to be ignored or WONTFIXed or
combined with some other bug(s) and then somehow lost and never fixed,
or, at best, fixed only years later.  ... then if it's fixed, it's fixed
not in the version(s) everyone's using, but only in the absolutely
latest version, one which doesn't work down it into the distros for
another year or more.  So you might say I'm pessimistic about bug reports.



>
> Best regards, Michael.

Re: BUG in Tramp: After sleep, saving file hangs

[Michael Albinus]

ken <gebser@mousecar.com> writes:

Hi Ken,

> emacs: 24.3.1
>
> tramp: 2.2.624.3
>
>> And honestly, I believe this problem will be handled better if you
>> submit it as Emacs bug.
>
> True, probably.  I thought I'd post it here first, see if someone could
> come up with a quick response.

The Tramp manual gives a recommendation for this case:

--8<---------------cut here---------------start------------->8---
  • TRAMP does not recognize if a ‘ssh’ session hangs

     ‘ssh’ sessions on the local host hang when the network is down.
     TRAMP cannot safely detect such hangs.  The network configuration
     for ‘ssh’ can be configured to kill such hangs with the following
     command in the ‘~/.ssh/config’:

          Host *
               ServerAliveInterval 5
--8<---------------cut here---------------end--------------->8---

Does this work for you?

> submitted bug reports: they seem always to be ignored or WONTFIXed or
> combined with some other bug(s) and then somehow lost and never fixed,

Such things happen, yes. But my experience is, that Emacs maintainers
try to be responsive, and ignored or WONTFIXed bugs are the
minority. You can always protest if such things happen; often it is just
an oversight, or misunderstanding.

In particular, did you experience such behaviour with Tramp bugs?

> or, at best, fixed only years later.  ... then if it's fixed, it's fixed
> not in the version(s) everyone's using, but only in the absolutely
> latest version, one which doesn't work down it into the distros for
> another year or more.  So you might say I'm pessimistic about bug reports.

Well, to be honest: If it would need a fix, I doubt that somebody will
fix it for Emacs 24 / Tramp 2.2. The best you could expect is a patch
you could apply yourself.

There are 60 Tramp releases downloadable from ftp.gnu.org, and the
oldest one there is Tramp 2.0.39. Not to count intermediate releases
like Tramp 2.2.6-24.3 you are using. Tramp is almost a one-man-show; do
you believe I'm able to maintain every single release?

Support for Emacs 23 has been stopped some months ago; but the bleeding
edge Tramp 2.4.0 still supports Emacs 24. If I would fix something
there, you would be able to use it, w/o changing your Emacs version.

Best regards, Michael.

Re: BUG in Tramp: After sleep, saving file hangs

[ken]

On 07/19/2018 05:01 AM, Michael Albinus wrote:
> --8<---------------cut here---------------start------------->8---
>   • TRAMP does not recognize if a ‘ssh’ session hangs
>
>      ‘ssh’ sessions on the local host hang when the network is down.
>      TRAMP cannot safely detect such hangs.  The network configuration
>      for ‘ssh’ can be configured to kill such hangs with the following
>      command in the ‘~/.ssh/config’:
>
>           Host *
>                ServerAliveInterval 5
> --8<---------------cut here---------------end--------------->8---
>
> Does this work for you?

Michael, thanks for your kind and helpful reply.

The first time after I put that line in my config file, it worked
almost, but not completely.  It hung at:

Opening connection for lunar using scp...done

After about a half minute I did C-g, then did C-x C-s a second time and
it worked.  That's a vast improvement over most previous behavior. 

The second time (again, after coming out of sleep mode) going into an
already-opened buffer, making a change and then trying to save to disk
(on the remote machine), it worked flawlessly.

The third time (doing exactly the same as I did the first and second
times), it again worked flawlessly.

So I'm optimistic.  Thanks again.

Re: BUG in Tramp: After sleep, saving file hangs

[Michael Albinus]

ken <gebser@mousecar.com> writes:

Hi Ken,

> The first time after I put that line in my config file, it worked
> almost, but not completely.  It hung at:
>
> Opening connection for lunar using scp...done
>
> After about a half minute I did C-g, then did C-x C-s a second time and
> it worked.  That's a vast improvement over most previous behavior. 

There are several timeouts in Tramp for 60 seconds. Maybe you wait that
time, in order to see what Tramp tells?

> The second time (again, after coming out of sleep mode) going into an
> already-opened buffer, making a change and then trying to save to disk
> (on the remote machine), it worked flawlessly.
>
> The third time (doing exactly the same as I did the first and second
> times), it again worked flawlessly.
>
> So I'm optimistic.  Thanks again.

We could try to catch this. Start a new Emacs with the -Q option, and
set tramp-verbose to 10. Try to provoke the error. If it happens, send
the whole resulting Tramp debug buffer, and tell exactly the steps you
have applied in order to provoke the bug.

If you prefer, you could send this off-list; the debug buffer could be
huge. Maybe we find something there.

Best regards, Michael.

Bugs in Tramp in general [was: Re: BUG in Tramp: After sleep, saving file hangs]

[ken]

On 07/19/2018 05:01 AM, Michael Albinus wrote:
> In particular, did you experience such behaviour with Tramp bugs?

That question is not possible to answer, as this is the first bug (?)
I've ever encountered in Tramp... and I've been using it a long time. 
As I'm sure you know, there was ange-ftp before tramp and I was using
that back in the early-mid 1990s and I never had a problem with that
either.  I can recall only once having a difficulty (and I'm not sure
whether it was ange-ftp or tramp) and that turned out to be a change in
syntax used when opening a remote file.  That was a very long time ago,
so I might not be recalling correctly, but IIRC the (then) new syntax
required an initial slash (/) prepending the host:path/file
specification. But not at all a bug.  Tramp has been excellent software
and for a long time.  This was my first-ever issue with it.

Thanks again for your response.