bug#23883: 24.4.51; tramp sometimes reports "Args out of range" error when resuming editing a remote file after a period of inactivity

bug#23883: 24.4.51; tramp sometimes reports "Args out of range" error when resuming editing a remote file after a period of inactivity

[MICHAEL J MCCLENNEN]

To reproduce this bug:

1) Edit a remote file using tramp.
2) Wait for at least 24 hours.
3) Make a change to the file in the local buffer.  Emacs will then attempt to query
   the remote system using tramp in order to check whether it has changed on disk
   before allowing any local editing.
4) Sometimes (not always) this check aborts with the error message "Args out of range".
   The buffer is left in read-only mode, preventing you from working on the file.

I have actually figured out where in the code the error occurs, and have a fix for it.  The
error occurs at the last line of the function `tramp-process-actions' in tramp.el, in the
call to `delete-region'.

The following patch will fix the problem:
3593c3593
< 	  (let (buffer-read-only) (if (< pos (point)) (delete-region pos (point)))))))))
---
> 	  (let (buffer-read-only) (delete-region pos (point))))))))

I have tested this patch, and it solves the problem without causing any other 
problems that I can detect.  I would appreciate if you would apply this patch 
to the Emacs source.



In Aquamacs 3.2 GNU Emacs 24.4.51.2
(x86_64-apple-darwin14.0.0, NS apple-appkit-1343.14)
of 2014-11-07 (Aquamacs-3.2) on watson.local
Operating System: OS X Version 10.8.5 (Build 12F2560)
Configured using:
`configure --with-ns --without-x 'CFLAGS=-arch x86_64 -O3 -g
-mtune=corei7 -mmacosx-version-min=10.6' 'LDFLAGS=-arch x86_64 -O3 -g
-mtune=corei7 -mmacosx-version-min=10.6''

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  savehist-mode: t
  smart-frame-positioning-mode: t
  shell-dirtrack-mode: t
  aquamacs-autoface-mode: t
  recentf-mode: t
  osx-key-mode: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  show-paren-mode: t
  delete-selection-mode: t
  cua-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t



Memory information:
((conses 16 866516 175475)
(symbols 48 45081 0)
(miscs 40 2907 3571)
(strings 32 103591 20533)
(string-bytes 1 4546451)
(vectors 16 50097)
(vector-slots 8 1858179 143262)
(floats 8 1921 792)
(intervals 56 167401 177)
(buffers 960 779))

bug#23883: 24.4.51; tramp sometimes reports "Args out of range" error when resuming editing a remote file after a period of inactivity

[Michael Albinus]

MICHAEL J MCCLENNEN <mmcclenn@geology.wisc.edu> writes:

Hi,

> To reproduce this bug:
>
> 1) Edit a remote file using tramp.
> 2) Wait for at least 24 hours.
> 3) Make a change to the file in the local buffer.  Emacs will then attempt to query
>    the remote system using tramp in order to check whether it has changed on disk
>    before allowing any local editing.
> 4) Sometimes (not always) this check aborts with the error message "Args out of range".
>    The buffer is left in read-only mode, preventing you from working on the file.

Thanks for the report. I'll try to reproduce it with the recent Emacs
from git; I cannot promise I'll succeed.

Which remote file name have you used? I'm mainly interested in the
method name.

> I have actually figured out where in the code the error occurs, and have a fix for it.  The
> error occurs at the last line of the function `tramp-process-actions' in tramp.el, in the
> call to `delete-region'.
>
> The following patch will fix the problem:
> 3593c3593
> < 	  (let (buffer-read-only) (if (< pos (point)) (delete-region pos (point)))))))))
> ---
>> 	  (let (buffer-read-only) (delete-region pos (point))))))))
>
> I have tested this patch, and it solves the problem without causing any other 
> problems that I can detect.  I would appreciate if you would apply this patch 
> to the Emacs source.

Well, this change would prevent to raise an error. However, it seems
like there is a reason that POS is not valid anymore when the problem
happens. I believe we shall try to understand why this happens; maybe
there's something we need to fix as well, but only to fight the
symptoms.

Could you please try to provoke the error again? Prior this, pls set
tramp-verbose to 10, and send the Tramp debug buffer once it happened.

Best regards, Michael.

bug#23883: 24.4.51; tramp sometimes reports "Args out of range" error when resuming editing a remote file after a period of inactivity

[Michael Albinus]

MICHAEL J MCCLENNEN <mmcclenn@geology.wisc.edu> writes:

> Hi Michael,

Hi Michael,

I have added 23883@debbugs.gnu.org to Cc, again, for the archives. Pls
keep it.

I'll remove all sensitive data in my replies. In case you send me traces
containing sensitive data, you might drop off the debbugs.gnu.org
address.

> I have now done as you asked.  The attached file contains the contents
> of the tramp debug buffer generated while reproducing the bug I
> reported, with tramp-verbose set to 10.

> 08:06:26.507825 tramp-maybe-open-connection (6) # /bin/sh -i

Here Tramp did detect that the connection was broken, and it tries to
reconnect. First it opens a local shell

> 08:06:26.711851 tramp-wait-for-regexp (6) #
> :100644 100644 cdbd4deb831678472fbf42ae92a141e68cd938a2 0000000000000000000000000000000000000000 Mdoc/1.2/changelog_doc.tt\0diff --git a/doc/1.2/changelog_doc.tt b/doc/1.2/changelog_doc.tt
> index cdbd4de..d361f88 100644
> --- a/doc/1.2/changelog_doc.tt
> +++ b/doc/1.2/changelog_doc.tt

[...]

> -New output block B<countries> returns a list of the ISO-3166-1 country codes.
> +New output block B<C<countries>> returns a list of the ISO-3166-1 country codes.
>  The output includes the continent on which each country is located.
>
>  =back
> Write failed: Broken pipe
> Write failed: Broken pipe
> #$

Wow, ~540 lines garbage from the previous session. And we also see the
"Write failed: Broken pipe" message. In 'tramp-maybe-open-connection',
the current position in that buffer is preserved.

> 08:06:26.826303 tramp-send-command (6) # exec ssh   -o ControlPath=/var/folders/r4/fdb5rfz92fn2s0yg7yj897mr0000gs/T/tramp.897QcO.%r@%h:%p -o ControlMaster=auto -o ControlPersist=no -e none synonymy

Now Tramp tries to reconnect to the remote host synonymy. First, it did
a cleanup of the connection buffer. The preserved position is not valid
anymore.

> 08:06:28.950301 tramp-process-actions (6) #
> Last login: Sat Jul 23 22:10:45 2016 from ice.geology.wisc.edu
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> 	LC_ALL = "en_US.utf8",
> 	LANG = (unset)
>     are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> 	LC_ALL = "en_US.utf8",
> 	LANG = (unset)
>     are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").
> ====== synonymy ======
> ~ (501)$
> 08:06:28.950516 tramp-get-connection-property (7) # process-buffer nil
> 08:06:28.951307 tramp-process-actions (3) # Waiting for prompts from remote shell...failed

And here we are. Tramp claims a problem. The preserved position from
above is given to `tramp-process-actions' as argument, for further
handling. And because it is not valid anymore (remember, a large value),
it is out of the scope of the current buffer contents. Error.

> You asked about the remote file name: they are mainly Perl source
> code, but the same thing happens with a variety of file types.  I am
> hoping that you can figure out what method is causing the problem from
> the debug trace.  If not, can you tell me which ones should be
> checked?  I can trace through the code while it is working and try to
> figure out which method is being called.

Well, likely it is sufficient to reset the preserved position when the
buffer is cleaned up. Something like this:

--8<---------------cut here---------------start------------->8---
*** /home/albinus/src/tramp/lisp/tramp-sh.el.~608444a67599b876fa6c861338e6ebe4b8468d31~	2016-07-29 14:55:39.779097246 +0200
--- /home/albinus/src/tramp/lisp/tramp-sh.el	2016-07-29 14:51:04.279291259 +0200
***************
*** 4668,4674 ****
  	    (tramp-error vec 'file-error "Awake did fail")))
        (file-error
         (tramp-cleanup-connection vec t)
!        (setq p nil)))
  
      ;; New connection must be opened.
      (condition-case err
--- 4668,4675 ----
  	    (tramp-error vec 'file-error "Awake did fail")))
        (file-error
         (tramp-cleanup-connection vec t)
!        (setq pos (with-current-buffer (tramp-get-connection-buffer vec) (point))
! 	     p nil)))
  
      ;; New connection must be opened.
      (condition-case err
--8<---------------cut here---------------end--------------->8---

The line numbers in tramp-sh.el might differ (I've applied the patch in
Tramp's master repo), but you shall find where to apply.

Could you, pls, tell whether it works for you?

>   -- Michael

Best regards, Michael.

bug#23883: 24.4.51; tramp sometimes reports "Args out of range" error when resuming editing a remote file after a period of inactivity

[Michael Albinus]

Version: 25.2

Hi Michael,

> And here we are. Tramp claims a problem. The preserved position from
> above is given to `tramp-process-actions' as argument, for further
> handling. And because it is not valid anymore (remember, a large value),
> it is out of the scope of the current buffer contents. Error.
>
>> You asked about the remote file name: they are mainly Perl source
>> code, but the same thing happens with a variety of file types.  I am
>> hoping that you can figure out what method is causing the problem from
>> the debug trace.  If not, can you tell me which ones should be
>> checked?  I can trace through the code while it is working and try to
>> figure out which method is being called.
>
> Well, likely it is sufficient to reset the preserved position when the
> buffer is cleaned up. Something like this:
>
> *** /home/albinus/src/tramp/lisp/tramp-sh.el.~608444a67599b876fa6c861338e6ebe4b8468d31~	2016-07-29 14:55:39.779097246 +0200
> --- /home/albinus/src/tramp/lisp/tramp-sh.el	2016-07-29 14:51:04.279291259 +0200
> ***************
> *** 4668,4674 ****
>   	    (tramp-error vec 'file-error "Awake did fail")))
>         (file-error
>          (tramp-cleanup-connection vec t)
> !        (setq p nil)))
>
>       ;; New connection must be opened.
>       (condition-case err
> --- 4668,4675 ----
>   	    (tramp-error vec 'file-error "Awake did fail")))
>         (file-error
>          (tramp-cleanup-connection vec t)
> !        (setq pos (with-current-buffer (tramp-get-connection-buffer vec) (point))
> ! 	     p nil)))
>
>       ;; New connection must be opened.
>       (condition-case err
>
> The line numbers in tramp-sh.el might differ (I've applied the patch in
> Tramp's master repo), but you shall find where to apply.
>
> Could you, pls, tell whether it works for you?

I've committed a modified version of this patch to the Emacs and Tramp
repositories, closing this bug. If it doesn't work for you, feel free to
reopen.

>>   -- Michael

Best regards, Michael.