bug#26011: 26.0.50; tramp should respect large-file-warning-threshold

bug#26011: 26.0.50; tramp should respect large-file-warning-threshold

[Robert Marshall]

If from dired you attempt to copy a file to a tramp ssh dired (I have
dired-dwim-target set to t) and that file is very large, emacs will pause
for some time and eventually stop with:

tramp-file-name-handler: Memory exhausted--use C-x s then exit and
restart Emacs

If tramp is going to open the file and it is large I think it should
warn the user (respecting large-file-warning-threshold?) rather than
going ahead without confirmation and erroring with an alarming message!

Robert

In GNU Emacs 26.0.50 (build 4, x86_64-unknown-linux-gnu, GTK+ Version 2.24.25)
 of 2017-03-06 built on ct-lt-579
Repository revision: 0fae08d0072f74d97ca70b91a4d46d8d28a03952
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.7 (jessie)


Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS NOTIFY GNUTLS LIBXML2
FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK2 X11

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

bug#26011: 26.0.50; tramp should respect large-file-warning-threshold

[Michael Albinus]

Robert Marshall <robert.marshall@codethink.co.uk> writes:

Hi Robert,

> If from dired you attempt to copy a file to a tramp ssh dired (I have
> dired-dwim-target set to t) and that file is very large, emacs will pause
> for some time and eventually stop with:
>
> tramp-file-name-handler: Memory exhausted--use C-x s then exit and
> restart Emacs
>
> If tramp is going to open the file and it is large I think it should
> warn the user (respecting large-file-warning-threshold?) rather than
> going ahead without confirmation and erroring with an alarming message!

Well, this happens in case Tramp inserts the file into a temporary
buffer.  What about this patch:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
index 071ef79..8561962 100644
--- a/lisp/tramp-sh.el
+++ b/lisp/tramp-sh.el
@@ -2147,6 +2147,11 @@ file names."
 First arg OP is either `copy' or `rename' and indicates the operation.
 FILENAME is the source file, NEWNAME the target file.
 KEEP-DATE is non-nil if NEWNAME should have the same timestamp as FILENAME."
+  ;; Check, whether file is too large.  Emacs checks in `insert-file-1'
+  ;; and `find-file-noselect', but that's not called here.
+  (abort-if-file-too-large
+   (tramp-compat-file-attribute-size (file-attributes (file-truename filename)))
+   (symbol-name op) filename)
   ;; We must disable multibyte, because binary data shall not be
   ;; converted.  We don't want the target file to be compressed, so we
   ;; let-bind `jka-compr-inhibit' to t.  `epa-file-handler' shall not
--8<---------------cut here---------------end--------------->8---

> Robert

Best regards, Michael.

bug#26011: 26.0.50; tramp should respect large-file-warning-threshold

[Robert Marshall]

Michael Albinus <michael.albinus@gmx.de> writes:

> Robert Marshall <robert.marshall@codethink.co.uk> writes:
>
> Hi Robert,
>
>> If from dired you attempt to copy a file to a tramp ssh dired (I have
>> dired-dwim-target set to t) and that file is very large, emacs will pause
>> for some time and eventually stop with:
>>
>> tramp-file-name-handler: Memory exhausted--use C-x s then exit and
>> restart Emacs
>>
>> If tramp is going to open the file and it is large I think it should
>> warn the user (respecting large-file-warning-threshold?) rather than
>> going ahead without confirmation and erroring with an alarming message!
>
> Well, this happens in case Tramp inserts the file into a temporary
> buffer.  What about this patch:
>
> diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
> index 071ef79..8561962 100644
> --- a/lisp/tramp-sh.el
> +++ b/lisp/tramp-sh.el
> @@ -2147,6 +2147,11 @@ file names."
>  First arg OP is either `copy' or `rename' and indicates the operation.
>  FILENAME is the source file, NEWNAME the target file.
>  KEEP-DATE is non-nil if NEWNAME should have the same timestamp as FILENAME."
> +  ;; Check, whether file is too large.  Emacs checks in `insert-file-1'
> +  ;; and `find-file-noselect', but that's not called here.
> +  (abort-if-file-too-large
> +   (tramp-compat-file-attribute-size (file-attributes (file-truename filename)))
> +   (symbol-name op) filename)
>    ;; We must disable multibyte, because binary data shall not be
>    ;; converted.  We don't want the target file to be compressed, so we
>    ;; let-bind `jka-compr-inhibit' to t.  `epa-file-handler' shall not
>

Yes that's better thank you, you still get the transient error if I
continue (my file was 4Gig) but I guess it's too late to do anything
else by the time it gets there


Robert

bug#26011: 26.0.50; tramp should respect large-file-warning-threshold

[Michael Albinus]

Robert Marshall <robert.marshall@codethink.co.uk> writes:

Hi Robert,

> Yes that's better thank you, you still get the transient error if I
> continue (my file was 4Gig) but I guess it's too late to do anything
> else by the time it gets there

Once you confirm to copy/rename a huge file, its contents is loaded into
a temporary buffer. And yes, if it's too large, memory is exhausted.
There's nothing Tramp could know in advance, and prevent it.

If you handle regularly such huge files, Tramp's "scp" method is
superiour to "ssh".

I've pushed the patch to both Emacs and Tramp repositories, closing the bug.

> Robert

Best regards, Michael.