bug#40156: 26.3; append-to-file does not work on nonexistent file over ssh

bug#40156: 26.3; append-to-file does not work on nonexistent file over ssh

[Sean Devlin]

Hi folks,

The append-to-file function (and the underlying write-region in append
mode) fails over ssh (via tramp) when applied to a nonexistent file. The
same invocation creates the file transparently when invoked locally.

I think tramp should create the file as in the local usage, if
possible. If this is not possible, the discrepancy should be documented
somewhere. (Apologies if it is documented somewhere; I tried to find it
and could not.)

The following (edited) eshell transcript demonstrates the behavior
(though it works the same in Lisp):

Welcome to the Emacs shell

> ~ $ ls 
> ~ $ append-to-file "something" nil nonexistent.txt 
> ~ $ cat nonexistent.txt 
> something~ $ 
> ~ $ 
> ~ $ cd /ssh:remote: 
> /ssh:remote:/home/user $ ls 
> /ssh:remote:/home/user $ append-to-file "something" nil nonexistent.txt 
> File ‘/ssh:remote:/home/user/nonexistent.txt’ not found on remote host
> /ssh:remote:/home/user $ 
> /ssh:remote:/home/user $ touch nonexistent.txt 
> /ssh:remote:/home/user $ 
> /ssh:remote:/home/user $ append-to-file "something" nil nonexistent.txt 
> /ssh:remote:/home/user $ 
> /ssh:remote:/home/user $ cat nonexistent.txt 
> something/ssh:remote:/home/user $ 

In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109))
of 2019-09-02 built on builder10-14.porkrind.org
Windowing system distributor 'Apple', version 10.3.1975

Configured using:
'configure --with-ns '--enable-locallisppath=/Library/Application
Support/Emacs/${version}/site-lisp:/Library/Application
Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Eshell

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq dired
dired-loaddefs rfc822 mml mml-sec epa derived epg epg-config gnus-util
rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils pp files-x tramp-cache tramp-sh tramp
tramp-compat tramp-loaddefs trampver shell parse-time format-spec advice
auth-source cl-seq eieio byte-opt bytecomp byte-compile cconv eieio-core
cl-macs gv eieio-loaddefs password-cache pcmpl-unix em-unix em-term term
disp-table easymenu ehelp em-script em-prompt em-ls cl-loaddefs cl-lib
em-hist em-pred em-glob em-dirs em-cmpl em-basic em-banner em-alias
pcomplete comint ansi-color ring esh-var esh-io esh-cmd esh-opt esh-ext
esh-proc esh-arg esh-groups eshell esh-module esh-mode esh-util
elec-pair time-date tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/ns-win ns-win ucs-normalize mule-util
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 236075 13044)
(symbols 48 23407 1)
(miscs 40 55 96)
(strings 32 40652 1787)
(string-bytes 1 1228751)
(vectors 16 40619)
(vector-slots 8 799289 20336)
(floats 8 72 298)
(intervals 56 830 0)
(buffers 992 14))

bug#40156: 26.3; append-to-file does not work on nonexistent file over ssh

[Sean Devlin]

Hi folks,

I did a little digging, and I think the problem is in tramp-sh-handle-write-region around line 3240:

> 	  ;; If `append' is non-nil, we copy the file locally, and let
> 	  ;; the native `write-region' implementation do the job.
> 	  (when append (copy-file filename tmpfile 'ok))



It looks like copy-file fails if filename doesn’t exist. Adding a test for file existence (i.e. file-exists-p) to the condition seems to solve the problem, though admittedly I only tested for my own narrow use case.

Thanks!

bug#40156: 26.3; append-to-file does not work on nonexistent file over ssh

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

Version: 27.2

Sean Devlin <spd@toadstyle.org> writes:

> Hi folks,

Hi Sean,

> I did a little digging, and I think the problem is in
> tramp-sh-handle-write-region around line 3240:
>
>> 	  ;; If `append' is non-nil, we copy the file locally, and let
>> 	  ;; the native `write-region' implementation do the job.
>> 	  (when append (copy-file filename tmpfile 'ok))
>
> It looks like copy-file fails if filename doesn’t exist. Adding a test
> for file existence (i.e. file-exists-p) to the condition seems to
> solve the problem, though admittedly I only tested for my own narrow
> use case.

Thanks, and yes, your analysis is correct. I've committed a respective
patch to Emacs master. I've extended also tramp-tests.el
accordingly. The patch is appended.

Since Emacs 27.1 is already in pretest, I haven't applied it there. The
next Tramp release in GNU ELPA, 2.4.3.3, will contain the patch. It is
expected end of March.

> Thanks!

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1348 bytes --]

diff --git a/lisp/net/tramp-sh.el b/lisp/net/tramp-sh.el
index 7ac41d16c8..06dca31227 100644
--- a/lisp/net/tramp-sh.el
+++ b/lisp/net/tramp-sh.el
@@ -3306,7 +3306,8 @@ tramp-sh-handle-write-region

 	  ;; If `append' is non-nil, we copy the file locally, and let
 	  ;; the native `write-region' implementation do the job.
-	  (when append (copy-file filename tmpfile 'ok))
+	  (when (and append (file-exists-p filename))
+	    (copy-file filename tmpfile 'ok))

 	  ;; We say `no-message' here because we don't want the
 	  ;; visited file modtime data to be clobbered from the temp
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index e220420d8c..e6c6b28c58 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -2356,7 +2356,14 @@ tramp-test10-write-region
 		(write-region nil nil tmp-name 3))
 	      (with-temp-buffer
 		(insert-file-contents tmp-name)
-		(should (string-equal (buffer-string) "foobaz"))))
+		(should (string-equal (buffer-string) "foobaz")))
+	      (delete-file tmp-name)
+	      (with-temp-buffer
+		(insert "foo")
+		(write-region nil nil tmp-name 'append))
+	      (with-temp-buffer
+		(insert-file-contents tmp-name)
+		(should (string-equal (buffer-string) "foo"))))

 	    ;; Write string.
 	    (write-region "foo" nil tmp-name)