bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

[Dan McCarthy]

[-- Attachment #1: Type: text/plain, Size: 4680 bytes --]

On a remote machine:

dan@solstice:/tmp$ touch a b c
dan@solstice:/tmp$ tar cf test.tar a b c
dan@solstice:/tmp$ bzip2 -z test.tar
dan@solstice:/tmp$ file test.tar.bz2
test.tar.bz2: bzip2 compressed data, block size = 900k

If you visit that remote file and save it, the result will have the .bz2
extension but won't actually be compressed:

dan@solstice:/tmp$ file test.tar.bz2
test.tar.bz2: POSIX tar archive (GNU)

This doesn't happen with compressed remote files which aren't tar archives.

In GNU Emacs 28.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.31,
cairo version 1.16.0)
 of 2022-04-12 built on october.example.org
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Slackware 15.0 x86_64

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  hexl-follow-ascii: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/.emacs.d/lisp/android-mode hides
/home/blakestone/.emacs.d/elpa/android-mode-20190903.811/android-mode
~/.emacs.d/lisp/connection hides
/home/blakestone/.emacs.d/elpa/connection-20191111.446/connection
~/.emacs.d/lisp/link hides
/home/blakestone/.emacs.d/elpa/link-20191111.446/link
/home/blakestone/.emacs.d/elpa/transient-20220425.1314/transient hides
/usr/local/share/emacs/28.1/lisp/transient
/home/blakestone/.emacs.d/elpa/dictionary-20201001.1727/dictionary hides
/usr/local/share/emacs/28.1/lisp/net/dictionary
~/.emacs.d/lisp/greek hides /usr/local/share/emacs/28.1/lisp/language/greek
~/.emacs.d/lisp/longlines hides
/usr/local/share/emacs/28.1/lisp/obsolete/longlines

Features:
(shadow sort mail-extr ffap emacsbug message rmc puny rfc822 mml mml-sec
epa epg rfc6068 epg-config gnus-util rmail rmail-loaddefs mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-hg
vc-git diff-mode easy-mmode vc-bzr vc-dispatcher tar-mode arc-mode
archive-mode jka-compr pcmpl-unix pcmpl-gnu with-editor compat dired-aux
dired dired-loaddefs tramp-cache tramp-sh executable misearch
multi-isearch edmacro kmacro cua-base cus-load acme-search quack
thingatpt compile text-property-search cmuscheme scheme advice tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
pcomplete comint ring parse-time iso8601 time-date ls-lisp format-spec
ansi-color c-xref rx session vanc-init bindat vanc-mode hexl ecf-mode
derived cl-extra help-mode dcm-general server cl info package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 166897 25031)
 (symbols 48 15286 4)
 (strings 32 52764 4059)
 (string-bytes 1 1727857)
 (vectors 16 31379)
 (vector-slots 8 983720 94988)
 (floats 8 84 70)
 (intervals 56 9204 1194)
 (buffers 992 22))

[-- Attachment #2: Type: text/html, Size: 5103 bytes --]

bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

[Lars Ingebrigtsen]

Dan McCarthy <daniel.c.mccarthy@gmail.com> writes:

> dan@solstice:/tmp$ touch a b c
> dan@solstice:/tmp$ tar cf test.tar a b c
> dan@solstice:/tmp$ bzip2 -z test.tar 
> dan@solstice:/tmp$ file test.tar.bz2 
> test.tar.bz2: bzip2 compressed data, block size = 900k
>
> If you visit that remote file and save it, the result will have the .bz2
> extension but won't actually be compressed:
>
> dan@solstice:/tmp$ file test.tar.bz2 
> test.tar.bz2: POSIX tar archive (GNU)
>
> This doesn't happen with compressed remote files which aren't tar archives.

I can reproduce this problem with Emacs 29, too.  (And if I edit the
file locally instead of via tramp, the tar file is saved compressed, so
it seems to be tramp-related.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

Hi,

>> dan@solstice:/tmp$ touch a b c
>> dan@solstice:/tmp$ tar cf test.tar a b c
>> dan@solstice:/tmp$ bzip2 -z test.tar
>> dan@solstice:/tmp$ file test.tar.bz2
>> test.tar.bz2: bzip2 compressed data, block size = 900k
>>
>> If you visit that remote file and save it, the result will have the .bz2
>> extension but won't actually be compressed:
>>
>> dan@solstice:/tmp$ file test.tar.bz2
>> test.tar.bz2: POSIX tar archive (GNU)
>>
>> This doesn't happen with compressed remote files which aren't tar archives.
>
> I can reproduce this problem with Emacs 29, too.  (And if I edit the
> file locally instead of via tramp, the tar file is saved compressed, so
> it seems to be tramp-related.)

Well, the problem seems to exist since Emacs 27, likely due to some
subtle changes in basic-save-buffer-1 or basic-save-buffer-2. I haven't
poked further there, because it looks like saving the buffer with this
constellation happened twice in Emacs 26, which looks not optimal.

The problem is an exotic feature of write-region, which first looks for
a file name handler in FILENAME, and if there isn't one, it checks
VISIT.

write-region of the compressed remote tar file in the scenario above is
called like

--8<---------------cut here---------------start------------->8---
  ...
  tramp-file-name-handler(write-region nil nil "/ssh:detlef:/tmp/tmpfSxlA9" nil "/ssh:detlef:/tmp/test.tar.bz2" "/ssh:detlef:/tmp/test.tar.bz2" nil)
  write-region(nil nil "/ssh:detlef:/tmp/tmpfSxlA9" nil "/ssh:detlef:/tmp/test.tar.bz2" "/ssh:detlef:/tmp/test.tar.bz2")
  basic-save-buffer-2()
  basic-save-buffer-1()
  basic-save-buffer(t)
  save-buffer(1)
  funcall-interactively(save-buffer 1)
  call-interactively(save-buffer nil nil)
  command-execute(save-buffer)
--8<---------------cut here---------------end--------------->8---

That is, a remote temporary file ("/ssh:detlef:/tmp/tmpfSxlA9") is
written, and the magic extension of the remote VISIT argument
"/ssh:detlef:/tmp/test.tar.bz2" is ignored.

When FILENAME is a local temporary file, the handler for VISIT
(jka-compr-handler) is triggered. That's the second test Lars has applied.

I've pushed a fix to master, which shall fix this. Dan, could you pls
check?

In parallel I'll try to write a test case for this problem. But that's
unrelated to closing this bug.

Best regards, Michael.

bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

[Dan McCarthy]

[-- Attachment #1: Type: text/plain, Size: 2611 bytes --]

It works perfectly. Thanks a lot!

On Tue, May 3, 2022 at 8:36 AM Michael Albinus <michael.albinus@gmx.de>
wrote:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> Hi,
>
> >> dan@solstice:/tmp$ touch a b c
> >> dan@solstice:/tmp$ tar cf test.tar a b c
> >> dan@solstice:/tmp$ bzip2 -z test.tar
> >> dan@solstice:/tmp$ file test.tar.bz2
> >> test.tar.bz2: bzip2 compressed data, block size = 900k
> >>
> >> If you visit that remote file and save it, the result will have the .bz2
> >> extension but won't actually be compressed:
> >>
> >> dan@solstice:/tmp$ file test.tar.bz2
> >> test.tar.bz2: POSIX tar archive (GNU)
> >>
> >> This doesn't happen with compressed remote files which aren't tar
> archives.
> >
> > I can reproduce this problem with Emacs 29, too.  (And if I edit the
> > file locally instead of via tramp, the tar file is saved compressed, so
> > it seems to be tramp-related.)
>
> Well, the problem seems to exist since Emacs 27, likely due to some
> subtle changes in basic-save-buffer-1 or basic-save-buffer-2. I haven't
> poked further there, because it looks like saving the buffer with this
> constellation happened twice in Emacs 26, which looks not optimal.
>
> The problem is an exotic feature of write-region, which first looks for
> a file name handler in FILENAME, and if there isn't one, it checks
> VISIT.
>
> write-region of the compressed remote tar file in the scenario above is
> called like
>
> --8<---------------cut here---------------start------------->8---
>   ...
>   tramp-file-name-handler(write-region nil nil
> "/ssh:detlef:/tmp/tmpfSxlA9" nil "/ssh:detlef:/tmp/test.tar.bz2"
> "/ssh:detlef:/tmp/test.tar.bz2" nil)
>   write-region(nil nil "/ssh:detlef:/tmp/tmpfSxlA9" nil
> "/ssh:detlef:/tmp/test.tar.bz2" "/ssh:detlef:/tmp/test.tar.bz2")
>   basic-save-buffer-2()
>   basic-save-buffer-1()
>   basic-save-buffer(t)
>   save-buffer(1)
>   funcall-interactively(save-buffer 1)
>   call-interactively(save-buffer nil nil)
>   command-execute(save-buffer)
> --8<---------------cut here---------------end--------------->8---
>
> That is, a remote temporary file ("/ssh:detlef:/tmp/tmpfSxlA9") is
> written, and the magic extension of the remote VISIT argument
> "/ssh:detlef:/tmp/test.tar.bz2" is ignored.
>
> When FILENAME is a local temporary file, the handler for VISIT
> (jka-compr-handler) is triggered. That's the second test Lars has applied.
>
> I've pushed a fix to master, which shall fix this. Dan, could you pls
> check?
>
> In parallel I'll try to write a test case for this problem. But that's
> unrelated to closing this bug.
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 3368 bytes --]

bug#55166: 28.1; tar-mode doesn't re-compress when saving remote files

[Michael Albinus]

Version: 29.1

Dan McCarthy <daniel.c.mccarthy@gmail.com> writes:

Hi Dan,

> It works perfectly. Thanks a lot!

Thanks for the feedback, I'm closing the bug.

Later this year, I'll release Tramp 2.6 on GNU ELPA. This will contain
the patch, and it will work also with older Emacsen.

Best regards, Michael.