[BUG] Package installer corrupts image & zip files

[BUG] Package installer corrupts image & zip files

[Jambunathan K]

Would it be possible to address the following bug before my
OpenDocumentText exporter lands in elpa? I will also file a formal bug
report in few hours.

Summary:

I distribute an image file (png) and a OpenOffice extension (oxt) file
as part of the bundle. These files are getting mangled by the package
installer.

Steps for reproduction:

1. Update package-archives to point to my archive

--8<---------------cut here---------------start------------->8---
(setq package-archives
      '(("gnu" . "http://elpa.gnu.org/packages/")
	("org-odt" . "http://repo.or.cz/w/org-mode/org-jambu.git/blob_plain/HEAD:/packages/")))
--8<---------------cut here---------------end--------------->8---

2. Install the org-odt package.

3. Visit the following files:
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/files/org-mode-unicorn.png
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/BasicODConverter-0.8.0.oxt

   Note that the above two files are corrupt. (oxt file is a zip archive)

4. Visit these files in the downloaded tar. Note that they are sane.

Jambunathan K.

-- 

bug#8870: Package installer corrupts image & zip files

[Jambunathan K]

[ Use M-x report-emacs-bug rather than a [BUG] tag in the subject line.  ]

Would it be possible to address the following bug before my
OpenDocumentText exporter lands in elpa? I will also file a formal bug
report in few hours.

Summary:

I distribute an image file (png) and a OpenOffice extension (oxt) file
as part of the bundle. These files are getting mangled by the package
installer.

Steps for reproduction:

1. Update package-archives to point to my archive

--8<---------------cut here---------------start------------->8---
(setq package-archives
      '(("gnu" . "http://elpa.gnu.org/packages/")
	("org-odt" . "http://repo.or.cz/w/org-mode/org-jambu.git/blob_plain/HEAD:/packages/")))
--8<---------------cut here---------------end--------------->8---

2. Install the org-odt package.

3. Visit the following files:
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/files/org-mode-unicorn.png
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/BasicODConverter-0.8.0.oxt

   Note that the above two files are corrupt. (oxt file is a zip archive)

4. Visit these files in the downloaded tar. Note that they are sane.

Jambunathan K.

bug#8870: Additional Info

[Jambunathan K]

Many Thanks to whoever filed this bug report on behalf of me.

,---- Emacs Version
| GNU Emacs 24.0.50.1 (i386-mingw-nt5.1.2600) of 2011-06-13 on 3249CTO
`----

,---- tar Version
| tar (GNU tar) 1.23
| Packaged by Cygwin (1.23-1)
| Copyright (C) 2010 Free Software Foundation, Inc.
| License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
| 
| Written by John Gilmore and Jay Fenlason.
`----

bug#8870: Package installer corrupts image & zip files

[Chong Yidong]

Jambunathan K <kjambunathan@gmail.com> writes:

> I distribute an image file (png) and a OpenOffice extension (oxt) file
> as part of the bundle. These files are getting mangled by the package
> installer.

I can't reproduce this, either with the org-odt package at your URL, nor
with a simple multi-file package containing a gif and a png file.

It's possible this is some kind of Windows problem related to tar mode.
Can you trying visiting the tar file in Emacs, and see if the image is
untarred correctly?

C-x C-f org-odt-XXXX.tar
C-s .png RET
RET

Is the image displayed correctly here?

bug#8870: Package installer corrupts image & zip files

[Jambunathan K]

Chong Yidong <cyd@stupidchicken.com> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> I distribute an image file (png) and a OpenOffice extension (oxt) file
>> as part of the bundle. These files are getting mangled by the package
>> installer.
>
> I can't reproduce this, either with the org-odt package at your URL, nor
> with a simple multi-file package containing a gif and a png file.
>
> It's possible this is some kind of Windows problem related to tar mode.
> Can you trying visiting the tar file in Emacs, and see if the image is
> untarred correctly?
>
> C-x C-f org-odt-XXXX.tar
> C-s .png RET
> RET
>
> Is the image displayed correctly here?

The above sequence (which uses tar-extract) DTRT.


1 Hint for the final fix 
-------------------------

tar-untar-buffer needs to use the right coding system for
extraction. 

For example, the following "stupid change" in `tar-untar-buffer' fixes
the original bug.


  (let ((coding-system-for-write 'no-conversion)) 
    (write-region start end name))


2 Observation 
--------------
1. Visiting the png file embedded within the original tar file is OK
2. Visiting the png file in the
   ~/.emacs.d/elpa/org-odt-YYYYMMDD/contrib/odt/files/org-mode-unicorn.png
   is NOT OK.


3 Analysis 
-----------

3.1 Behaviour of tar-untar-buffer 
==================================

1. `package-install-file' uses `tar-untar-buffer' for extraction.
2. `tar-untar-buffer' invokes `write-region' and an immediate query
   for `last-coding-system-used' reports "raw-text-dos" for the png
   file.

3.2 Behaviour of tar-extract 
=============================

1. `tar-extract' uses `decode-coding-region' with coding set to
   `no-conversion' for the png file. The coding is decided by a prior
   call to `set-auto-coding'.

   I would assume that a value of `no-conversion' is picked from the
   following entry in `auto-coding-alist'.


  ("\\.\\(jpe?g\\|png\\|gif\\|tiff?\\|p[bpgn]m\\)\\'" . no-conversion)


Jambunathan K.

bug#8870: Package installer corrupts image & zip files

[Chong Yidong]

Jambunathan K <kjambunathan@gmail.com> writes:

> tar-untar-buffer needs to use the right coding system for
> extraction.

Could you verify that a standalone `M-x tar-untar-buffer' (called
outside of package.el) also exhibits the bug?

If so, I think setting coding-system-for-write to no-conversion is
probably the correct fix, indeed.

bug#8870: Package installer corrupts image & zip files

[Jambunathan K]

Chong Yidong <cyd@stupidchicken.com> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> tar-untar-buffer needs to use the right coding system for
>> extraction.
>
> Could you verify that a standalone `M-x tar-untar-buffer' (called
> outside of package.el) also exhibits the bug?

Yes, the standalone `M-x tar-untar-buffer' ALSO corrupts the zip and png
files.

> If so, I think setting coding-system-for-write to no-conversion is
> probably the correct fix, indeed.

Could you please apply this fix then?

bug#8870: Package installer corrupts image & zip files

[Chong Yidong]

Jambunathan K <kjambunathan@gmail.com> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> Jambunathan K <kjambunathan@gmail.com> writes:
>>
>>> tar-untar-buffer needs to use the right coding system for
>>> extraction.
>>
>> Could you verify that a standalone `M-x tar-untar-buffer' (called
>> outside of package.el) also exhibits the bug?
>
> Yes, the standalone `M-x tar-untar-buffer' ALSO corrupts the zip and png
> files.

Thanks for checking.  I've committed the change.