Could distributed tarballs be compressed with xz instead of bzip2?

Could distributed tarballs be compressed with xz instead of bzip2?

[Ulrich Mueller]

It would be considerably smaller (sizes are for "bzip2 -9" and "xz -6"
which is the default):

   51056640 emacs-24.0.92.tar.gz
   41410560 emacs-24.0.92.tar.bz2
   34443264 emacs-24.0.92.tar.xz

Also xz can be uncompressed much faster:

   $ time bunzip2 emacs-24.0.92.tar.bz2
   real    0m11.840s
   user    0m10.393s
   sys     0m0.423s

   $ time unxz emacs-24.0.92.tar.xz
   real    0m4.251s
   user    0m3.833s
   sys     0m0.412s

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Chong Yidong]

Ulrich Mueller <ulm@gentoo.org> writes:

> It would be considerably smaller (sizes are for "bzip2 -9" and "xz -6"
> which is the default):
>
>    51056640 emacs-24.0.92.tar.gz
>    41410560 emacs-24.0.92.tar.bz2
>    34443264 emacs-24.0.92.tar.xz
>
> Also xz can be uncompressed much faster:

I'll do that for Emacs 24.4.  Thanks for the suggestion.

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Chong Yidong]

Chong Yidong <cyd@gnu.org> writes:

> I'll do that for Emacs 24.4.  Thanks for the suggestion.

Whoops, I mean 24.1.

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Harald Hanche-Olsen]

[Chong Yidong <cyd@gnu.org> (2012-02-01 02:51:09 UTC)]

> Ulrich Mueller <ulm@gentoo.org> writes:
> 
> > Also xz can be uncompressed much faster:
> 
> I'll do that for Emacs 24.4.  Thanks for the suggestion.

On the other hand, a cursory glance at the xz manual page indicates
that xz may have rather extreme memory requirements: Several gigabytes
in some cases! Is that something to worry about?

- Harald

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Harald Hanche-Olsen]

[Harald Hanche-Olsen <hanche@math.ntnu.no> (2012-02-01 09:31:13 UTC)]

> On the other hand, a cursory glance at the xz manual page indicates
> that xz may have rather extreme memory requirements: Several gigabytes
> in some cases! Is that something to worry about?

I looked a bit more carefully, and it seems there is little reason to
worry, at least if compression is done with the default setting (-6).
Even with compression level at -9, the memory requirement for
decompression is only 65 MiB, which is not a problem on modern
computers (though I think many emacs users are still on quite old
hardware). I guess the extreme memory requirements come from unusual
combinations of the many flags affecting the detailed operations of
the compression algorithm, and should be of little concern for regular
use.

- Harald

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Stefan Monnier]

>> > Also xz can be uncompressed much faster:
>> I'll do that for Emacs 24.4.  Thanks for the suggestion.
> On the other hand, a cursory glance at the xz manual page indicates
> that xz may have rather extreme memory requirements: Several gigabytes
> in some cases! Is that something to worry about?

Not really, especially as long as we also distribute a .gz version.


        Stefan

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Ulrich Mueller]

>>>>> On Wed, 01 Feb 2012, Harald Hanche-Olsen wrote:

>> On the other hand, a cursory glance at the xz manual page indicates
>> that xz may have rather extreme memory requirements: Several gigabytes
>> in some cases! Is that something to worry about?

> I looked a bit more carefully, and it seems there is little reason to
> worry, at least if compression is done with the default setting (-6).
> Even with compression level at -9, the memory requirement for
> decompression is only 65 MiB, which is not a problem on modern
> computers (though I think many emacs users are still on quite old
> hardware). I guess the extreme memory requirements come from unusual
> combinations of the many flags affecting the detailed operations of
> the compression algorithm, and should be of little concern for regular
> use.

I'd stay with the default setting of -6. The amount of memory required
for unpacking increases from 9 MB to 65 MB when going from -6 to -9.
Being somewhat conservative is probably better than risking any
trouble on embedded systems.

Re: Could distributed tarballs be compressed with xz instead of bzip2?

[Ulrich Mueller]

>>>>> On Wed, 01 Feb 2012, Chong Yidong wrote:

>> Ulrich Mueller <ulm@gentoo.org> writes:
>>> It would be considerably smaller (sizes are for "bzip2 -9" and
>>> "xz -6" which is the default):
>>>
>>>    51056640 emacs-24.0.92.tar.gz
>>>    41410560 emacs-24.0.92.tar.bz2
>>>    34443264 emacs-24.0.92.tar.xz
>>>
>>> Also xz can be uncompressed much faster:

>> I'll do that for Emacs 24.4.  Thanks for the suggestion.

> Whoops, I mean 24.1.

What was the reason that you didn't?

Ulrich