Why aren't ELPA packages compressed?

Why aren't ELPA packages compressed?

[Ulrich Mueller]

Some packages in ELPA have sizes around 5 MB, so I wonder why they can
only be downloaded as uncompressed tarballs?

Everything that is necessary to handle compressed files should already
be there in Emacs.

Ulrich

Re: Why aren't ELPA packages compressed?

[Ivan Shmakov]

>>>>> Ulrich Mueller <ulm@gentoo.org> writes:

 > Some packages in ELPA have sizes around 5 MB, so I wonder why they
 > can only be downloaded as uncompressed tarballs?

 > Everything that is necessary to handle compressed files should
 > already be there in Emacs.

	I guess that the reason’s similar to the HTTPS case: the Emacs
	compression support may be dependent on the presence of specific
	libraries (as in: Zlib) on the target system.

	That said, it should be perfectly possible to configure the
	Apache HTTP server at elpa.gnu.org for transparent compression
	support; that way, the data would be sent compressed by Apache
	whenever the client claims to support compression.  (By the
	means of the ‘Accept-Encoding: gzip’ HTTP request header.
	A glance at url-http-create-request seems to suggest it /does/
	implement support for such requests.)

-- 
FSF associate member #7257  http://am-1.org/~ivan/      … 3013 B6A0 230E 334A

Re: Why aren't ELPA packages compressed?

[Stefan Monnier]

> 	I guess that the reason’s similar to the HTTPS case: the Emacs
> 	compression support may be dependent on the presence of specific
> 	libraries (as in: Zlib) on the target system.

Exactly (actually, linking with zlib is not needed since we can call
gunzip as an external program, but that still means that gzip needs to
be installed).

> 	That said, it should be perfectly possible to configure the
> 	Apache HTTP server at elpa.gnu.org for transparent compression
> 	support;

Should yes.  See my latest attempt at

http://stackoverflow.com/questions/26895894/get-apache-to-auto-decompress-a-file-but-only-if-needed


        Stefan

Re: Why aren't ELPA packages compressed?

[Ted Zlatanov]

On Mon, 04 May 2015 16:56:50 +0000 Ivan Shmakov <ivan@siamics.net> wrote: 

IS> 	That said, it should be perfectly possible to configure the
IS> 	Apache HTTP server at elpa.gnu.org for transparent compression
IS> 	support; that way, the data would be sent compressed by Apache
IS> 	whenever the client claims to support compression.  (By the
IS> 	means of the ‘Accept-Encoding: gzip’ HTTP request header.
IS> 	A glance at url-http-create-request seems to suggest it /does/
IS> 	implement support for such requests.)

That still stores the uncompressed tarballs locally on the client.  It
also requires the server to do more work--it's cheaper to compress just once.

Ted

Re: Why aren't ELPA packages compressed?

[Tom Tromey]

>>>>> "Ulrich" == Ulrich Mueller <ulm@gentoo.org> writes:

Ulrich> Some packages in ELPA have sizes around 5 MB, so I wonder why they can
Ulrich> only be downloaded as uncompressed tarballs?

Ulrich> Everything that is necessary to handle compressed files should already
Ulrich> be there in Emacs.

When I wrote package.el, one goal was that installing a package would
not require any external tool.  So, e.g., that is why I chose tar
archives and not some other format -- tar-mode.el can unpack them
without a helper.

From NEWS I see that zlib was added in 24.  So, that's the reason -- at
the time it wasn't available.  It sounds like it is optional, though,
which seems like a problem for users.

Tom

Re: Why aren't ELPA packages compressed?

[Ulrich Mueller]

>>>>> On Tue, 05 May 2015, Tom Tromey wrote:

> When I wrote package.el, one goal was that installing a package
> would not require any external tool. So, e.g., that is why I chose
> tar archives and not some other format -- tar-mode.el can unpack
> them without a helper.

>> From NEWS I see that zlib was added in 24. So, that's the reason --
> at the time it wasn't available. It sounds like it is optional,
> though, which seems like a problem for users.

As was previously remarked, gzip could be used if Emacs isn't linked
against zlib. And I wonder if there are still any systems nowadays
where neither gzip nor zlib isn't readily available.

Ulrich

Re: Why aren't ELPA packages compressed?

[Ted Zlatanov]

On Tue, 5 May 2015 15:31:16 +0200 Ulrich Mueller <ulm@gentoo.org> wrote: 

>>>>>> On Tue, 05 May 2015, Tom Tromey wrote:
>> When I wrote package.el, one goal was that installing a package
>> would not require any external tool. So, e.g., that is why I chose
>> tar archives and not some other format -- tar-mode.el can unpack
>> them without a helper.

>>> From NEWS I see that zlib was added in 24. So, that's the reason --
>> at the time it wasn't available. It sounds like it is optional,
>> though, which seems like a problem for users.

UM> As was previously remarked, gzip could be used if Emacs isn't linked
UM> against zlib. And I wonder if there are still any systems nowadays
UM> where neither gzip nor zlib isn't readily available.

Both compressed and uncompressed tarballs could be available, each with
a separate signature. Then package.el could just get the smallest one it
can handle locally (so stuff like bz2 could also work). The web server
would require no more than the original storage for each compression
method but the downloads would be far smaller and faster. And it's
backwards compatible.

Ted

Re: Why aren't ELPA packages compressed?

[Eli Zaretskii]

> Date: Tue, 5 May 2015 15:31:16 +0200
> From: Ulrich Mueller <ulm@gentoo.org>
> Cc: emacs-devel@gnu.org
> 
> I wonder if there are still any systems nowadays where neither gzip
> nor zlib isn't readily available.

Yes (although not systems you are likely to be interested in).