compress files patch for mac/make-package

compress files patch for mac/make-package

[bryan d. o'connor]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

here's a simple patch to mac/make-package that adds a
--compressed-dist option to compress info files and .el
files that have been byte-compiled.  it defaults to "no".

when used, it cuts down the install size by ~35M.

     ...bryan



[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 1987 bytes --]

Index: make-package
===================================================================
RCS file: /cvsroot/emacs/emacs/mac/make-package,v
retrieving revision 1.22
diff -u -u -r1.22 make-package
--- make-package	10 Aug 2005 15:25:51 -0000	1.22
+++ make-package	17 Oct 2005 18:53:38 -0000
@@ -55,6 +55,7 @@
 self_contained=no
 app_symlink=no
 full_dist=yes
+compressed_dist=no
 build_in_place=no
 keep_directory=no
 
@@ -94,6 +95,8 @@
        with_app=no ;;
     --without-full-dist | -without-full-dist | -no-full-dist | -no-full)
        full_dist=no ;;
+    --compressed-dist)
+       compressed_dist=yes ;;
     --self-contained | -self-contained | --with-self-contained-app | -sc)
        self_contained=yes ;;
     -app-symlink | --app-symlink | -symlink | --symlink | --asl)
@@ -138,6 +141,7 @@
                           --without-app.
       --without-full-dist Do not include all the .el files in the distribution.
                           This is discouraged except if disk space is critical.
+      --compressed-dist   Compress .el and info files in the distribution.
       --app-symlink       Have the Emacs.app executable be a symlink
                           to the install in [prefix]/bin/emacs and have
                           the emacs executable link to emacs-${version}
@@ -296,6 +300,14 @@
     echo "Removing unneeded .el files"
     sharedir=$installprefix/share/emacs/$version
     find $sharedir/lisp $sharedir/leim -name "*.elc" -print | sed 's|\(.*\)\.elc$|/bin/rm -f \1.el|' | /bin/sh -s
+fi
+
+if test "$compressed_dist" == "yes" -a "$full_dist" == "yes"; then
+    echo "Compressing .el files"
+    sharedir=$installprefix/share/emacs/$version
+    find $sharedir/lisp $sharedir/leim -name "*.elc" -print | sed 's|\(.*\)\.elc$|/usr/bin/gzip -9 \1.el|' | /bin/sh -s
+    echo "Compressing info files"
+    find $installprefix/info -type f ! -name dir -print | sed 's|\(.*\)$|/usr/bin/gzip -9 \1|' | /bin/sh -s
 fi
 
 if test "$with_app" == "yes"; then

[-- Attachment #3: Type: text/plain, Size: 1 bytes --]



[-- Attachment #4: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    here's a simple patch to mac/make-package that adds a
    --compressed-dist option to compress info files and .el
    files that have been byte-compiled.  it defaults to "no".

Do all Emacs operations handle the compressed .el files?
Maybe they do, now that Auto Compress mode is normally enabled,
but I would not be surprised if there are some bugs to fix.
It would be useful if some people would compress the .el files
and then try things for a couple of weeks and report.

Meanwhile, your patch is small enough to install without papers,
so I will ask someone to do that.

Re: compress files patch for mac/make-package

[Romain Francoise]

"Richard M. Stallman" <rms@gnu.org> writes:

> Do all Emacs operations handle the compressed .el files?

There were a handful of bugs with packages autoloading the .el file
explicitly instead of just specifying the package name, I fixed them a
few weeks ago.

The Emacs CVS packages in Debian use compressed .el files since
September 8th and apart from the aforementioned autoload bugs, no
problem has arisen.

-- 
Romain Francoise <romain@orebokech.com> | I used to think there is no
it's a miracle -- http://orebokech.com/ | future left at all.

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    There were a handful of bugs with packages autoloading the .el file
    explicitly instead of just specifying the package name, I fixed them a
    few weeks ago.

    The Emacs CVS packages in Debian use compressed .el files since
    September 8th and apart from the aforementioned autoload bugs, no
    problem has arisen.

That is good news.

Would you like to change `make install' to compress the
.el files?

Re: compress files patch for mac/make-package

[Romain Francoise]

"Richard M. Stallman" <rms@gnu.org> writes:

> Would you like to change `make install' to compress the
> .el files?

Ok, I'll do that in a few days' time in case someone objects to the
change.

-- 
Romain Francoise <romain@orebokech.com> | I used to think there is no
it's a miracle -- http://orebokech.com/ | future left at all.

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Date: Wed, 19 Oct 2005 09:16:25 +0200
> Cc: emacs-devel@gnu.org
> 
> "Richard M. Stallman" <rms@gnu.org> writes:
> 
> > Would you like to change `make install' to compress the
> > .el files?
> 
> Ok, I'll do that in a few days' time in case someone objects to the
> change.

I suggest that we be careful and conservative here.

First, there are some .el files that don't have a corresponding .elc
file; I think such .el file should not be compressed, since then Emacs
will not work without gzip being available.

Second, I think this compression should not be done on non-Posix
systems where gzip is not available.

And third, I think that we should not compress as part of tarring the
distribution.

Re: compress files patch for mac/make-package

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Romain Francoise <romain@orebokech.com>
>> Date: Wed, 19 Oct 2005 09:16:25 +0200
>> Cc: emacs-devel@gnu.org
>> 
>> "Richard M. Stallman" <rms@gnu.org> writes:
>> 
>> > Would you like to change `make install' to compress the
>> > .el files?
>> 
>> Ok, I'll do that in a few days' time in case someone objects to the
>> change.
>
> I suggest that we be careful and conservative here.
>
> First, there are some .el files that don't have a corresponding .elc
> file; I think such .el file should not be compressed, since then Emacs
> will not work without gzip being available.
>
> Second, I think this compression should not be done on non-Posix
> systems where gzip is not available.
>
> And third, I think that we should not compress as part of tarring the
> distribution.

Maybe add an

    make install-gzip

target?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    Maybe add an

	make install-gzip

That's not a good idea.  First of all, most people won't think to use
it.  Second, it departs from our standard.  The `install' target
should do the compression.

Re: compress files patch for mac/make-package

[Lennart Borgman]

Richard M. Stallman wrote:

>    Maybe add an
>
>	make install-gzip
>
>That's not a good idea.  First of all, most people won't think to use
>it.  Second, it departs from our standard.  The `install' target
>should do the compression.
>  
>
I do not know how to do it, but it is important to not gzip the files on 
platforms where gzip might not be available.

An alternative would perhaps be to ship a set of GNU utilities for Emacs 
on those platforms, but that is a rather big and complicated thing at 
present.

Re: compress files patch for mac/make-package

[Romain Francoise]

Lennart Borgman <lennart.borgman.073@student.lu.se> writes:

> An alternative would perhaps be to ship a set of GNU utilities for
> Emacs on those platforms, but that is a rather big and complicated
> thing at present.

The long term plan is to get rid of the gzip dependency for good and use
zlib to provide decompression primitives directly in Emacs.  (On systems
where zlib isn't available as a shared library, we'd provide the
possibility to include a static version in Emacs.)

There's an item in etc/TODO.

-- 
Romain Francoise <romain@orebokech.com> | The world is a fine place,
it's a miracle -- http://orebokech.com/ | and worth fighting for.
                                        | --Ernest Hemingway

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Date: Thu, 20 Oct 2005 09:20:06 +0200
> Cc: emacs-devel@gnu.org
> 
> The long term plan is to get rid of the gzip dependency for good and use
> zlib to provide decompression primitives directly in Emacs.

Then let's postpone compression of .el files until that goal is
reached.

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    > The long term plan is to get rid of the gzip dependency for good and use
    > zlib to provide decompression primitives directly in Emacs.

    Then let's postpone compression of .el files until that goal is
    reached.

I see no reason to postpone it.  I stand by my decision:
let's change `make install' to compress these files
during installation if gzip is available.

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: "Richard M. Stallman" <rms@gnu.org>
> Date: Thu, 20 Oct 2005 19:37:38 -0400
> Cc: romain@orebokech.com, emacs-devel@gnu.org
> 
>     > The long term plan is to get rid of the gzip dependency for good and use
>     > zlib to provide decompression primitives directly in Emacs.
> 
>     Then let's postpone compression of .el files until that goal is
>     reached.
> 
> I see no reason to postpone it.

The reasons to postpone it were spelled out by Kim: this is an
unnecessary and somewhat controversial change, which could conceivably
produce bugs, while the advantages it brings are minimal.  IMHO, we
shouldn't make it until after the release.

Re: compress files patch for mac/make-package

[Kim F. Storm]

Romain Francoise <romain@orebokech.com> writes:

> The long term plan is to get rid of the gzip dependency for good and use
> zlib to provide decompression primitives directly in Emacs.  (On systems
> where zlib isn't available as a shared library, we'd provide the
> possibility to include a static version in Emacs.)

So why not let this issue rest until we have native compression support
in Emacs (at which time harddisks are >10TB, so it really doesn't matter
anymore to compress .el files :-)

If Debian-folks want to compress .el files and it happens to work for
Debian, then it's no problem with me.  But that is NOT an issue we
have to deal with in Emacs for release 22.

I DEFINITELY think this is a non-issue which can EASILY wait until
AFTER the release  (or shall I spell all of it in capital letters)?

CAN PEOPLE PLEASE CONCENTRATE ON **FIXING** THE REMAINING ISSUES FOR
THE RELEASE, RATHER THAN PROPOSE NEW THINGS THAT WILL **DEFINITELY**
CAUSE PROBLEMS ON SOME PLATFORMS.  PLEASE!!!!  @#$%&@#$%&!!!!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: compress files patch for mac/make-package

[Romain Francoise]

storm@cua.dk (Kim F. Storm) writes:

> If Debian-folks want to compress .el files and it happens to work for
> Debian, then it's no problem with me.  But that is NOT an issue we
> have to deal with in Emacs for release 22.

You're barking up the wrong tree; I wasn't the one proposing to do that
in Emacs, Richard did.

The Debian folks (me) are perfectly happy the way things are, so please
leave them out of your bashing.

-- 
Romain Francoise <romain@orebokech.com> | I like the streets when
it's a miracle -- http://orebokech.com/ | they're empty, I can make the
                                        | rest up.

Re: compress files patch for mac/make-package

[Kim F. Storm]

Romain Francoise <romain@orebokech.com> writes:

> storm@cua.dk (Kim F. Storm) writes:
>
>> If Debian-folks want to compress .el files and it happens to work for
     ^
    e.g.  

>> Debian, then it's no problem with me.  But that is NOT an issue we
>> have to deal with in Emacs for release 22.
>
> You're barking up the wrong tree; I wasn't the one proposing to do that
> in Emacs, Richard did.
>
> The Debian folks (me) are perfectly happy the way things are, so please
> leave them out of your bashing.

Sorry for the confusion.

But I was NOT bashing Debian-folks (or anyone else who would like to
compress .el files as part of a distribution which includes Emacs).

I'm bashing those who suggested to put support for this _into_ Emacs
at this time (and if that was Richard, I'm bashing him :-).

In any case, this (non-)issue has taken far too much time already.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: storm@cua.dk (Kim F. Storm)
> Date: Thu, 20 Oct 2005 11:23:33 +0200
> Cc: Lennart Borgman <lennart.borgman.073@student.lu.se>, emacs-devel@gnu.org
> 
> I DEFINITELY think this is a non-issue which can EASILY wait until
> AFTER the release

Seconded.  There's no reason to risk unneeded breakage due to this
unnecessary change.

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> Cc: Romain Francoise <romain@orebokech.com>,  emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Wed, 19 Oct 2005 22:13:19 +0200
> 
> Maybe add an
> 
>     make install-gzip
> 
> target?

That'd be fine, I think.

Re: compress files patch for mac/make-package

[Romain Francoise]

Eli Zaretskii <eliz@gnu.org> writes:

> First, there are some .el files that don't have a corresponding .elc
> file; I think such .el file should not be compressed, since then Emacs
> will not work without gzip being available.

That's pretty obvious...

> Second, I think this compression should not be done on non-Posix
> systems where gzip is not available.

How would you compress files on systems where gzip isn't available
anyway?

> And third, I think that we should not compress as part of tarring the
> distribution.

make-dist and `make install' are completely independent.

(Besides, compressing the .el files inside the tarball would make the
compression less efficient and result in a bigger tarball.)

-- 
Romain Francoise <romain@orebokech.com> | I just thought I'd go out
it's a miracle -- http://orebokech.com/ | with a little bit more style.

Re: compress files patch for mac/make-package

[Kim F. Storm]

Romain Francoise <romain@orebokech.com> writes:

>> file; I think such .el file should not be compressed, since then Emacs
>> will not work without gzip being available.

> make-dist and `make install' are completely independent.
>
> (Besides, compressing the .el files inside the tarball would make the
> compression less efficient and result in a bigger tarball.)

How can you uncompress the tarball without gunzip ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> From: storm@cua.dk (Kim F. Storm)
> Date: Wed, 19 Oct 2005 23:37:06 +0200
> 
> How can you uncompress the tarball without gunzip ?

With DJTARNT (which can only handle .tar.gz files, not compressed .gz
files in general).

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 19 Oct 2005 22:16:01 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > First, there are some .el files that don't have a corresponding .elc
> > file; I think such .el file should not be compressed, since then Emacs
> > will not work without gzip being available.
> 
> That's pretty obvious...

But wasn't mentioned in the discussion; I wanted to be sure it's not
forgotten in the en-masse compression.

> > Second, I think this compression should not be done on non-Posix
> > systems where gzip is not available.
> 
> How would you compress files on systems where gzip isn't available
> anyway?

Sorry, I wasn't clear enough: I meant that the files should not be
compressed on systems where gzip isn't part of the default
installation, even if the specific machine where "make install" runs
happens to have gzip.

> > And third, I think that we should not compress as part of tarring the
> > distribution.
> 
> make-dist and `make install' are completely independent.

I know.  Still, I wanted to be sure this is understood and agreed
upon.

> (Besides, compressing the .el files inside the tarball would make the
> compression less efficient and result in a bigger tarball.)

That's not the important aspect; the important aspect is that
compressing .el files in the distribution prevent users of some
systems from having access to the sources of Emacs parts written in
Lisp, which I think goes against the GNU license.

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    Sorry, I wasn't clear enough: I meant that the files should not be
    compressed on systems where gzip isn't part of the default
    installation, even if the specific machine where "make install" runs
    happens to have gzip.

Why not?  It seems to me that if the given machine
has gzip there is no harm in using it.

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: "Richard M. Stallman" <rms@gnu.org>
> CC: romain@orebokech.com, emacs-devel@gnu.org
> Date: Thu, 20 Oct 2005 19:37:36 -0400
> 
>     Sorry, I wasn't clear enough: I meant that the files should not be
>     compressed on systems where gzip isn't part of the default
>     installation, even if the specific machine where "make install" runs
>     happens to have gzip.
> 
> Why not?  It seems to me that if the given machine
> has gzip there is no harm in using it.

Because, unlike on GNU and Unix systems, the installed tree is
frequently moved to other machines or archived to become a binary
distribution.

Re: compress files patch for mac/make-package

[Richard M. Stallman]

    Because, unlike on GNU and Unix systems, the installed tree is
    frequently moved to other machines or archived to become a binary
    distribution.

Ok, if Windows users generally think .el files shouldn't be compressed
on Windows, then let's not compress them on Windows.

Re: compress files patch for mac/make-package

[Romain Francoise]

"Richard M. Stallman" <rms@gnu.org> writes:

> Ok, if Windows users generally think .el files shouldn't be compressed
> on Windows, then let's not compress them on Windows.

Here's the patch to implement this plan.  I'll install it in a few days.

Index: Makefile.in
===================================================================
RCS file: /cvsroot/emacs/emacs/Makefile.in,v
retrieving revision 1.305
diff -c -r1.305 Makefile.in
*** Makefile.in	15 Sep 2005 12:36:49 -0000	1.305
--- Makefile.in	29 Oct 2005 10:29:34 -0000
***************
*** 223,228 ****
--- 223,231 ----
  # By default, we uphold the dignity of our programs.
  INSTALL_STRIP =
  
+ # We use gzip to compress installed .el files.
+ GZIP = @GZIP@
+ 
  # ============================= Targets ==============================
  
  # Program name transformation.
***************
*** 473,478 ****
--- 476,489 ----
  	   (cd lisp; tar -chf - *.el *.elc) \
  	     |(cd ${lispdir}; umask 022; tar -xvf - && cat > /dev/null) || exit 1; \
  	   (cd ${lispdir}; find . -exec chown $${LOGNAME:-$$USERNAME} {} ';') ; \
+ 	else true; fi
+ 	-unset CDPATH; \
+ 	if [ -x ${GZIP} ]; \
+ 	then \
+ 	   echo "Compressing *.el ..." ; \
+ 	   (cd ${lispdir}; for f in `find . -name "*.elc" -print`; do \
+ 		${GZIP} -9n `echo $$f|sed 's/.elc$$/.el/'` ; \
+ 	    done) \
  	else true; fi
  	-unset CDPATH; \
  	thisdir=`/bin/pwd`; \
Index: configure.in
===================================================================
RCS file: /cvsroot/emacs/emacs/configure.in,v
retrieving revision 1.399
diff -c -r1.399 configure.in
*** configure.in	25 Oct 2005 01:09:39 -0000	1.399
--- configure.in	29 Oct 2005 10:29:36 -0000
***************
*** 1272,1277 ****
--- 1272,1278 ----
  AC_PATH_PROG(INSTALL_INFO, install-info)
  AC_PATH_PROG(INSTALL_INFO, install-info,, /usr/sbin)
  AC_PATH_PROG(INSTALL_INFO, install-info,:, /sbin)
+ AC_PATH_PROG(GZIP, gzip)
  
  dnl Add our options to ac_link now, after it is set up.
  
Index: leim/Makefile.in
===================================================================
RCS file: /cvsroot/emacs/emacs/leim/Makefile.in,v
retrieving revision 1.70
diff -c -r1.70 Makefile.in
*** leim/Makefile.in	4 Jul 2005 15:27:32 -0000	1.70
--- leim/Makefile.in	29 Oct 2005 10:29:36 -0000
***************
*** 37,42 ****
--- 37,44 ----
  # Where to install LEIM files.
  INSTALLDIR=${datadir}/emacs/${version}/leim
  
+ GZIP = @GZIP@
+ 
  # On Xenix and the IBM RS6000, double-dot gets screwed up.
  dot = .
  
***************
*** 237,242 ****
--- 239,251 ----
  	  rm -f  ${INSTALLDIR}/.\#*       ${INSTALLDIR}/*/.\#* ; \
  	  rm -f  ${INSTALLDIR}/*~         ${INSTALLDIR}/*/*~ ; \
  	  rm -f  ${INSTALLDIR}/*.orig     ${INSTALLDIR}/*/*.orig ; \
+ 	else true; fi
+ 	-unset CDPATH; \
+ 	if [ -x ${GZIP} ]; \
+ 	then \
+ 	   (cd ${INSTALLDIR}; for f in `find . -name "*.elc" -print`; do \
+ 		${GZIP} -9n `echo $$f|sed 's/.elc$$/.el/'` ; \
+ 	    done) \
  	else true; fi
  	-chmod -R a+r ${INSTALLDIR}
  
-- 
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
                                        | ever free! --Bryan W. Procter

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Date: Sat, 29 Oct 2005 12:42:25 +0200
> 
> + 	if [ -x ${GZIP} ]; \

Is the -x option sufficiently portable to be used in our Makefiles?

Re: compress files patch for mac/make-package

[Romain Francoise]

Eli Zaretskii <eliz@gnu.org> writes:

> Is the -x option sufficiently portable to be used in our Makefiles?

I think so.

-- 
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
                                        | ever free! --Bryan W. Procter

Re: compress files patch for mac/make-package

[Eli Zaretskii]

> From: Romain Francoise <romain@orebokech.com>
> Date: Tue, 18 Oct 2005 08:00:35 +0200
> 
> The Emacs CVS packages in Debian use compressed .el files since
> September 8th and apart from the aforementioned autoload bugs, no
> problem has arisen.

IMHO, Sep 8 till today is a ridiculously short period, to make any
meaningful conclusions about possible problems.